Over the last few weeks, Daniel Elstner, Johannes Schmid and I have updated our Clutter tutorial to the latest Clutter 0.9 API and I think we are now finished. I think Clutter 1.0 will appear soon, without major API changes needed in the tutorial.
There’s a html version and a PDF.
libglom has been mostly theoretical until now. But I just removed all the UI code and the GTK+ dependency so it can now be used realistically by applications other than Glom itself. This small libglom document-parsing C++ example shows how easily you can get the basic details of the Glom system. This is only in svn for now.
And here is a small libglom test case in Java. (Yes, that’s not a well structured junit test case – I’m just throwing code in there as IÂ create the API.)
I’m using SWIG to create a Java API for the libglom C++ API. SWIG seems very capable and handles real C++, but the syntax is alien to me so I’d be very glad if a SWIG expert chose to do that job for me.
Over the last couple of weeks I’ve been relearning Java and JSP and learning about Google’s GWT . I learnt Java when it was first released but I last used it around six years ago and I never used it seriously in a large project.
I’m now up to speed on new stuff in the Java language, such as annotation, generics, and enumerated types, some of which I really missed not having before. A newer edition of Bruce Eckel’s Thinking In Java was great for that. I’ve always liked how he includes comparisons to other programming languages (C++, Python, C#) and is unafraid to criticize the API that he’s demonstrating, though I wonder how interesting that is to other people.
For JSP I tried a different style of book: Head First Servlets and JSP. It’s full of clip art and silly captioned kung-fu photos and cartoons but that really did help me to keep reading. I’m only slightly ashamed.
No, I’m not going to do less C and C++. I love C++. It’s just not what’s used for web development.
Of course any web site these days must use AJAX to provide a more interactive page, exchanging data with the server and updating the page without doing a whole page refresh. I’d like to avoid writing or maintaining Javascript if at all possible, so Google Web Toolkit seems like a good choice. Bizarrely, it generates javascript from Java, and it even has (imperfect) Javascript implementations of some of the standard Java library. It’s weird but it seems to work. I believe it’s used for gmail and Google Maps.
I’m using the GWT in Action book, though the online GWT documentation seems fine, and the self-hosting runnable examples are really helpful.
This also included learning about the maven and ant build/configuration tools. maven feels roughly equivalent to autotools plus pkg-config, with ant being roughly equivalent to make. As far as I can tell maven is the in thing.
So far maven is working out well for me, though all the XML to use the necessary plugins feels like voodoo and it has taken me days to get some plugins working, for instance to build WAR files to deploy to tomcat, and to use JNI. The incredibly obscure error messages don’t help and when it does what I want I am thankful but never quite sure how it knew what I wanted.
I also feel weird about using plugins from sites that I’ve never heard of. Without being familiar with the history and community, I have no way to know what is the official best plugin to solve particular problems.
This is an opportunity to give Eclipse another chance. After all, it’s meant to be wonderful for Java development, right?
But I seem to need various extras to work with common things like JSP and maven, which forces me to use the hateful “Software Updated and Add-ons” feature. The problems with this are numerous:
I managed to install the WebTools add-on in one of my Eclipse installations. But it has a rather hacky MSWindows-centric wizard, which assumes that all of Tomcat is installed in one directory and won’t let me get further until that is true. Of course on Linux distros things are split up between /usr/lib, /usr/etc, etc. Some helpful Fedora people seem enthuasiastic about fixing it.
Anyway I generally dislike the idea that I must use a particular project wizard for things that are orthogonal. I shouldn’t be forced to choose between my programming language, my build system, or some of my dependencies. They should be independently interchangable. I suspect that I’d be happy with Eclipse just as an editor, using the command-line with my hand-edited build files, but I never seem to get that far. Eclipse should make it more obvious how to do that because it’s currently a rather hidden feature in some of the new-project wizards.
Note that I’ve tried standard Eclipse in Ubuntu Intrepid (Even Jaunty only has Eclipse 3.2), Eclipse 3.4 in Jaunty (by downloading it and running it from a directory, which works surprisingly well), and “Fedora Eclipse” 3.4Â in Fedora 10 in a vmware image.
I’m not ready to give up just yet. I’ll take another run at using Eclipse just as a simple editor, trying not to use any special add-ons. But my patience surprised me.
This isn’t just to refresh these skills and prove to myself that I can still learn. I plan to use this to create a web UI for Glom. My plan is roughly:
Although AJAX allows us to do more than the old submit-form/get-new-page UI of CGI, I am still not looking forward to dealing with the increased use of async coding compared to desktop coding, batching information up to reduce client/server communication, and being restricted to aggregations of serializable/copy-by-value primitive types when doing that.
My initial inept attempts at all this are in my online_glom repository on github. There’s hardly anything to see but there’s probably already plenty to correct.
I believe that a document-based GNOME application should do these things, even I start two instances from the panel’s Applications menu rather than starting the second instance from the application’s File menu. (For now, let’s ignore single-instance tabbed applications such as gedit.)
Firefox does this. Gnumeric and Abiword don’t, though it works for a second instance that I open via File/New. They probably start extra File/New instances in the same process and keep track of them via static data.
They could solve the problem by enforcing a single-instance via libunique, so the one process knows about all “instances”. Yesterday I easily added libunique support in my experimental gtkmm document/view framework, Bakery, because I want to fix this problem in Glom.
However, having all application instances in one process gets annoying fast. Applications do hang or crash sometimes and it’s nice if that does not affect every open instance of the application. People probably don’t notice this because they rarely use File/New or File/Open. They probably do notice that File/Quit and the Documents menu don’t work as they’d expect.
So I guess we need some RPC to allow each application instance’s process to talk to all the others, maybe using a D-Bus interface. But that would quickly get complex and buggy and would provide multiple opportunities for code duplication and inconsistent behavior. I guess some shared API is needed. Has anyone else thought about this or even solved the problem already? Is there even one application that does it properly already?
Maybe it would be simplest for application instances to register themselves with a desktop-wide D-Bus service which could also be asked for a list of open documents.
Back when I used Apple MacOS (versions 5 through 8) this was normal and worked, probably partly because the Finder maintained a list of open documents and didn’t even try to open second instances of already-open documents. The MacOS toolbox API was incredibly primitive back then so I guess that applications solved this individually. But it was a simpler time.
I wonder if MacOS X has actual API to help applications solve this problem.
This is our small collection of books.
The gtkmm web site also has a list of C++ books we recommend. David and Michael really liked Accelerated C++, confirming the good things I’ve heard about it.
I’d like to buy some Beagle boards for the trainees to play with. Does anyone know of some good general books about deploying Linux to arbitrary embedded hardware? I’d also like them to have some book about setting up custom Debian and/or Fedora repositories, as most embedded projects seem to do, ideally with a proper autobuilder. They should learn about OpenEmbedded and Poky too.
Today I released Glom 1.10. There are no big new features compared to Glom 1.8. It’s mostly just a port to libgda-4.0 and the addition of an experimental sqlite backend, with limited functionality compared to the default PostgresSQL backend, for embedded use. Johannes did most of the libgda-4.0 port and Armin did the sqlite backend.
libgda-4.0 is a significant improvement over libgda-3.0, giving us better performance with huge numbers of rows . It also allows us to remove some code from Glom, such as code to escape binary data for SQL queries. Vivien Malerba was very responsive to our feedback and took our patches quickly. I look forward to using future API to construct SQL queries from conceptual parts so we can remove that code too.
We will try to get Ubuntu packages into the Openismus PPA soon, at least for Ubuntu Jaunty, which otherwise has a (recent, bugfixed, woohoo!) Glom 1.8 in the official repository.
I recently took another look at boost::python. It was a much better experience than when I first tried boost::python for Glom in 2005, probably because my use of the Python C API in the meantime has helped me understand what boost::python is doing. I have a mostly-done patch to use it in Glom 1.12 (Glom 1.10 will be released soon). This should make the code simpler and much more robust, allowing me to add more Python API to Glom, allowing people to drive more of the Glom UI via scripts.
As a side-effect this will force us to enable C++ exceptions in the Maemo build for Glom 1.12, increasing code size, but that might be less of an issue by then.
I do find the boost::python documentation fragmented and unfocused, spending too much time congratulating itself about its use of various design patterns and generic programming techniques in the implementation, instead of just telling me how to achieve common tasks. It often assumes knowledge of the Python C API, as if it is based on original proposals or internal documentation. Many companies are using boost::python so I’m surprised that none have arranged for more useful documentation to be written. I’d be happy to do it if someone wants to pay for my time.
In fact, boost::python’s API maps closely to the C API, so you probably need to know both, though I hoped that boost::python would make it clearer and more explicit.
However, the people on the boost::python mailing list have been very helpful when the documentation has not been clear or where I have made silly mistakes.
Getting the CFLAGS and LIBS for boost python in your configure.ac is insanely difficult and fragile. There are some .m4 scripts out there, but I can’t get any of them to work. Why on earth don’t they install a .pc file? They can’t all be Windows programmers.
And despite using unstable APIs, they don’t seem to allow parallel installs. For instance, on my Ubuntu Linux system, the headers are directly under /usr/include/ rather than /usr/include/boost-python-1.0/. GNOME gets this stuff right.
The need for parallel installs is even greater for boost::python because there are various possible incompatible configurations, any of which you are likely to find on your system. At the moment you will just get compiler or linker errors (which distro packagers don’t understand) instead of being able to explicitly depend on a specific version with a specific build (which distro packagers could understand).
This will make life difficult for Glom distro packagers, but I think it’s still worth it.
Over the last month or so our two trainees, David King and Michael (not Mathias) Hasselmann have made good progress getting familiar with GTK+ and associated tools on Linux. They are on an intensive schedule, but they have the time to learn how things really work, so they don’t have to feel that any part is a mystery. For instance, they know now how to create custom GObjects and GTK+ widgets rather than just how to put widgets together in Glade. Now they will move on to C++, moving through gtkmm and then to Qt, with detours through Maemo and Scratchbox.
We hope to offer training to customers in the near future and this is giving us a good idea of what to cover and how.
Beyond just coding, Daniel and I are helping them to form good open source habits, creating developers in our image, so they can be creators of quality and fighters against entropy. I’ve encouraged them to blog about the experience and generally get involved in the community as an important part of their training, so don’t hesitate to give your advice.
It’s also interesting to see how the move to Germany has been for David, registering for various things and finding an apartment. It seems easier in Berlin than in Munich, and easier now than when I moved to Germany 10 years ago. Daniel‘s help has been a big time-saver, I guess.
I set up planet.openismus.com to show their blogs and all our others too.
Over the last few months we (Openismus) fixed several nasty regressions in Glom 1.8 compared to 1.6. The latest release is fairly usable. We really must set up some LDTP testing scripts or suchlike to catch these regressions before our .0 releases.
The Openismus PPA has updated packages for Ubuntu Intrepid, based on Iain Lane’s packages. The official Glom packages for Ubuntu Intrepid are 1.6, which was finally updated to a bug-fixed release, thanks again to Iain Lane.
Work on Glom 1.10 is going well, though it’s largely a port to libgda-4.0.
I’m blogging about this a little late because I waited for some (internal) Flumotion Incorporated releases before mentioning it.
Between July and September 2008 I did about a man-month of work for Openismus on Flumotion‘s user manual (HTML, or PDF, or DocBook in the GNOME help browser), the first major piece of user documentation I’ve done. I usually write developer documentation. Flumotion is an audio/video streaming system. From the introduction:
Flumotion can broadcast your content live or on-demand in a range of popular audio and video formats. It supports a wide range of input hardware thanks to its use of Linux, GStreamer and other open source software. You can get started quickly with Flumotion’s simple user interface then just place a link to the streamed content on your website.
Flumotion allows processing to be spread across multiple machines, so your platform can scale to handle more viewers of more streams in more formats. Its open source architecture makes it more efficient and more flexible than competing systems, making better use of your hardware.
I worked from a table-of-contents that I agreed with Thomas Vander Stichele over a couple of days in person. There was a previous version but it was very developer-focused, immediately talking about the internal architecture of the software. Lots of that existing content was reused, updated, and expanded as sysadmin information, but the first few chapters are new, actually making the manual user-focused.
We wanted an immediate introduction to Flumotion while providing a path to its full functionality and clearly offering the necessary information for later deployment, configuration, and security. I think we succeeded. We also added a reference section that’s generated from the Flumotion components themselves. These and other sections are heavily interlinked so it’s easy to get more information regardless of where you jump into the manual, because it’s natural for different people to focus on their area of concern.
I needed to ask lots of questions and do follow-up interrogation to get the raw information before writing content. Along the way I found myself acting as a tester, filing several Flumotion bugs that were then fixed, so Flumotion now works better as well as being better documented. The Flumotion/Fluendo developers were incredibly helpful and never lost patience with me, making the work a real pleasure.
Note that neither the HTML or PDF at flumotion.net have yet had any Flumotion-specific style or layout applied to them.
I’ve been writing DocBook documentation for a long time, starting with the gtkmm manual, so I’m generally familiar with the tools. (Though I had to learn lots of new markup and correct terminology for documenting an application instead of an API). Applications can just install their DocBook XML directly and leave the rendering to the GNOME Help browser (yelp), but you often need the document in HTML or PDF format to show on a website.
The DocBook XSL stylesheets for XHTML are still the correct way to generate HTML and work well. In the past I used docbook2pdf (a simple way to use jade) to generate PDF , as you can see in the gtkmm-documentation Makefile.am. But it quickly became obvious that docbook2pdf wasn’t good enough. For instance, it did not include our index in the PDF and didn’t offer any way to customize the PDF output.
So I tried fop again and actually got it working, at least on Fedora Linux 9. fop uses Java so it had not really been packaged on Linux distributions before Java was open-sourced. I was pleasantly surprised at how much progress has been made since my last attempts. In the past it was frustrating to hear that we should be using fop because that was the only system being maintained, but never finding anyone who had managed to get it working.
Using fop means generating a .fo (XML) file from your DocBook XML, by processing it with the official fop XSL stylesheet from the DocBook people. fop then processes this .fo file to create a .pdf file. You can influence the end result by setting some XSL parameters when generating the .fo file.
Even on Fedora 9 there were various bugs to work around, such as:
As the flumotion-doc Makefile.am shows, we also chose a useful set of DocBook XSL Stylesheet parameters, which are surprisingly well documented. These avoid some problems (maybe bugs, though its debatable) and add some attractive admonition icons.
That Makefile.am is the result of much work so I hope it’s useful to other people trying the same thing. Note that it has a little extra complication because we bring in some XML examples inline, just like we bring C++ examples in to the gtkmm book.
Unfortunately I can’t confirm yet that these bugs are really fixed in Fedora 10 because Fedora 10 has some even more serious bugs that prevent me from getting that far:
And Ubuntu Intrepid (the first Ubuntu to package fop) has the same problem: fop crashes with SEVERE: Couldn’t find hyphenation pattern en. Plus the workaround for the FOP PNG problem (and the scaling) doesn’t work due to an apparently totally-broken Imagemagick on Ubuntu.
This is rather disappointing. I was looking forward to using fop for the gtkmm book, for both HTML and PDF, hoping that the syntax highlighting stuff would work on our inline example code. Note to self: This might be useful help for that.