Tag Archives: Glom

www.glom.org

Glom 1.22

General, , ,

I released stable Glom 1.22.0 a few days ago.

For easy installing, I also created a Glom 1.22 package for Ubuntu Precise, in the Openismus PPA. Ubuntu Precise normally has Glom 1.20. I wish there was some similar way to create packages for Fedora, or someone to do that for me.

Beside the multiple bug fixes, Glom 1.22 has a few new features:

  • Details: Foreign key ID fields: Add a New button next to the existing Find button.
  • Allow custom (not related) choices to be translated, with only the original text being stored in the database. This only happens when the choices are restricted, at least for now.
  • Related Choices: Default to showing the primary key but allow the field to be be other than the primary key.
  • Related Choices: Allow the user to specify a sort order.
  • Allow the database title to be translated.
  • Added command-line utilities to help with translation via po files.

I like writing regression tests

General, ,

Over the last few months I have developed the habit of writing tests when fixing bugs in Glom. These run during “make check” or “make distcheck” so I can be sure that the bugs have not come back.  (I know these tests should be smaller and even more numerous, but I can’t yet see a sensible way to break them up).

I had to do lots of build-system and code work to make this possible, because some code had to be moved out of the UI code, and many tests depend on a temporary local server instance. But now, it’s easy to add new tests and it feels so good.

When the tests succeed I have a feeling of confidence in the code, instead of a feeling of uncertainty. When they fail I feel glad that I had a chance to fix a regression before releasing a new version. I am surprised at how much more enjoyable this has made coding.

Online Glom: More Translation

General, , , ,

Online Glom’s standard UI strings are now translated too, instead of just the strings that are in the .glom files. I added some initial translation files (mentioned here too) but I need people to translate them, please. Feel free to just email the file to me. I am tempted to add them to the desktop UI’s translations so I can copy them across.

I also changed it from using a lang= token in the URL to using GWT’s regular locale= query value, re-simplifying much of my previous Online Glom code to support translations.

Now that I see how each new translation adds another set of gwt-java-to-javascript permutations to the already-slow build, so it now builds 41 permutations, I might switch later from the Constants to the Messages technique for GWT translation, because I don’t think that string lookup will be a big performance problem.

Online Glom: Now easy to build

General, , , ,

Building gwt-glom

At least on Ubuntu, it’s now easy to build and test gwt-glom. You can just do:

$ sudo apt-add-repository ppa:openismus-team/openismus-glom-unstable
$ sudo apt-get install default-jdk maven2 libjava-libglom-java glom-utils
$ git clone git://gitorious.org/online-glom/gwt-glom.git
$ cd gwt-glom
$ mvn gwt:run

This opens the GWT Development Mode GUI, which serves gwt-glom via jetty and lets you see it in your browser.

Chrome is the most likely to have the gwt plugin that you need for development mode, though it’s available for some versions of Firefox.

How we made it this easy

GWT projects, like other Java projects, typically use maven (mvn) for their build system. maven is nothing like autotools.

Maven usually downloads dependencies automatically, either from the central maven repository, from some (maybe private) other maven repository that you specify. This sounds unstable, but you specify exact version numbers, so you can be sure that your project will continue to build. So maven doesn’t have the separation of building and packaging that I’m used to in the C/C++/autotools world. It feels odd to me, but I’m going with the flow.

However, java-libglom uses JNI to provide a Java API around a C++ (libglom) API, so it uses both Java (architecture-independent) and C++ (compiled and linked for particular architectures).

java-libglom installs a native shared libary. We packaged that for Ubuntu in our Glom PPA (stable and unstable) as libjava-libglom-java so you can install it easily.

java-libglom also creates .pom and .jar files (for the API, the sources, and the javadoc). These are in the central maven repository so maven can just download the .jar.

That libjava-libglom-java Ubuntu package also installs the .pom and .jar files that maven needs, but those are only useful for mvn-debian, which is apparently only useful for building other Debian/Ubuntu packages. There is no apparently no way to using mvn-debian’s local repository while also using the central maven repository for other stuff.

Java with autotools

By the way, java-libglom uses autotools, although the autotools Java support is barely useful and limiting, so we have custom rules for most stuff. However, autotools does let us generate the Java and C++ files from swig, and build the native shared library. That seemed harder to do with maven, though maven would have made it easier to deal with the generated Java code.

I do like how maven just defaults to using your .java files, and test ,java files properly if you put them in the correct places. For instance, see the maven quickstart project.

Online Glom: Translations

General, , ,

This week I changed libglom’s TranslatableItem API to allow Online Glom to use the recent translations of the Glom example files. This requires the latest unstable version of libglom-1.22 and java-libglom.

There is now a language drop-down list at the top-right, and that is shown in the URL as, for instance, &lang=de (I might change that to locale instead of lang). Choosing a different language will change the table titles, field titles, group titles, etc, to the chosen language.

I have deployed that to the Online Glom test server. For instance:

I have not yet done the same for the translatable strings that are in gwt-glom’s Java source code, such as “Search”, and the “Open” and “Details” buttons. I’m not much looking forward to the awkward way that it should apparently be done. gettext’s _() macro is much cleaner.

Online Glom: Implementing a search box

General, , ,

I just implemented my first feature in Online Glom after taking over from Ben Konrath. Luckily I could cargo-cult his work, with some help from Eclipse’s code navigation and refactoring. I added a text box for an easy full text search of the current table, to filter the rows shown. For instance, you can filter the list of packages here.

I’m proud of myself, but I can’t help feeling that it needed far too many code changes, in far too many interconnected classes, just to add a text box in the browser and then use the resulting text to slightly change a method call on the server. I took the time to write a detailed ChangeLog/commit-message just to be sure that I understood what was happening.

This is apparently how things are in the land of Java. In its defence, there are reasons for the abstractions and separations. For instance, OnlineGlom code does the GWT thing of allowing for different View implementations for different clients, such as mobile or desktop, but it only has one type of Views for now. It also uses a Model/View/Presenter architecture, via its Activities, which makes it possible to test more code logic without getting the UI involved. And there’s the Places idea, which maps history tokens (parts of URLs) to Activities and vice-versa.

I wish there were some way to have the useful architecture without the repetitive code that gets in the way of the interesting stuff.

Glom: Translating the example files

General, ,

Glom‘s files can contain translations for the various table names, fields, relationships, choices, layout groups, reports, etc. This lets multiple people use the same database UI in different languages. Glom doesn’t require programming, so the .glom files don’t use gettext. But Glom can export the translations to .po files so translators can use their familiar tools. This week I added some command line utilities to do that more easily, plus some make rules to use them on the Glom examples.

I can now type “make examples_export_po” to export .po files and “make examples_import_po” to read the improved translations back into the Glom example files.

I put the .po files in Glom’s git repository, hoping that people will translate them, and maybe add more .po files. I wonder if these could even show up on Glom’s l10n.gnome.org page. Update: They do show up there now, thanks to Claude Paroz, and we already have some translations.

Online Glom

General, , , ,

Ben Konrath recently finished up a first real milestone in Online Glom development, and I have deployed it, with the example Glom files, on an Amazon EC2 instance. Online Glom is still a read-only UI, so you can’t edit data in the database, but I think it’s already useful for some situations. Well, I do want to add the quick search feature soon to really finish off the read-only functionality. Report-generation would be good too.

The Film Production Manager example is a fairly good example of the complex systems that Glom can support, without SQL and without programming.

Here’s a screenshot of the same thing in the desktop UI.

I have also deployed my old Debian Repository Analyzer, full of real data, so you can get a feel for navigating around the related records. I spent lots of time updating that for the latest python-apt API and libgda (with python) APIs, and it’s now in a gitorious project. Many thanks to Michael Vogt for his help with python-apt.

Now that David King has packaged java-libglom (as libjava-libglom-java) for Ubuntu Oneiric in the Openismus PPA, it’s really easy to work with the gwt-glom code on that distro.

What’s Next

This proves that GWT was a sane choice, though development has not been as quick as I’d hoped . But I don’t think a generic framework can ever be developed as rapidly as most data-driven web sites that start as quick hacks. The difficulties have shown that I was right to focus on a restricted set of functionality at first. You can get a sense of how development has progressed by looking at the gwt-glom commit log, though there’s lots of work in java-libglom too.

I now plan to take the development further myself. That’s a nice way to get more deeply reacquainted with Java and web development. It’s easier to hack on the project at this stage, before it gets huge, rather than trying to do this from scratch. Ben Konrath has already made the large architectural decisions so that I don’t have to. I’m even enjoying Eclipse, which is a much more pleasant experience with Java than with C++. Well, using the Eclipse IDE is still like shopping in a flea market but with Java you will quickly find useful things.

I plan to do things roughly in this order:

  • Add the Quick Find feature, for searching. Update: Done
  • Add report generation.
  • Add print layout printing.
  • Maybe investigate how to make theming via CSS easier.
  • Allow editing of data. This will be a big task.

At the same time, I will play with avoiding the need for java-libglom’s JNI binding to the libglom C++ library. I would need to reimplement Glom document parsing in Java, but that would be easy as it’s just XML. But I would also need some replacement for GdaSqlBuilder, to build SQL queries without manually concatenating and escaping text.

 

Glom 1.20

General, , , ,

It has taken over a year of hard work but Glom 1.20 is finally ready.

This is the most stable and most usable version of Glom yet, with a few new features, now using gtkmm 3. Here are some updated screenshots. The major changes:

  • Simplified main window.
  • Glom can now store and display PDFs. It can store any file format, though it can only display images and PDFs.
  • Print Layout: Major overhaul with improved UI and new functionality.
  • Related Records: Allow the developer to specify how many rows to show.
  • Choice drop-downs can show more than 2 fields.
  • Choice drop-down fields are aligned.
  • Choice drop-downs can show related fields.
  • List columns have sensible widths.

Around 20 bugs from bugzilla were fixed, plus several more.There are over 30 tests that prevent regressions in mostly non-UI code. I still wish I could get some kind of automated UI testing working.

We have packaged Glom 1.20 (and several dependencies) for Ubuntu 11.10 (Oneiric) in the Openismus PPA. We might do something similar for Fedora.

Unfortunately, it will probably take a year for Glom 1.20 to officially get into distros such as Ubuntu (bug) and Fedora (bug) officially, and then they won’t deliver bug fix updates. This is partly due to lack of the volunteer packagers’ time and partly due to unfriendly distro policies towards applications. My only hope now is something like Glick.

Glom: Avoiding SQL Injection

General, ,

I recently made sure that Glom is not vulnerable to common SQL injections, which can risk destroying your data or providing unintended access to data. I’m mentioning it here so people can tell me if it’s not good enough. Be kind.

For the regular Glom desktop UI, it’s probably not a big issue, because only a small number of users have access anyway. But I believe in doing things properly and this certainly is an issue for Online Glom (more about that soon).

It turned out that things were mostly safe already, thanks to our use of GdaSqlBuilder instead of raw string concatenation. We still build a few special queries manually, to change database structure or permissions, and those would never be used by Online Glom, but I fixed (1, 2, 3, 4) them anyway. As a bonus, this avoids problems with special characters. Vivien Malerba, who is always incredibly helpful and responsive, fixed some related issues (1, 2, 3, 4, 5) in libgda.

I added some SQL injection tests to make sure. I would welcome ideas (or patches) to test extra vulnerabilities. Like all of Glom’s tests that use an actual database, this tests libglom with both PostgreSQL and SQLite, though SQLite isn’t used by the regular Glom build.

Likewise, I made sure (1, 2) that we use g_shell_quote() when building shell commands, for instance to invoke (spawn) postgres or tar. Of course, I would rather not do that anyway. And I made sure that we properly escape values in our database connection strings.