Glom 1.2

General, ,

Glom logo Glom 1.2 is now out (see the announcement). It has a few new features and less bugs than the 1.0 branch. It should be available in Ubuntu Edgy soon. Glom 1.0 is apparently now available in Fedora 5 and 6, and I expect that to be updated to 1.2 soon.

For Glom 1.4, I hope to finally port to the latest libgda API, and add a dependency on gtksourceviewmm, to do source-code highlighting of Python code in calculated fields and buttons scripts. And hopefully Rasmus Toftdahl Olesen‘s Relationships Overview feature will also be added.

Updates of non-essential stuff in Ubuntu

Ubuntu 6.06 (Dapper) will be the recommended stable and supported Ubuntu release for a while yet. It only has Glom 1.0.3, though a 1.0.8 version exists with lots of bugfixes. I can understand that a stable version of a distro doesn’t want to add features by jumping to Glom 1.2, but I don’t understand why bugfix updates aren’t allowed. Generally it seems that updates are only allowed for security bugfixes.

Sure, you have to be careful. If Ubuntu updated OpenOffice in Ubuntu Dapper to fix a crasher then they’d have to be very careful that they weren’t introducing new bugs. But nobody is using Glom for anything critical yet, and it’s not in the officially supported repository. Just because an X update broken things once, it doesn’t seem necessary to freeze the Glom version. Of course, I want people to get my bugfixes so I can get more feedback, so I can make Glom good enough to be relied upon.

Ubuntu Mountain View Summit

General,

Canonical has generously offered to pay my way to the Ubuntu MountainView developers summit in November, to bring some extra points of view from the GNOME project.

I’ll arrive on Thursday evening, so I can spend Friday taking a look at San Francisco. I need to find a relatively cheap place to stay for 2 nights. This will be my second free junket of the year, though the last time I spent most of my tourist day in the hotel with a nasty cold. The California sun should make that unlikely this time. If anybody would like to meet to eat vegetarian on Friday night (3rd November), just comment or email me.

I have a strange adolescent star-truck enthusiasm about visiting the west coast for the first time – and only my second time in the states. Back in my early pre-internet twenties, I wanted nothing more than to go work in Silcon Valley, because it seemed like the center of the software world. I probably would have done it if the visa restrictions hadn’t required a university degree. Now I feel that southern California’s urban sprawl is probably as characterless and disfunctional as the U.K.’s Thames Valley where I lived at the time, and I’m content to enjoy the quality of life in Europe, working with the world’s best software people via the internet. But I suspect that visiting the Googleplex will still feel special.

Overselling Portland

General

I have nothing against promotion, but overselling your product becomes a mistake pretty quickly. Statements such as “For the first time, ISVs are able to port their applications to Linux regardless of desktop environment.” will just ruin OSDL’s reputation and kill OSDL’s opportunity to help those ISVs. This isn’t a one-off mistake either.

Let’s make it clear:

  • a) Portland 1.0 is a set of command-line tools to help applications to integrate with the Linux desktop. That is useful.
  • b) When ISVs think of porting their applications to Linux, they think first of GUI toolkits, file handling, networking.

a) is not b). We have lots of great b). For instance, ISVs who have suffered with MFC or win32 love GTK+ (or gtkmm, etc) or Qt. They are a bit annoyed that they can’t really use win32 or MFC on Linux, but they aren’t surprised and they are pleased that the alternative is a good one.

But if you tell them that Portland 1.0 is the alternative platform they are looking for, they will very quickly conclude:
1. “Erm, command-line? So, Linux hasn’t done much in the last 10 years. Bye.”
2. “Platform? No it’s not. How stupid do you think I am? How stupid are you? Bob, don’t forward me any more calls from OSDL.”

And Portland is never meant to be a desktop-agnostic GNOME/KDE-combining GUI toolkit. If you let people think that then it’s your mistake not theirs.

OSDL (and the Portland project) are doing great stuff, making a better Linux desktop, and creating a marketplace for ISVs. Don’t ruin it with this nonsense.

HRB 164185 (Openismus GmbH)

Uncategorized, ,

Openismus GmbH now 100% officially exists. All the steps are complete. It’s done. I win. Wikipedia can tell you what a German GmbH is, so ignore my summary if you want accuracy.

This is the easiest form of company to set up in Germany (or an OHG if you are selling physical products). Anything else (such as a GbR) isn’t really a company (“Firma”) and doesn’t have limited liability like a U.K. Ltd company. It’s ridiculously difficult and expensive to set one up, compared to a U.K. Ltd company. On the other hand, you don’t need a GmbH (or GbR) to do business, and Germany makes it very easy to do freelance work without a company. In the U.K. you tend to need a Ltd company, though that’s maybe just so you can pay less tax.

In fact, a person with a GmbH pays more overall tax than an individual and a GmbH demands more (expensive) administration. Yet it’s that difficulty that gives a GmbH an air of respect. Every now and then Germans politicians discuss making it easier but then a bunch of them point out how awful that would be because some of them might fail. Hello? Jobs? Muppets.

Also, clients outside of Germany need to deal with something that’s recognizably a company without having to understand German law enough to know that you don’t need a company in Germany. Note that residents of Germany may set up a U.K. Ltd company instead, thanks to the EU, thus avoiding some of Germany’s beaurocracy, but that still looks suspicious to German clients, and isn’t common enough yet for the procedures to be well understood by German accountants. But I expect this to become accepted in future, leaving the GmbH as a provincial anachronism.

Alternatively, some EU-wide form of company will become more accepted. There’s already an SE company form, but it’s limited to companies with a minimum capital of 105,000 Euros. I’m still convinced that national governments of the EU will fight to the last to preserve their incompatible islands of tax complication. How else can they promise cash to their backers in the form of tax loopholes and allowances. The only EU-wide companies big enough to influence them are big enough to pay their accountants to deal with it.

Anyway, the process went like this.

  • Week 0: I tell the accountant that I want to form the GmbH. I’m going on holiday for three weeks, and I want everything ready by the time I get back. This includes
    • Creating a draft “Satzung” (agreement, contract) for the GmbH and sending it to me.
    • Arranging an appointment with the notary. He’s a guy who sits between you and the government, because some things are meant to be difficult. His purpose is to send you a bill for his time.
  • Week 4: Get back from holiday. Nothing has been done. Start again. My accountant is consistently unreliable, but now I’m dealing with someone new there, so there’s hope again. He gets the Satzung and the appointment with the Notar.
  • Week 5: Arrange an appointment with my bank to open the company bank account with the necessary 25,000 Euros starting capital, because the accountant says I need this before meeting the notary. But the person at the bank says that they can’t do this until I get the piece of paper from the notary. A telephone call establishes that the bank can have it its way and the notary won’t mind. Apparently Hypovereinsbank do things differently.
  • Meet the notary. His role is justified by the legal requirement for him to read your contracts aloud, such as the Satzung. He doesn’t ring a bell while doing this, but he really does sit there and read it to you, and you listen. Apparently this can take 3 hours for complex real estate deals. This costs me 418.18 Euros for half an hour of being read to. Good work if you can get it. In Romania they have other names for this, and the EU tries to stop it. But in Germany they send you a detailed bill for it.
    Update: I also received a bill from the Landesjustizkasse Bamberg for the registration and publication of the fact that it was registered, for 322.39 Euros. So that’s 740.57 Euros just for the beaurocracy, ignoring the cost of an accountant to navigate that beaurocracy for you.
  • Send the notary’s documents to the bank. They have a nice seal and piece of string in the light blue and white Bavarian colours.
  • Week 6: The bank has the documents it needs to open the “GmbH i.G” account. i.G. means in Gründung, or “being founded”. Then I am allowed to put the required 25,000 Euros into the account. That can hurt when you have zero need for capital, such as when starting a software company. Opinions differ, but I don’t think it’s something that you get to see again as an individual, because paying it back to yourself is punishable by 3 years of prison. You can’t take it as profit, because profit is what you have above the starting amount. You can use it to pay yourself a salary, but of course you’ll be paying tax on that again (you paid tax on it when you earned it in the first place as an individual), plus you pay the social security contribution on it twice (you and your company pay it), so I guess that’s about 10,000 Euros that you lose, if you never close your company, just because the starting capital requirement is so high. I believe the minimum starting capital for a U.K. Ltd company is 1 pound. A notable difference.
  • Week 7: Send the bank’s printout of the account balance to the notary.
  • Week 8: Receive the printout from the business registry (Handelsregistrar) showing that the business is really registered, with a number. Send this document to the bank so they can remove “i.G.” from the account name.
  • Profit.

Glom 1.1/1.2

General, ,

Glom logoI’ve been working on a Glom 1.1/1.2 branch recently, trying to quickly add some features in time to release a stable version for the Ubuntu Edgy release. The Glom 1.1.6 announcement has a list of what’s new. My favourite new features are:

Add Related Table

This is a time-saver that lets you quickly add a table and a relationship that uses it. So, for instance, if you have

Album
Nebraska
Asbury Park
Closing Time

you can quickly add an ID field, and a related table, so you have:

Album Artist ID
Nebraska 1
Asbury Park 1
Closing Time 2

with

Artist ID Artist Name
1 Bruce Springsteen
2 Tom Waits

In future, I’d like it to take existing data from existing fields and put that into the related record, so that in one step we could get those two tables from this:

Album Artist Name
Nebraska Bruce Springsteen
Asbury Park Bruce Springsteen
Closing Time Tom Waits

Use of the pygda API

Via the records.connection object you can now use the whole pygda API (The libgda Python API) from your button scripts and calculated fields, so there is no limit to what you can do with the underlying database. Be careful with this great power. More details on the Glom API page.

This feature was easy to implement thanks to the wonderful pygobject_new() function from pygtk, which is a lot like gtkmm’s Glib::wrap().

Debian Repository Analyzer for license compliance

General

Over the last few weeks I’ve been using spare hours to create a utility to help companies discover the licenses of their software and to help them decide what licenses they should use for software that uses open source dependencies, and what they should do with their sources and modifications. Assuming that they are using a debian apt dependency system.

It’s a python script that uses the python-apt API to discover all the packages in a debian repository, and then looks for licenses in the source tarball or debian diff. It then tries to figure out what license texts are actually the same, using Python’s SequenceMatcher, because many license files contain short but irrelevant bits of unique text at the start. That can take an hour or so.

Then it creates a .glom file that you can open in Glom (>= 1.1.3). That creates a database and fills it, then lets you explore the data. At this point a human can give the licenses names such as “GPL”, “LGPL”, etc, which will then show up against all the packages that had that license text. And then you can see all the licenses of each package’s dependencies.

Here is version 0.1.4, under the GPL, as I believe is required by the python-apt license. Update: Here is a screenshot.

For now, I had to hardcode the base URL of the repository. I haven’t actually tested it completely with the example repository that’s in the sources.list, but it does work with the (secret) repository for which it was written.

Tips on using python-apt

Michael Vogt has been very helpful with my questions about python-apt, but he’s not to blame for my python coding. Remember, I’m a C++ coder. Python-apt is very useful, but the API is currently rather obscure and confused by the presence of two similar APIs alongside each other. Michael is working on making it sane.

I was also frustrated by the general lack of documentation of Python APIs. It’s often very difficult to know what type of object is likely to be returned by a method and to know what methods an object supports.

So here are some of the things that I learnt from Michael Vogt, so that they don’t just sit in my Inbox where nobody else can see them.

Using a local sources.list and cache instead of your system’s

You need to set a bunch of config variables. Here’s my full list so far:

apt_pkg.Config.Set("Dir::Etc::sourcelist", "./sources.list")
apt_pkg.Config.Set("Dir::Cache::archives", "./tmp_apt_archives")
apt_pkg.Config.Set("Dir::State", "./tmp_apt_varlibapt")  #usually /var/lib/apt
apt_pkg.Config.Set("Dir::State::Lists",  "./tmp_apt_varlibdpkg") #usually /var/lib/dpkg/
apt_pkg.Config.Set("Dir::State::status", "./tmp_apt_varlibdpkg/status") #If we don't set this then we will pick up packages from the local system, from the default status file.

You will need to make sure that those directories exist, along with some sub-directories.

After calling cache.update(), remember to do this, otherwise no packages will be found:

cache.open(apt.progress.OpProgress())

Getting the name of a package:

When iterating over a cache, you can do this (imagine the indenting because WordPress doesn’t want to show it to you):

for pkg in cache:
candver = cache._depcache.GetCandidateVer(pkg._pkg)
name = candver.ParentPkg.Name

Getting the full URI of a tarball:

You can get a tarball URI like so (again, imagine the indenting),

srcrec = srcrecords.Lookup(source_package_name)
if srcrec:
for (the_md5hash, the_size, the_path, the_type) in srcrecords.Files:
if(the_type == "tar"):
tarball_uri = the_path

but it only gives you the second half of the URI. To get the whole thing, if you have the latest version of python-apt from Ubuntu Edgy, do

full_uri = srcrec.Index.ArchiveURI(tarball_uri)


	

	
	

		
			



							


	

		
				

			Evolution and IMAP
		

		
		

			General, 					

	


		

		
Evolution is great 


I love Evolution. I’ve used it for years for POP3 access and it has easily handled huge amounts of email and allowed me to quickly find information. I can not use a command-line email client to read my email. I am not like that. Years ago when I first started using Linux, I had a Windows partition (actually a removable disk) just so I could boot into that to check my email with Outlook. That was tedious. Then Evolution let me use Linux all day every day. It was familiar but far more stable and faster and less confusing than Outlook.


Evolution is still great for POP3 and getting better all the time, and possibly for some other protocols that I haven’t used.


Evolution with IMAP is not so great 


But about a month ago I switched Evolution from POP3 to IMAP, because I got tired of not having access to old emails when away from my main PC, for instance when I use my laptop or when I use the webmail interface. This has mostly given me what I wanted, though dreamhost’s squirrelmail times out when searching my old messages, which are archived to a separate folder after a few days for performance reasons. But it means that I don’t stumble over not having the information in an email that someone sent me yesterday, and then realise the next day that I forgot to check it while at home.


However, it’s very slow, so slow that it rarely finishes whatever it’s doing before I shut my computer down. The status bar is always showing a couple of “Working oxb1208ad8” or similar messages. I think it’s doing more things with hexidecimal names, but it only shows two at a time. Every time I click on a folder it seems to give itself lots more work, and is determined to finish whatever it started doing for the last folder, even though I’ve lost patience with that already and just hope it can forget and maybe be quicker with another folder instead.


I’m not 100% convinced that this is caused by Evolution or instead by IMAP in general, or maybe by a problem with my IMAP server, or with IMAP servers in general.


Maybe (I have no idea) IMAP is a fundamentally slow or high-latency protocol, but it seems theoretically possible for Evolution to not be slow about showing me what’s already on my local disk. If it wants to check for new stuff on the server then that should happen in the background. I’m not saying it’s easy to implement, but it should be possible.


This whole downloading local copies thing is confusing in Evolution:


    

  • The account setup (Edit/Preferences/Mail Accounts/Edit/Receiving Options) has an “Automatically synchronize remote mail locally” checkbox.
    • 

  • Right-clicking on a folder in the Sidebar anch choosing Properties, allows me to check “Copy folder content locally for offline operation”. Isn’t this the same thing? And is it recursive? Am I really gaining anything by tediously setting this for the 40 folders in my folder for mailing lists?
    • 

  • You can force a download of local copies by choosing File/Work Offline, but a) Didn’t I just specify in the first two ways that it should always do that, and b) I can’t check new emai while it’s doing this.
    • 



I just built Evolution 2.8 from source (I usually use 2.6 on Ubuntu Dapper) to see whether it’s faster, but I don’t notice any significant difference.


I suspect that some of the slowdown is caused by me having 40 filter rules to sort my emails into different folders for each mailing list. Unlike with POP3, it has to tell the server to move these instead of just moving them on the local disk. I guess server-based rules would be more efficient but I have no way to specify such rules with my hosted email server, and Evolution certainly doesn’t offer any way to do this. Turning off email filters or junk filtering (in the account setup) doesn’t make it noticeably faster when switching folders, but I didn’t try this for a few hours, and Evolution does seem to get slower with use.


By the way, why can’t Evolution just arrange all my mailing list emails automatically without me having to create folders and filters for each one?


It also seems slightly faster when not using encrypted IMAP (with TLS or SSL), but not fast enough, and a) I like encryption, and b) It was fast enough when using encrypted POP3.


I have tried Thunderbird. It seems slightly faster, but I haven’t tried recreating my 40 filter rules to test whether it’s faster when doing that. It does at least show a pulsing progress bar when it has to download an email, instead of showing me a blank email for a minute or so. It has the same UI problems when saying what should be downloaded locally – there’s no way to say “download everything and keep downloading everything, so you can be fast”. One great thing about IMAP is that you can easily try other email clients.


I’m not complaining about the hard-working developers. Evolution does a lot of great stuff and I’m not ready to move away from it, particularly because Evolution integrates more with my desktop than Thunderbird is ever likely to. I’m sure they’ll fix whatever they can in time – I recognise that getting an email application right is not easy. I just wanted to get my thoughts written down. I am not jumping on the bandwagon.


Update: I should give Evolution 2.8 more of a chance over the next few days. I guess it does feel a bit snappier.

	

	
	

		
			



							


	

		
				

			New employee
		

		
		

			Uncategorized, 					

	


		

		
Openismus now has its third (after me and Johannes Schmid) employee, also part time: reclusive Berlin genius Daniel Elstner. In October he’ll start doing some work with gtkmm for a client, and writing some documentation for a project that Johannes has been doing. His enormous brain will also be very useful for bouncing ideas off. He’ll be required to blog too.

	

	
	

		
			



							


	

		
				

			Secure silent mail server
		

		
		

			General, 					

	


		

		
An Openismus client demanded that all email communication with them should be over TLS tunnelling, with the mail server being on-site, so my regular hosted mail accounts won’t do, and even a hosted server wouldn’t be enough. Frankly, I’d rather not have the trouble and expense of maintaining my own servers, but it’s a nice client so to please them I set up a mail server like so:


    

  • Buy a quiet (no fan) low-power PC. I chose an ichbinleise PC LL10. It’s not very pretty or powerful or small, but it’s enough for a mail server. Ichbinleise (MR Computertechnik), seem to use cases and parts from Silverstone.

      

    • The new AMD Geode-based PCs are cheaper and require much less power, but pre-built systems don’t seem to be properly on the market yet.
      • 

    • This list of mini PCs is interesting.
      • 

    • It’s not low-power and fanless, I think, but the A-Open Mini PC is pleasantly Mac Mini-like, though I suppose the new intel Mac Minis will support Linux distros some time soon.
      • 

    

    • 

  • Get a new DSL broadband login from Via, so I can have a static IP Address. The contrast between their web site (simple, what you need) and service (simple, quick) and that of T-Online/Deutsche-Telekom (insanely complex, obscure, beaurocratic, inflexible, expensive) is  vast.
    • 

  • Get an extra domain name just for this email account. I’m not going to risk putting my regular email through this experimental system.
    • 

  • The domain registrar doesn’t let me enter IP addresses for the DNS name server, so I can’t set up my own DNS server that specifies which of the PCs on my network is the mail server, and of course I can’t create a ns.something.com without having a DNS server. This is probably normal – I wonder how this chicken and egg problem is normally solved. So, I used EasyDNS to get a DNS server, and told the domain registrar to use that name server. Then my DNS entries at EasyDNS specify my static IP address.
    • 

  • I set up port-forwarding rules on my router (a cheap wireless/DSL thing that I got from Deutsche Telekom), telling it to direct traffic on the POP and IMAP ports to the mail server PC.
    • 

  • Instead of using port forwarding, I suspect I could have a local DNS server (which would know about the IP addresses on my local subnet) in addition to the EasyDNS server (which can’t use my local IP addresses because they aren’t really assigned, being behind a NAT), but I’m not sure how that would work.
    • 

    Now to install the mail server software:


    

  • I tried Ubuntu Breezy (I have one thousand CDs, so it’s easy to find one, and was too lazy to download Dapper), but  installation failed half-way saying there was a problem with the CD drive.
    • 

  • Ubuntu Dapper installed perfectly.
    • 

  • Thanks to a reply to my previous blog about mail servers, I found a great page about installing an email server on Ubuntu. It explained what the point of the different parts is, and told me exactly what to do to install postfix and dovecot with support for IMAP and SMTP using TLS. There’s a scary series of commands that it tells me to type in, but if I have to do that then I’d rather not have to figure it out by myself.
    • 

  • I found it odd that the email account is defined by the linux shell account, with the same password, so I’d have to add a new shell account just to add a new email account. Those wiki documents mention something about virtual mail hosts, but they seem to be about serving mail for multiple domains. Maybe that’s also how you separate email accounts from shell accounts.
    •