For the past week or so I’ve been improving how Glom shows big tables in its list view. I created a local copy of the musicbrainz database for testing. For instance, the “artist” table has almost 600,000 rows. Glom 1.6 is awful at this, but Glom in svn trunk is getting better. There are several problems which I’ll mention in different posts.
By the way, I made it slightly easier to use Glom on an existing database, but it will still act oddly on non-Glom field types, and it needs you to edit an XML file. I’m now ready to add a use-an-existing-database feature to the GUI and make it handle other field types, and maybe other database constraints, if anybody wants to pay Openismus to do that.
GtkTreeView calls gtk_tree_model_iter_next() on every row
The first problem is that GtkTreeView always calls gtk_tree_model_iter_next() on every row in your model, even if it has millions of rows. Even if you have a custom GtkTreeModel and even if you are using fixed-height mode. Surprisingly that happens quite quickly, but it’s really not sensible.
And this makes it even less possible to attach extra data via pointers in the user_data fields in your GtkTreeIter, because you can’t free that data until the model is destroyed, because you have no other way of knowing when nobody else could be using that GtkTreeIter. GtkTreeIter is not reference-counted – it’s copied by value. Even tinymail (otherwise very efficient) must have a huge array in memory so that the GtkTreeIter’s index into that array stays valid.
So GtkTreeView just isn’t made for millions of records. I don’t want to implement a new GtkTreeView, but if someone does then I hope we have something in GTK+ that we can all use. A generic Iterator type in Glib (Philip Van Hoof’s page) might be a small start on that.
As far as I remember Banshee guys did something similar in trunk (a tree view that does not pull invisible data). You might want to check that out.
I know gtk_tree_model_iter_next() isn’t called for every row in all cases, I made sure it’s not the case in my app. I would say, in addition to fixed height, the columns need to be fixed width too. Maybe something else is needed, I don’t remember at the moment.
Patrys, yeah they did that in Mono, so it isn’t much use (other than as inspiration) to most people.
squentin, really it does, and it must do that even in your app. See gtk_tree_view_build_tree() here:
http://svn.gnome.org/viewvc/gtk%2B/trunk/gtk/gtktreeview.c?view=markup
There’s a big “do { … } while (gtk_tree_model_iter_next (tree_view->priv->model, iter))” loop
Fixed-height mode just stops it from requesting the _values_ for each row. Which would take even longer.
Ok, sorry you’re right it does. I wrote it a long time ago, I must have confused it with the get_value call :( Still it’s very fast, even in my case where the get_next is a perl function, with 20,000 rows.