More evolution-reliability-speed

04Jan10

Evolution recently moved to a sqlite summary db rather than a custom summary db implementation. Its great to see such reuse of code.

However, it’s not really a complete transition yet as I’ve had cause to find out today. I’ve blogged before about performance with the sqlite summary sqlite database. Today I was greet with a crash-on-startup bug which happily has a patch upstream already. Before I looked in the bug tracker though, I did some house cleaning.

I started with a 900MB folders.db. Doing a vacuum on the db dropped that to 300MB. It doesn’t appear to be something that evolution does itself. Firefox too appears to lack an automatic vacuum. sqlite is an embedded database, and its wonderful at doing that, but its not as install-and-forget as (say) PostgreSQL which does autovacuum. So an additional tip is vacuum your folders, e.g. with http://www.gnome.org/~sragavan/evolution-rebuild-summarydb, a helper script that will run vacuum on all your account summary db’s. Note that it *does not rebuild*, it solely vacuums, and as such does not add or delete (modulo bugs in sqlite) data to the summary database.

After the housecleaning, I checked that the sqlite database was in good condition:

sqlite3 folders.db
pragma integrity_check;

This returned a number of indexing issues, so I reindexed:

reindex;

Evolution now starts up and crashes in a fraction of a second – a big improvement. Finally, I started looking at the evolution code as I now was fairly confident it was a bug – it was in a sqlite callback function – and the column the function extracts data from (flags) is missing a NOT NULL constraint, but the code doesn’t check for NULL – boom. From there to finding the bug report and existing patch was trivial.

And this is where my comment on reliability turns up: Evolution doesn’t anticipate NULL flag values in its code, so why does it insert them into the database at all ? I suspect its due to some aspect of the incremental conversion to using sqlite summaries. More concerning for me is the possibility that there are many other such crash bugs lurking in the new sqlite based code.

There are possibly some clues as to the excessive table scans done by evolution in the use of a flags bitset rather than separate columns, but I haven’t looked close enough to really say.

About these ads


3 Responses to “More evolution-reliability-speed”

  1. 1 rdb

    Thanks for this, it helped my setup (too many firefox tabs, evolution) all in
    1Gb on Ubuntu.
    Rather than each applications each implementing autovacuum, would the approach of keeping a list of sqlite DB filenames that get vacuumed on system startup work better – it takes a while for the DBs to crud up after all.

    • 2 rbtcollins

      Something like that might be good, yes. Possibly just a policy setting for sqlite to control things might be enough.


  1. 1 The Art of Being Dorian » A Day in the Life of Dorian for 2010-01-04

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


Follow

Get every new post delivered to your Inbox.

Join 1,046 other followers

%d bloggers like this: