27 Aug 2009

27Aug09

Does your test suite take too long (e.g. 5 minutes). Or did it and you solved it? Or it doesn’t but its getting worse?

Tell me more, I’d like to know :-)

About these ads


4 Responses to “27 Aug 2009”

  1. 5 minutes for unit tests. 12 minutes for functional tests.

    The effect of this is that developers run a subset of the tests before checking in, and buildbot is responsible for catching any unexpected regressions.

    It would be nice if the tests took less time, but so far our half-hearted attempts to optimize them failed to discover any low-hanging fruit.

  2. More numbers: on my development machine (Intel Core 2 Duo, 1.8 GHz) running just the unit tests:

    Ran 6955 tests with 0 failures and 0 errors in 8 minutes 2.594 seconds.

    That’s ~70 ms per test, on average. It would be interesting to compare this with numbers from other projects.

  3. Ours did (Drizzle) and we fixed it. How?

    1) use transactions. Lots of disk Io sucks
    2) use a ramdisk or libeatmydata to have sync() not really sync (amazingly, on a typical development workstation, where you don’t have all your RAM free as you also run firefox, libeatmydata is faster than a ram disk)
    3) run things in parallel (you have N cpu cores, run at least N tests at once)


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


Follow

Get every new post delivered to your Inbox.

Join 976 other followers

%d bloggers like this: