27 Aug 2009

Does your test suite take too long (e.g. 5 minutes). Or did it and you solved it? Or it doesn’t but its getting worse?

Tell me more, I’d like to know 🙂

4 thoughts on “27 Aug 2009

  1. 5 minutes for unit tests. 12 minutes for functional tests.

    The effect of this is that developers run a subset of the tests before checking in, and buildbot is responsible for catching any unexpected regressions.

    It would be nice if the tests took less time, but so far our half-hearted attempts to optimize them failed to discover any low-hanging fruit.

  2. More numbers: on my development machine (Intel Core 2 Duo, 1.8 GHz) running just the unit tests:

    Ran 6955 tests with 0 failures and 0 errors in 8 minutes 2.594 seconds.

    That’s ~70 ms per test, on average. It would be interesting to compare this with numbers from other projects.

  3. Ours did (Drizzle) and we fixed it. How?

    1) use transactions. Lots of disk Io sucks
    2) use a ramdisk or libeatmydata to have sync() not really sync (amazingly, on a typical development workstation, where you don’t have all your RAM free as you also run firefox, libeatmydata is faster than a ram disk)
    3) run things in parallel (you have N cpu cores, run at least N tests at once)

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s