Friday, January 28, 2011

Ordered Unit Tests

Due to the way classes are constructed in Python, and to how tests are grouped in classes when using the unittest module, ordinarily these tests are executed in arbitrary order (sorted by the method name hash in cPython 2.x)

It is not diffcult to imagine situations when we'd like our unittests to run in order (most likely from simpler to more complex tests). Python 3 metaclasses include a feature that unittest could use to ensure ordered execution of tests, if it where coded to do so (the "__prepare__" method in the metaclass can change the assemble-time class dictionary to an ordered dict, and anottate tehe execution ordeer from there).

However, I wanted an implementation that would work in Python 2. So I came up with this "ordered" decorator that can ensure execution order for tests using unittest. It is deliciously hackish, as you can see: the methods are still called in arbitrary order - but what runs is the decorator code, which then takes care of calling the next test in order, regardless of the one unittest thinks is running. :-)

Appart from having to use the functools.wraps decorator (to preserve method names so that unittest actually finds them -- I didn try without it, it might work), the code is rather straightforward once it is written down.

With you, tonight's metapython bit:


  1. Hello, This is a bit off-topic wrt the post I'm placing a comment on (but I coudln't find a way to contact you!). Your paste ... I wanted to ask a few questions about it ... Specifically why the use of threading and some kind of thread-local state and a crude state machine ? Just trying to get my head around what you implemented there as I might find something similar quite useful in my framework.

    Please contact me via email or irc (prologic on FreeNode or prologic at shortcircuit dot net dot au).


  2. That's awesome!

    And in case you are using Python 3, you can use some new metadata tricks to make it work without the decorator.

    Here you go: