Django unit testing gotcha: test case methods run in alphabetical order
09 March 2009 | Matt Perdeaux | Backend, Quality & standards
After a good few hours of increasing frustration, I managed to work out why the Django unit tests I was working on were doing funny things. As it turns out, the individual test case methods are, by default, sorted alphabetically by the TestLoader.
This is worth knowing if you're testing things that rely on the tests being run in order, such as creating, editing and deleting of records in the database.
For instance:
import unittest
from myapp.models import MyModel
class MyModelTestCase(unittest.TestCase):
def test_create_item(self):
obj = MyModel.objects.create(title="hello")
.... run tests ....
def test_edit_item(self):
obj = MyModel.objects.get(id=1)
obj.title = "goodbye"
obj.save()
.... run tests ....
def test_delete_item(self):
obj = MyModel.objects.get(id=1)
obj.delete()
.... run tests ....
In the above example, when the tests are sorted alphabetically, "test_create_item" is run first, but "test_delete_item" will be run before "test_edit_item", leading to a few unexpected test failures for no apparent reason.
It looks like this is all to do with the "sortTestMethodsUsing" attribute of the TestLoader class, and I daresay there is a way of overriding it in Django, but for now, I'm just happy knowing how my tests are being sorted so I can name them sensibly.
