~ / insights / why-my-emails-vanished-in-django-test
2026-09-04 django python testing transactions backend

Why my emails vanished in Django Tests?

I spent three days on why Django's default test runner silences transaction.on_commit.

I recently ran into something that felt weird at first but turned out to be a classic Django testing quirk. I was working on a user registration endpoint where, after creating a new user, I wanted to send an OTP activation email. To keep things clean, I wrapped the email-sending call inside transaction.on_commit so it would only fire once the database commit succeeded, like so:

class RegisterView(generics.CreateAPIView):
    """POST auth/register - create an inactive user and email an OTP."""
    ...
    def perform_create(self, serializer):
        with transaction.atomic():
            user = serializer.save()
            transaction.on_commit(lambda: send_verification_otp.delay(user.id))

In development, everything worked perfectly. The user got created, the transaction closed, and the email went out. Then I wrote what I thought was a straightforward test (pretty obvious, I think)

class RegisterViewTestCase(APITestCase):
    def test_register_queues_otp_task(self):
        payload = {
            'email': 'test@example.com',
            'password': 'StrongPass123!',
            ...
        }

        response = self.client.post(
            reverse("accounts:register-user"), 
            payload, 
            format='json'
            )
        
        self.assertEqual(response.status_code, status.HTTP_201_CREATED)
        ...
        self.assertEqual(len(mail.outbox), 1)
        ...

and it always fails at self.assertEqual(len(mail.outbox), 1).

My first thought I must have messed up the email backend config during tests, went back to check thrice but it was correct, I added enabled logging during the test, add print statements and even added on inside send_verification_otp to see if it was being called, but no surprise, it wasn’t, went to read the Django 6.1 & Django 6.0 release notes, maybe it’s an new feature in those version. And it took several days of poking around to realize the real issue lay in how Django runs tests. It wasn’t a bug in my code; it was a feature of the test runner that I didn’t know.

As APITestCase and APITransactionTestCase map directly to TestCase and TransactionTestCase respectively, I will occasionally mix this terminology for brevity.

How does it all start?

To understand this the beginning, I had to figure out how Django test classes are built. Django’s test hierarchy stacks behavior in layers like this:

unittest.TestCase
       │
SimpleTestCase       (No database access allowed)
       │
TransactionTestCase  (Real database; resets state via SQL TRUNCATE/FLUSH)
       │
   TestCase          (Inherits TransactionTestCase; overrides reset via Transactions)

Because TestCase is a subclass of TransactionTestCase, it has all of its predecessor’s methods and attributes (duh), but this isn’t our concern, we are more interested in how it completely overrides the setup and teardown lifecycle methods (_fixture_setup and _fixture_teardown). This is where the magic—and the trap—happens.

How TransactionTestCase Works

Using TransactionTestCase, Django treats the database as a standard, persistent service. Here is the lifecycle:

Django loads any fixtures you defined. The database connection stays in standard (autocommit) mode. Any statements (e.g. Model.objects.create() runs an actual INSERT statement) that immediately commits to the database. If your code uses transaction.atomic() or triggers transaction.on_commit(), those transactions open, commit, and close against the actual database engine (that’s what I assumed was the default process for TestCase). Because records were physically written to disk, Django cannot just undo them. It runs call_command('flush'), which executes SQL TRUNCATE statements across every table to wipe all data clean.

class TransactionTestCase(SimpleTestCase):
    @classmethod
    def _fixture_setup(cls):
        for db_name in cls._databases_names(include_mirrors=False):
            ...
            if cls.fixtures:
                call_command("loaddata", *cls.fixtures, verbosity=0, database=db_name)

    def _fixture_teardown(self):
        for db_name in self._databases_names(include_mirrors=False):
            # Flush the database
            ...
            call_command(
                "flush",
                verbosity=0,
                interactive=False,
                database=db_name,
                ...
            )

And everything comes at a cost Truncating dozens or hundreds of tables and re-seeding them after every single test method causes severe disk I/O bottlenecks. That’s why we usually avoid this for large test suites.

How TestCase ruins Everything (for me)

TestCase inherits from TransactionTestCase specifically to replace that expensive table flush with lightweight transactional rollbacks, on paper that’s a great optimization, In my case it changes the everything:

TransactionTestCase: [ Test Method ] ──> Commits to DB ──> SQL TRUNCATE (Slow)
TestCase:            [ SAVEPOINT ] ──> [ Test Method ] ──> ROLLBACK (Instant)

Here is what actually happens when you run a standard test:

  1. Class-level Setup: Before any test runs in the class, TestCase opens an outer transaction.atomic() block this spans the whole class. Data created in setUpTestData() is inserted once and shared.
  2. Before each test: Instead of calling the parent’s setup, TestCase calls self._enter_atomics(). This issues a SAVEPOINT command to the database.
  3. During the test: Your code runs inside this savepoint. Even if your code explicitly uses with transaction.atomic():, Django treats it as an inner nested savepoint. No query is ever permanently committed to the database disk.
  4. After each test: TestCase skips the parent’s flush command entirely. Instead, it invokes self._rollback_atomics(), issuing a ROLLBACK TO SAVEPOINT. The database instantly discards the changes in memory.
class TestCase(TransactionTestCase):

    @classmethod
    def _enter_atomics(cls):
        """Open atomic blocks for multiple databases."""
        atomics = {}
        for db_name in cls._databases_names():
            #  That's the tricky line  
            atomic = transaction.atomic(using=db_name)  
            atomic._from_testcase = True
            atomic.__enter__()
            atomics[db_name] = atomic
        return atomics

    @classmethod
    def _rollback_atomics(cls, atomics):
        """Rollback atomic blocks opened by the previous method."""
        for db_name in reversed(cls._databases_names()):
            # Here we rollback to the save point
            transaction.set_rollback(True, using=db_name) 
            atomics[db_name].__exit__(None, None, None)

    @classmethod
    def _fixture_setup(cls):
        ...
        # The transaction.atomic() is injected here.
        cls.atomics = cls._enter_atomics()  
        if not cls._databases_support_savepoints():
            if cls.fixtures:
                for db_name in cls._databases_names(include_mirrors=False):
                    call_command(
                        "loaddata",
                        *cls.fixtures,
                        **{"verbosity": 0, "database": db_name},
                    )
            cls.setUpTestData()

    def _fixture_teardown(self):
        if not self._databases_support_transactions():
            return super()._fixture_teardown()
        try:
            for db_name in reversed(self._databases_names()):
                if self._should_check_constraints(connections[db_name]):
                    connections[db_name].check_constraints()
        finally:
            # And Here we call the rollback to save point method
            self._rollback_atomics(self.atomics)

The Side Effect: Why on_commit Died

According to the Django documentation here:

Savepoints (i.e. nested atomic() blocks) are handled correctly. That is, an on_commit() callable registered after a savepoint (in a nested atomic() block) will be called after the outer transaction is committed, but not if a rollback to that savepoint or any previous savepoint occurred during the transaction

to illustrate this more clearly, this is what happens:

with transaction.atomic():  # Outer atomic, The one applied by TestCase
    with transaction.atomic():  # Inner atomic block, yours
        transaction.on_commit(foo)

# foo() will be called when leaving the outermost block

and since we always rollback, that’s why nothing happens, pretty simple, Right? So we had three things specifically fail for me, but only one was obvious:

  • Virtual Commits vs. Real Commits: In TestCase, a line like with transaction.atomic(): doesn’t commit to the db, it creates a nested savepoint. If your production code relies on raw connection-level commits or Celery workers (like a mine did), TestCase will hide those rows from other connections because the transaction is still uncommitted.
  • The Silent Failure of on_commit: Hooks registered with transaction.on_commit() only execute when the outermost transaction successfully commits. In TransactionTestCase, the block commits normally, and the hook runs immediately. But in TestCase, the outermost transaction is never committed (it is rolled back). This means on_commit callbacks never execute by default unless manually invoked.
  • Celery reluctance to run: Tasks register with celery only executes when A) Celery is set to eager mode CELERY_TASK_ALWAYS_EAGER, in this case, it executes the function normally. B) We mock .delay(), technically we didn’t run the function we just made sure it was called successfully with correct parameters.

That was exactly why my email vanished. The test created the user, registered the callback, rolled back the whole thing, and the callback never saw the light of day.

How I Fixed It

I had three options I could do:

Option A — Switch to APITransactionTestCase

Drop-in replacement. Real commits, real on_commit hooks fire. The cost is speed — Django flushes the whole database after every test method, so this gets painful fast on large suites. Good when the test class is small or when you need full production fidelity.

class RegisterViewTestCase(APITransactionTestCase):  # was APITestCase
    def test_register_queues_otp_task(self):
        ...

Option B — captureOnCommitCallbacks() (Django 4.0+)

This is probably what you should use if you’re on a recent Django version. It keeps TestCase speed while actually executing the on_commit hooks inside the context manager:

with self.captureOnCommitCallbacks(execute=True):
    response = self.client.post(reverse("accounts:register-user"), payload, format='json')

self.assertEqual(response.status_code, status.HTTP_201_CREATED)
self.assertEqual(len(mail.outbox), 1)

I went with Option A instead because I only learned about this one while writing this post. My test class was small enough that the speed hit wasn’t a problem, and TransactionTestCase was the first thing that worked — I stopped there.

Option C — Mock on_commit directly

If you only care that the callback was registered, not that it ran, you can short-circuit the whole thing:

@patch("django.db.transaction.on_commit", side_effect=lambda f: f())
def test_register_queues_otp_task(self, mock_on_commit, mock_delay):
    ...

This calls the callback immediately when on_commit is invoked, bypassing the transaction machinery entirely. It’s the lightest option but also the furthest from production behavior — you’re not testing that the timing is right, just that the callback is wired up.

I just switched and since I didn’t have too many test in this test class and running it wasn’t too time consuming, I’d recommend you think it through before committing to a choice.

Although it wasn’t the most time-consuming mistake I made, It was pretty interesting to figure out how smartly hidden this was.

The rule I walk away with: if your code uses on_commit, your test class decides whether it fires. TestCase — it doesn’t. TransactionTestCase — it does, slowly. captureOnCommitCallbacks — it does, and you stay fast.

Three days on something that came down to one line of inheritance. Annoying at the time, but it’s the kind of thing you only have to learn once.

arrow_back All Posts