Separate app context for test client requests

[torotil]

Hi, I’ve been using flask, flask-sqlalchemy together with pytest written unit tests for a while. With the upgrade to flask-sqlalchemy 3.0 and flask 2.2 I’m now unable to get my tests working because they fail with a “AssertionError: Popped wrong app context.” error that seems to be related to 84c722044.

I’ve been using the test-wide app context as described in the flask-sqlalchemy documentation

import pytest

@pytest.mark.fixture
def app_ctx(app):
    with app.app_context():
        yield

In order to separate the DB session for the test method from anything executed within a test request I’m also using a custom test client that wraps each test request into its own app_context.

class Client(flask.testing.FlaskClient):
    """Flask test client that opens a new DB session for each request."""

    def open(self, *args, **kwargs):
        """Use a specific app-context for each request so it gets its own db session."""
        with self.application.app_context():
            return super().open(*args, **kwargs)

The “Popped wrong app context.” occurs when I issue two requests within the same test method:

def test_client(app):
    with app.test_client() as client:
        client.get("/test")  # works fine
        client.get("/test2")  # error

I’ve added some debug output to flask/testing.py in order to see what’s happening and this is what I got:

PUSH <flask.ctx.AppContext object at 0x7fa5343659d0>  # first request, with app.app_context start
PUSH <flask.ctx.AppContext object at 0x7fa5343659d0>  # first request, normal enter_context
POP  <flask.ctx.AppContext object at 0x7fa5343659d0> # first request, with app.app_context end
PUSH <flask.ctx.AppContext object at 0x7fa5342c9ac0> # second request, with app.app_context start
POP  <flask.ctx.AppContext object at 0x7fa5343659d0> # second request, clean up old contexts ---> error
POP  <flask.ctx.AppContext object at 0x7fa5342c9ac0> # sexend request, with app.app_context stop

At this point I’m not even sure why the test clients stack handling interferes with my app context.

Any suggestions?

[davidism]

Neither of the code blocks you've shown were correct before, they just happened to work by accident before.

You do not need to override Client.open to push an app context. An app context is automatically pushed and managed by Flask when it receives a request. If your tests aren't working without that, it suggests an issue with something else in your tests.

Do not push app.test_client (with a with block) unless you actually need to preserve the context after the request ends. You most likely are not asserting anything about the request's context after the request, so you don't need to push the client.

[davidism]

that’s only the case if no app context exists yet

That's correct, and that's what you want.

This means fixtures, test method and the test request share the same DB session which has lead to a lot of weird failures

Yes, that's what you're causing right now by unconditionally pushing app contexts for every test. If you just make requests, and remove the with app_context, each request will have its own context.

I tried removing the app context and the tests failed again (due to a different error).

Yes, those are the actual errors you should be addressing.

[torotil]

Yes, that's what you're causing right now by unconditionally pushing app contexts for every test. If you just make requests, and remove the with app_context, each request will have its own context.

The reason for pushing the app_contexts is that I use flask-sqlalchemy.. It needs them in order to maintain a DB session, which I need in turn for every DB fixture I have. In a nutshell: Without a test-wide app_context writing tests with flask-sqlalchemy becomes quite tedious.

It seems to be kind of a stalemate: flask-sqlalchemy needs one, flask doesn’t want one. 🤷 The above is my workaround for it.

[torotil]

Note that in my case there is also Celery which again needs an app context (in order to use a DB session) to execute some of the tasks. During tests these tasks are executed synchronously. Usually tasks are triggered when handling requests, so this again creates nested app contexts …

[davidism]

It needs them in order to maintain a DB session, which I need in turn for every DB fixture I have

You shouldn't need this. There's no reason not to use a different session for each test, it would actually be bad to have one session (which is different than a connection) across requests.

[torotil]

I do use a different session for each test. The problem is that the same session by default also spans the test requests within the test.

Essentially there is these phases during each single test:

1. fixtures + setup
2. test code before request
3. test request
4. test code after the request
5. teardown

It’s very convenient if everything except 3. shares the same DB session and I don’t have to create one in every single stage.