Test timeout via marker kwargs

[fjetter]

At dask.distributed we started using pytest-asyncio for increasingly many tests. Before this, we've been decorating our plain asyncio tests using our own decorator which is not unlike to what pytest-asyncio is using under the hood, see https://github.com/dask/distributed/blob/833c5f6c040feaa4550fa343d8e6e4feef3f84d5/distributed/utils_test.py#L753-L772 but that has the obvious shortcoming that it doesn't integrate that nicely into pytest.

For our use case the timeout argument is very important since many tests are waiting until certain conditions are true. Moving to pytest-asyncio required us to use https://pypi.org/project/pytest-timeout/ but for portability reasons this plugin terminates the entire test process if a single test timeouts (see thread vs signal in their docs) which is not great.

A better integration with asyncio would be relatively easy to achieve and would allow us to improve our integration with pytest further. The syntax I had in mind is simply

@pytest.mark.asyncio(timeout=30)
async def test_requires_timeout():
    await stuff()

which, under the hood, would wrap the coro with https://docs.python.org/3/library/asyncio-task.html#asyncio.wait_for and would raise an asyncio.TimeoutError if the test runs for too long.
A default value for this timeout could be provided via a configuration file or similar.

If this overlap in scope is acceptable for this repo I would prepare and open a PR.

[Tinche]

Interesting, I'd be open to something like that!

[johnthagen]

Wanted to echo this comment from the related PR:

• Add timeout options #216 (comment)

It would be important to also allow disabling the timeout during an interactive debugging session, otherwise if you step into a function that is awaited and stop at a break point, the timeout will fire by the time you return causing the rest of the test to fail.