Repository navigation
weird behavior with tuples indirect parameterization #8281
Description
Activity
I have it with different python and pytest versions and the behavior between the versions is inconsistent
- I have added additional test_four inside the class exactly the same as test_three
- Python version 3.9 behaves the same as python 3.8
- Pytest version 5.4.3 behaves the same as version 6.2.2
Looks like pytest 4.6.11 is the only one that behave consistently in different python versions
import pytest @pytest.fixture(scope="session") def something(request): print(f'\n\tinitialized with {request.param}') return request.param @pytest.mark.parametrize('something', ['abcd', (1,2), [1,2]], indirect=True) def test_one(something): print(f'test_one {something}') @pytest.mark.parametrize('something', ['abcd', (1,2), [1,2]], indirect=True) def test_two(something): print(f'test_two {something}') class TestSomething: @pytest.mark.parametrize('something', ['abcd', (1,2), [1,2]], indirect=True) def test_three(self, something): print(f'test_three {something}') @pytest.mark.parametrize('something', ['abcd', (1,2), [1,2]], indirect=True) def test_four(self, something): print(f'test_four {something}')
============================= test session starts ============================== platform linux -- Python 3.6.12, pytest-4.6.11, py-1.10.0, pluggy-0.13.1 rootdir: /home collected 12 items test_something.py initialized with abcd test_one abcd .test_two abcd .test_three abcd .test_four abcd . initialized with (1, 2) test_one (1, 2) .test_two (1, 2) .test_three (1, 2) .test_four (1, 2) . initialized with [1, 2] test_one [1, 2] .test_two [1, 2] .test_three [1, 2] .test_four [1, 2] . ========================== 12 passed in 0.06 seconds ======================================================== test session starts ============================== platform linux -- Python 3.6.12, pytest-6.2.2, py-1.10.0, pluggy-0.13.1 rootdir: /home collected 12 items test_something.py initialized with abcd test_one abcd .test_two abcd .test_three abcd .test_four abcd . initialized with (1, 2) test_one (1, 2) . initialized with (1, 2) test_two (1, 2) . initialized with (1, 2) test_three (1, 2) . initialized with (1, 2) test_four (1, 2) . initialized with [1, 2] test_one [1, 2] . initialized with [1, 2] test_two [1, 2] . initialized with [1, 2] test_three [1, 2] . initialized with [1, 2] test_four [1, 2] . ============================== 12 passed in 0.04s =========================================================== test session starts ============================== platform linux -- Python 3.7.9, pytest-4.6.11, py-1.10.0, pluggy-0.13.1 rootdir: /home collected 12 items test_something.py initialized with abcd test_one abcd .test_two abcd .test_three abcd .test_four abcd . initialized with (1, 2) test_one (1, 2) .test_two (1, 2) .test_three (1, 2) .test_four (1, 2) . initialized with [1, 2] test_one [1, 2] .test_two [1, 2] .test_three [1, 2] .test_four [1, 2] . ========================== 12 passed in 0.05 seconds ======================================================== test session starts ============================== platform linux -- Python 3.7.9, pytest-6.2.2, py-1.10.0, pluggy-0.13.1 rootdir: /home collected 12 items test_something.py initialized with abcd test_one abcd .test_two abcd .test_three abcd .test_four abcd . initialized with (1, 2) test_one (1, 2) .test_two (1, 2) . initialized with (1, 2) test_three (1, 2) .test_four (1, 2) . initialized with [1, 2] test_one [1, 2] . initialized with [1, 2] test_two [1, 2] . initialized with [1, 2] test_three [1, 2] . initialized with [1, 2] test_four [1, 2] . ============================== 12 passed in 0.06s =========================================================== test session starts ============================== platform linux -- Python 3.8.7, pytest-4.6.11, py-1.10.0, pluggy-0.13.1 rootdir: /home collected 12 items test_something.py initialized with abcd test_one abcd .test_two abcd .test_three abcd .test_four abcd . initialized with (1, 2) test_one (1, 2) .test_two (1, 2) .test_three (1, 2) .test_four (1, 2) . initialized with [1, 2] test_one [1, 2] .test_two [1, 2] .test_three [1, 2] .test_four [1, 2] . ========================== 12 passed in 0.08 seconds ======================================================== test session starts ============================== platform linux -- Python 3.8.7, pytest-6.2.2, py-1.10.0, pluggy-0.13.1 rootdir: /home collected 12 items test_something.py initialized with abcd test_one abcd .test_two abcd .test_three abcd .test_four abcd . initialized with (1, 2) test_one (1, 2) .test_two (1, 2) .test_three (1, 2) .test_four (1, 2) . initialized with [1, 2] test_one [1, 2] . initialized with [1, 2] test_two [1, 2] . initialized with [1, 2] test_three [1, 2] . initialized with [1, 2] test_four [1, 2] . ============================== 12 passed in 0.04s ==============================I haven't read the issue in detail yet, but this might be similar to what's discussed here: #8102 (comment)
Specifically, pytest uses
isto check whether to reexecute a fixture. And python does not guarantee that two equal tuples will have the same identity. You can check what happens if you pull out the tuple to a global variable e.g.TUPLE = (1, 2)and replace all of the distinct(1, 2)s with it.Closing as resolved by #12600 (released in pytest 8.3.0): fixture cache-key comparison was changed from
is(object identity) back to==(value equality, with anisfallback only if==raises), so an equal-but-distinct(1, 2)tuple across indirect-parametrize calls now hits the cache instead of re-initializing the fixture, exactly as this issue described.
I found the combination of python 3.7 and current version of pytest (6.2.2) to act weird among indirect parameterization with tuples.
The following test:
Outputs:
The fixture was created twice with the same parameter (1, 2) although it is scoped to the session.
I've tested this in python 3.8 also, and there the fixture is initialized once per parameter (as expected).