Repository navigation
--import-mode=importlib breaks assertion rewriting #12044
Description
Activity
- changed the title
[-]Bad python module name breaks assertion rewriting[/-][+]--import-mode=importlib breaks assertion rewriting[/+]on Mar 12, 2024 Ok I was missing that after turning importlib back on, the issue returns, so in fact rewriting fails when importlib is turned on.
We need to verify what's up here
I suspect we are missing a bit where import lib will correctly fail for invalid module names, thus the assertions rewriter does as well, and then we incorrectly fall back to normal module loading
Most likely we try to find a test module inside a folder after not doing a sanity check on the filename, after that fails the normal import lib machinery takes over finding the module by path and working
I won't be able to verify before next week myself
I am having this issue as well. It seems that it only happens if I have my tests in a package:
root/ my_module/ __init__.py file.py tests/ __init__.py test_main.pyIf I remove
__init__.pyin the tests folder the assert rewriting works even in importlib mode. I should also note that my test files are valid module names, unlike the OP's.Reacted by Oskar Haarklou Veileborg, Thomas Liebetraut, Dovave Maier, mwchase and Jason R. CoombsI've had the same issue, the fix from @estyrke works, the test directories were erroneously packages, removing the
__init__.pys fixes assertion rewriting.Reacted by Noon van der SilkHitting the same problems, we keep out tests in the packages, but we need importlib to work-around issues with namespaced package. Keeping our tests inside the namespaced package, even by removing the
__init__.pyfrom the test folder breaks assertion rewriting when we upgraded from 8.0.2 to 8.2.0.Moving the tests outside the namespace package fix it, but it's an unfortunate limitation.
Actually, it only keep happening with the config:
[pytest] consider_namespace_packages = trueWithout this, removing the
__init__.pyfrom a test folder fix it.I'd noticed this issue recently in a number of my projects. I observed the issue most recently in the cssutils project when another user's bug report was missing helpful detail.
cssutils's tests are found incssutils/tests/.I was able to reproduce the issue simply with:
draft @ tree . └── cssutils ├── __init__.py └── test_simple.py 2 directories, 2 files draft @ cat cssutils/* class val: val = 1 def test_something_simple(): assert val.val == 0 draft @ pip-run pytest -- -m pytest --import-mode importlib cssutils/test_simple.py ============================================================== test session starts =============================================================== platform darwin -- Python 3.12.3, pytest-8.2.1, pluggy-1.5.0 rootdir: /Users/jaraco/draft collected 1 item cssutils/test_simple.py F [100%] ==================================================================== FAILURES ==================================================================== _____________________________________________________________ test_something_simple ______________________________________________________________ def test_something_simple(): > assert val.val == 0 E AssertionError cssutils/test_simple.py:6: AssertionError ============================================================ short test summary info ============================================================= FAILED cssutils/test_simple.py::test_something_simple - AssertionError =============================================================== 1 failed in 0.01s ================================================================
Moving the test to the root, removing the
__init__.py, not using--import-mode importlib, or downgrading topytest<8.1restores the assert rewrites.draft [1] @ pip-run 'pytest<8.1' -- -m pytest --import-mode importlib cssutils/test_simple.py ============================================================== test session starts =============================================================== platform darwin -- Python 3.12.3, pytest-8.0.2, pluggy-1.5.0 rootdir: /Users/jaraco/draft collected 1 item cssutils/test_simple.py F [100%] ==================================================================== FAILURES ==================================================================== _____________________________________________________________ test_something_simple ______________________________________________________________ def test_something_simple(): > assert val.val == 0 E assert 1 == 0 E + where 1 = val.val cssutils/test_simple.py:6: AssertionError ============================================================ short test summary info ============================================================= FAILED cssutils/test_simple.py::test_something_simple - assert 1 == 0 =============================================================== 1 failed in 0.01s ================================================================
In my projects, I'm unable to use any of those workarounds, so I'm hoping there's a more permanent fix in the works.
I just wasted about 8 hours trying to understand why I was no longer getting assertions rewritten and useful assertion failure output printed. Only after an ugly
git bisect, staring at the culprit commit for another hour, and removing__init__.pyfiles below thetests/directory on a whim, did it start working again.I wish the importance of not having
__init__.pyfiles with--import-mode=importlibwas called out in the docs at https://docs.pytest.org/en/stable/explanation/goodpractices.html#test-layout.@jmehnle as a pytest user trying to understand why somehow we lost pytest assertion rewriting in our project, and bumping into this issue (which is probably unrelated by the way but I need to continue my journey to be sure ...), I can definitely relate to the frustration.
As a open-source project maintainer (not a Pytest maintainer just to be 100% clear), I wish people making this kind of comments would understand how draining this is in the long run for maintainers, in particular this part:
I just wasted about 8 hours trying to understand why I was no longer getting assertions rewritten and useful assertion failure output printed.
On this kind of topics, I definitely recommend Settings Expectation for open-source participation https://snarky.ca/setting-expectations-for-open-source-participation/. There are some videos of the talk you can find online as well if you prefer video.
Reacted by Oliver Bestwalter, Freya Bruhin and Jason R. Coombs
Using
--import-mode=importlibis more lenient than the defaultimport-mode, and so it allowed me to use*.test.pyrather than a valid module name e.g.*_.test.py.Everything seemed to work correctly except explanations were missing:
instead of