conftest.py files in packages matching "test*" are being loaded, even if they're not in the testpaths list

a detailed description of the bug or problem you are having
output of pip list from the virtual environment you are using
pytest and operating system versions
minimal example if possible

I am defining testpaths = tests in pytest.ini and that's being picked up by pytest (see pytest output below), but then it's loading a conftest.py from another directory.

Reproduction steps

Environment setup

% uname -a
Linux m 5.15.12-arch1-1 #1 SMP PREEMPT Wed, 29 Dec 2021 12:04:56 +0000 x86_64 GNU/Linux
% python --version
Python 3.10.1
% python -m venv ../venv
% ../venv/bin/pip install pytest
Collecting pytest
Using cached pytest-6.2.5-py3-none-any.whl (280 kB)
Collecting attrs>=19.2.0
Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB)
Collecting iniconfig
Using cached iniconfig-1.1.1-py2.py3-none-any.whl (5.0 kB)
Collecting packaging
Using cached packaging-21.3-py3-none-any.whl (40 kB)
Collecting toml
Using cached toml-0.10.2-py2.py3-none-any.whl (16 kB)
Collecting py>=1.8.2
Using cached py-1.11.0-py2.py3-none-any.whl (98 kB)
Collecting pluggy<2.0,>=0.12
Using cached pluggy-1.0.0-py2.py3-none-any.whl (13 kB)
Collecting pyparsing!=3.0.5,>=2.0.2
Using cached pyparsing-3.0.6-py3-none-any.whl (97 kB)
Installing collected packages: pyparsing, toml, py, pluggy, packaging, iniconfig, attrs, pytest
Successfully installed attrs-21.4.0 iniconfig-1.1.1 packaging-21.3 pluggy-1.0.0 py-1.11.0 pyparsing-3.0.6 pytest-6.2.5 toml-0.10.2
WARNING: You are using pip version 21.2.4; however, version 21.3.1 is available.
You should consider upgrading via the '[snip]/venv/bin/python -m pip install --upgrade pip' command.
% ../venv/bin/pip list
Package    Version
---------- -------
attrs      21.4.0
iniconfig  1.1.1
packaging  21.3
pip        21.2.4
pluggy     1.0.0
py         1.11.0
pyparsing  3.0.6
pytest     6.2.5
setuptools 58.1.0
toml       0.10.2
WARNING: You are using pip version 21.2.4; however, version 21.3.1 is available.
You should consider upgrading via the '[snip]/venv/bin/python -m pip install --upgrade pip' command.

The test files:

% tree
.
├── pytest.ini
├── tests
│   ├── __init__.py
│   └── test_.py
└── testsinvalid
    └── conftest.py

2 directories, 4 files

Their contents:

% find -type f | xargs -IF sh -c "echo; echo === F; cat F"

=== ./pytest.ini
[pytest]
testpaths = tests

=== ./testsinvalid/conftest.py
raise Exception('loading excluded conftest')

=== ./tests/test_.py
def test_p():
    assert True

=== ./tests/__init__.py

The command:

PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 ../venv/bin/pytest

What I expect to see

% PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 ../venv/bin/pytest
=================================================================== test session starts ====================================================================
platform linux -- Python 3.10.1, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
rootdir: /[snip], configfile: pytest.ini, testpaths: tests
collected 1 item

tests/test_.py .                                                                                                                                     [100%]

==================================================================== 1 passed in 0.00s =====================================================================

What I actually see

% PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 ../venv/bin/pytest
ImportError while loading conftest '[snip]/testsinvalid/conftest.py'.
testsinvalid/conftest.py:1: in <module>
raise Exception('loading excluded conftest')
E   Exception: loading excluded conftest

Of note

If I mv testsinvalid anything-without-the-prefix-tests, the problem goes away.

I can reproduce. No idea why it's happening, though.

@The-Compiler

@The-Compiler I narrowed it down to this block here:

pytest/src/_pytest/config/__init__.py

Lines 511 to 513 in 1569fac

    
           for x in anchor.listdir("test*"): 
        
               if x.check(dir=1): 
        
                   self._getconftestmodules(x, importmode)

But I don't know enough about the codebase to know what it should be doing.

This has been added in d632a0d, and while I agree it is surprising, it is a documented feature:

if exists, load conftest.py and test*/conftest.py relative to the directory part of the first test path.

So unless we want to somehow change this (including a deprecation period), we can close this as "worked as intended".

That commit only moves the code around - it was initially added in 7bd60b5 some 12 years ago:

check and load test*/conftest.py early from anchors -
this makes it a bit more convenient to have command line options
available from a root directory of a project that does not
directly contain a conftest.py

The testpaths option was only added 5 years later, in 854e603. It seems reasonable to me to use that setting (if set) rather than assuming test* blindly. Still probably a backwards-incompatible change though.

Reading the code, using testpaths is not right here -- the testpaths themselves are iterated and used as anchors for try_load_conftest - i.e. the idea seems to be "let's also consider test* subdirs" inside of each testpath.

Hmm, sorry, I might be confusing the variable name testpaths with the config testpaths. Disregard the above :)

This has been added in d632a0d, and while I agree it is surprising, it is a documented feature:

if exists, load conftest.py and test*/conftest.py relative to the directory part of the first test path.

So unless we want to somehow change this (including a deprecation period), we can close this as "worked as intended".

Is this really expected behaviour? I'm in doubt so I'll share my setup using pytest 7.0.0:

├── lambda_layer
│   ├── layer.zip
│   └── python
│       ├── numpy
│       │   └── conftest.py     <--
├── pytest.ini
├── src
│   ├── mycodehere.py
├── tests
│   └── test_my_test_code_here.py

pytest.ini looks like this:

[tool:pytest]
addopts = -m 'not e2e'
testpaths = tests

When I execute pytest it fails, because numpy's conftest wants to import dependencies that I don't have installed. (I don't want to install those dependencies, I'm making a lambda layer, so those files are maybe there or maybe not, but it feels like it should not load all conftests under my repo root when I have specified testpaths)
Is this really expected behaviour?

if exists, load conftest.py and test*/conftest.py relative to the directory part of the first test path

Isn't my first test path tests as specified via testpaths in my pytest.ini? I guess when reading the manual regarding testpaths it does not really say anything about conftest loading behaviour. What to do with this?

@dhensen

Is this really expected behaviour?

@dhensen this seems like a different problem, because lambda_layer doesn't match tests*.

@dhensen

@dhensen, in your pytest.ini, it should be only [pytest] in the header, [tool:pytest] is only used for setup.cfg files.

I'm not sure why this is a feature though because I would expect running pytest tests/ and using testpaths = tests when running pytest without any arguments to have the same behaviour, I thought that was the point of testpaths.

I'm not sure why this is a feature though because I would expect running pytest tests/ and using testpaths = tests when running pytest without any arguments to have the same behaviour, I thought that was the point of testpaths.

I agree, seems like that feature was introduced prior to testpaths.

I guess it is possible to deprecate the behavior: we can detect a conftest.py was matched via test*/conftest.py, and issue a deprecation warning.

Closing as resolved: testpaths is now honored when determining which test*/conftest.py files to load at startup, implemented in #10988 (and refined by the regression fix in #11104 so unrelated sibling test* directories aren't scanned).

Navigation Menu

Repository navigation

conftest.py files in packages matching "test*" are being loaded, even if they're not in the `testpaths` list #9469

Reproduction steps

What I expect to see

What I actually see

Of note

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	for x in anchor.listdir("test*"):
	if x.check(dir=1):
	self._getconftestmodules(x, importmode)

conftest.py files in packages matching "test*" are being loaded, even if they're not in the testpaths list #9469

Description

Reproduction steps

What I expect to see

What I actually see

Of note

Activity

The-Compiler commented on Jan 3, 2022

mattalxndr commented on Jan 3, 2022

nicoddemus commented on Jan 4, 2022

The-Compiler commented on Jan 4, 2022

bluetech commented on Jan 4, 2022

bluetech commented on Jan 4, 2022

dhensen commented on Feb 14, 2022

nicoddemus commented on Feb 14, 2022

nicoddemus commented on Feb 14, 2022

JamieMcKernanKaizen commented on Mar 16, 2022

nicoddemus commented on Mar 16, 2022

Zac-HD commented on Jul 22, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

conftest.py files in packages matching "test*" are being loaded, even if they're not in the `testpaths` list #9469