Pytest's self tests are failing with pygments v2.19.0

@The-Compiler

a detailed description of the bug or problem you are having
output of pip list from the virtual environment you are using
pytest and operating system versions
minimal example if possible

Description

When running tox, I am seeing 4 test failures in test_terminal.py. I am not seeing these errors when I pin pygments>=2.7.2,<2.19.0. I suspect pygments is giving invalid control characters for resetting text formatting. I'm going to open a issue over there in a second.

I'm using Centos, but have also confirmed the issue on ubuntu 20 and debian 12. I've also confirmed it on python 3.13, 3.12, and 3.8.

Specific tests failing:

=================================================================================== short test summary info ===================================================================================
FAILED testing/test_terminal.py::test_color_yes - Failed: fnmatch: '\x1b[[]1m=*= test session starts =*=\x1b[[]0m'
FAILED testing/test_terminal.py::TestCodeHighlight::test_code_highlight_simple - Failed: nomatch: '    \x1b[[]0m\x1b[[]94mdef\x1b[[]39;49;00m \x1b[[]92mtest_foo\x1b[[]39;49;00m():\x1b[[]90m\x1b[[]39;49;00m'
FAILED testing/test_terminal.py::TestCodeHighlight::test_code_highlight_continuation - Failed: nomatch: '    \x1b[[]0m\x1b[[]94mdef\x1b[[]39;49;00m \x1b[[]92mtest_foo\x1b[[]39;49;00m():\x1b[[]90m\x1b[[]39;49;00m'
FAILED testing/test_terminal.py::TestCodeHighlight::test_code_highlight_custom_theme - Failed: nomatch: '    \x1b[[]0m\x1b[[]94mdef\x1b[[]39;49;00m \x1b[[]92mtest_foo\x1b[[]39;49;00m():\x1b[[]90m\x1b[[]39;49;00m'
XPASS testing/_py/test_local.py::TestLocalPath::test_make_numbered_dir_multiprocess_safe - #11603

How to recreate

Remove .tox folder to ensure clean run with no cached inputs
Run tox -e py312

Tox output

Details

<user>@:~/git-workspaces/pytest$ uv tool run --python=3.12 tox run -e py312 --list-dependencies                                                                              
.pkg: install_requires> python -I -m pip install 'setuptools-scm[toml]>=6.2.3' 'setuptools>=61'
.pkg: _optional_hooks> python /home/<user>/.cache/uv/archive-v0/CQu4szLQA3RcUPX2NK6AM/lib/python3.12/site-packages/pyproject_api/_backend.py True setuptools.build_meta
.pkg: get_requires_for_build_sdist> python /home/<user>/.cache/uv/archive-v0/CQu4szLQA3RcUPX2NK6AM/lib/python3.12/site-packages/pyproject_api/_backend.py True setuptools.build_meta
.pkg: get_requires_for_build_editable> python /home/<user>/.cache/uv/archive-v0/CQu4szLQA3RcUPX2NK6AM/lib/python3.12/site-packages/pyproject_api/_backend.py True setuptools.build_meta
.pkg: freeze> python -m pip freeze --all
.pkg: packaging==24.2,pip==24.3.1,setuptools==75.7.0,setuptools-scm==8.1.0
.pkg: build_sdist> python /home/<user>/.cache/uv/archive-v0/CQu4szLQA3RcUPX2NK6AM/lib/python3.12/site-packages/pyproject_api/_backend.py True setuptools.build_meta
py312: install_package_deps> python -I -m pip install argcomplete 'attrs>=19.2' 'colorama; sys_platform == "win32"' 'exceptiongroup>=1.0.0rc8; python_version < "3.11"' 'hypothesis>=3.56' iniconfig mock packaging 'pluggy<2,>=1.5' 'pygments>=2.7.2' requests setuptools 'tomli>=1; python_version < "3.11"' xmlschema
py312: install_package> python -I -m pip install --force-reinstall --no-deps /home/<user>/git-workspaces/pytest/.tox/.tmp/package/1/pytest-8.3.4.tar.gz
py312: freeze> python -m pip freeze --all
py312: argcomplete==3.5.3,attrs==24.3.0,certifi==2024.12.14,charset-normalizer==3.4.1,elementpath==4.7.0,hypothesis==6.123.4,idna==3.10,iniconfig==2.0.0,mock==5.1.0,packaging==24.2,pip==24.3.1,pluggy==1.5.0,Pygments==2.19.1,pytest @ file:///home/<user>/git-workspaces/pytest/.tox/.tmp/package/1/pytest-8.3.4.tar.gz#sha256=9026cce8eccca1572dc638b9bb2696fff62816778492485d39b9d9346b566833,requests==2.32.3,setuptools==75.7.0,sortedcontainers==2.4.0,urllib3==2.3.0,xmlschema==3.4.3
py312: commands[0]> pytest
===================================================================================== test session starts =====================================================================================
platform linux -- Python 3.12.5, pytest-8.3.4, pluggy-1.5.0
cachedir: .tox/py312/.pytest_cache
rootdir: /home/<user>/git-workspaces/pytest
configfile: pyproject.toml
testpaths: testing
plugins: hypothesis-6.123.4
collected 3753 items                                                                                                                                                                          

testing/_py/test_local.py .................................................................ss..............s.................x.....Xs.....................................sssssss...... [  4%]
......................x...                                                                                                                                                              [  4%]
testing/acceptance_test.py .....                                                                                                                                                        [  5%]
testing/code/test_code.py .................                                                                                                                                             [  5%]
testing/code/test_excinfo.py .................s.......s..............................................................................................................................   [  9%]
testing/code/test_source.py .......................................................                                                                                                     [ 10%]
testing/deprecated_test.py ..........                                                                                                                                                   [ 11%]
testing/freeze/tests/test_doctest.txt .                                                                                                                                                 [ 11%]
testing/freeze/tests/test_trivial.py ..                                                                                                                                                 [ 11%]
testing/io/test_pprint.py ......................................                                                                                                                        [ 12%]
testing/io/test_saferepr.py .............                                                                                                                                               [ 12%]
testing/io/test_terminalwriter.py .................................................                                                                                                     [ 13%]
testing/io/test_wcwidth.py .................                                                                                                                                            [ 14%]
testing/logging/test_fixture.py ...................                                                                                                                                     [ 14%]
testing/logging/test_formatter.py ....                                                                                                                                                  [ 15%]
testing/logging/test_reporting.py ..                                                                                                                                                    [ 15%]
testing/python/approx.py .sss..sssss.......................................sssssss................ss..........                                                                          [ 17%]
testing/python/collect.py ..                                                                                                                                                            [ 17%]
testing/python/fixtures.py ...........                                                                                                                                                  [ 17%]
testing/python/integration.py ....                                                                                                                                                      [ 17%]
testing/python/metafunc.py .........................................                                                                                                                    [ 18%]
testing/python/raises.py ........................                                                                                                                                       [ 19%]
testing/test_argcomplete.py ..                                                                                                                                                          [ 19%]
testing/test_assertion.py .........................................................................                                                                                     [ 21%]
testing/test_assertrewrite.py ................................................                                                                                                          [ 22%]
testing/test_capture.py .............................................................s..                                                                                                [ 24%]
testing/test_collection.py ..                                                                                                                                                           [ 24%]
testing/test_compat.py ...........                                                                                                                                                      [ 24%]
testing/test_config.py .............................................................                                                                                                    [ 26%]
testing/test_conftest.py ...........                                                                                                                                                    [ 26%]
testing/test_debugging.py .                                                                                                                                                             [ 26%]
testing/test_doctest.py .............                                                                                                                                                   [ 27%]
testing/test_entry_points.py .                                                                                                                                                          [ 27%]
testing/test_faulthandler.py ...                                                                                                                                                        [ 27%]
testing/test_findpaths.py ..............s...                                                                                                                                            [ 27%]
testing/test_helpconfig.py .                                                                                                                                                            [ 27%]
testing/test_junitxml.py ....                                                                                                                                                           [ 27%]
testing/test_legacypath.py ....                                                                                                                                                         [ 28%]
testing/test_main.py .....                                                                                                                                                              [ 28%]
testing/test_mark.py ..............                                                                                                                                                     [ 28%]
testing/test_mark_expression.py ............................................................................................................................................            [ 32%]
testing/test_monkeypatch.py ...............................                                                                                                                             [ 33%]
testing/test_nodes.py ....                                                                                                                                                              [ 33%]
testing/test_parseopt.py .............................                                                                                                                                  [ 33%]
testing/test_pastebin.py ....                                                                                                                                                           [ 34%]
testing/test_pluginmanager.py .........                                                                                                                                                 [ 34%]
testing/test_pytester.py .....................                                                                                                                                          [ 34%]
testing/test_recwarn.py .........................................................                                                                                                       [ 36%]
testing/test_runner.py ...............                                                                                                                                                  [ 36%]
testing/test_scope.py ....                                                                                                                                                              [ 36%]
testing/test_skipping.py .                                                                                                                                                              [ 36%]
testing/test_stash.py .                                                                                                                                                                 [ 36%]
testing/test_terminal.py .......................................                                                                                                                        [ 37%]
testing/test_tmpdir.py ...s...............                                                                                                                                              [ 38%]
testing/test_warning_types.py ..............                                                                                                                                            [ 38%]
testing/test_warnings.py .                                                                                                                                                              [ 38%]
testing/acceptance_test.py ......................................................x....................s.                                                                                [ 40%]
testing/code/test_excinfo.py .............sssssssss......                                                                                                                               [ 41%]
testing/deprecated_test.py ....                                                                                                                                                         [ 41%]
testing/examples/test_issue519.py .                                                                                                                                                     [ 41%]
testing/logging/test_fixture.py ........                                                                                                                                                [ 42%]
testing/logging/test_reporting.py ..................................................                                                                                                    [ 43%]
testing/python/approx.py .                                                                                                                                                              [ 43%]
testing/python/collect.py ...............................................................................                                                                               [ 45%]
testing/python/fixtures.py ........................................................................x................................................................................... [ 49%]
.....................s.......                                                                                                                                                           [ 50%]
testing/python/integration.py ...............                                                                                                                                           [ 50%]
testing/python/metafunc.py .................................................................                                                                                            [ 52%]
testing/python/raises.py ....                                                                                                                                                           [ 52%]
testing/python/show_fixtures_per_test.py ........                                                                                                                                       [ 52%]
testing/test_assertion.py ................................................                                                                                                              [ 54%]
testing/test_assertrewrite.py ..................................................................                                                                                        [ 55%]
testing/test_cacheprovider.py .........................................................                                                                                                 [ 57%]
testing/test_capture.py ......x....................................                                                                                                                     [ 58%]
testing/test_collection.py ................x............................................s................s..                                                                            [ 60%]
testing/test_compat.py ..                                                                                                                                                               [ 60%]
testing/test_config.py ..................x......................................................................................................................                        [ 64%]
testing/test_conftest.py .............s.............................                                                                                                                    [ 65%]
testing/test_debugging.py .............                                                                                                                                                 [ 65%]
testing/test_doctest.py ................................................................x...............................................................                                [ 69%]
testing/test_error_diffs.py ............                                                                                                                                                [ 69%]
testing/test_faulthandler.py ...                                                                                                                                                        [ 69%]
testing/test_helpconfig.py ........                                                                                                                                                     [ 69%]
testing/test_junitxml.py ..................................................................................................s...s........................                                [ 73%]
testing/test_legacypath.py ..........                                                                                                                                                   [ 73%]
testing/test_link_resolve.py .                                                                                                                                                          [ 73%]
testing/test_main.py ..............                                                                                                                                                     [ 73%]
testing/test_mark.py ......................................................................x.................                                                                           [ 76%]
testing/test_monkeypatch.py ....                                                                                                                                                        [ 76%]
testing/test_nodes.py ....                                                                                                                                                              [ 76%]
testing/test_parseopt.py .                                                                                                                                                              [ 76%]
testing/test_pastebin.py ...                                                                                                                                                            [ 76%]
testing/test_pathlib.py ............................................................s.........................................                                                          [ 79%]
testing/test_pluginmanager.py ....s............                                                                                                                                         [ 79%]
testing/test_pytester.py x........................                                                                                                                                      [ 80%]
testing/test_python_path.py .....                                                                                                                                                       [ 80%]
testing/test_recwarn.py .....                                                                                                                                                           [ 80%]
testing/test_reports.py ..................                                                                                                                                              [ 81%]
testing/test_runner.py ........................x...........................                                                                                                             [ 82%]
testing/test_runner_xunit.py .............                                                                                                                                              [ 82%]
testing/test_session.py ..........................                                                                                                                                      [ 83%]
testing/test_setuponly.py ..........................                                                                                                                                    [ 84%]
testing/test_setupplan.py ...                                                                                                                                                           [ 84%]
testing/test_skipping.py .........................................................................................                                                                      [ 86%]
testing/test_stepwise.py ..............                                                                                                                                                 [ 87%]
testing/test_terminal.py .................................................s..............F..........................................sss......s....FFF......................             [ 91%]
testing/test_threadexception.py ....                                                                                                                                                    [ 91%]
testing/test_tmpdir.py ....................                                                                                                                                             [ 91%]
testing/test_unittest.py ......................sssssss.................................s............                                                                                    [ 93%]
testing/test_unraisableexception.py ....                                                                                                                                                [ 93%]
testing/test_warning_types.py .                                                                                                                                                         [ 93%]
testing/test_warnings.py ...................sss..........                                                                                                                               [ 94%]
testing/acceptance_test.py .....                                                                                                                                                        [ 94%]
testing/python/collect.py .                                                                                                                                                             [ 94%]
testing/python/fixtures.py ..                                                                                                                                                           [ 94%]
testing/test_assertion.py ............                                                                                                                                                  [ 95%]
testing/test_assertrewrite.py .........                                                                                                                                                 [ 95%]
testing/test_capture.py ........................                                                                                                                                        [ 96%]
testing/test_collection.py ...                                                                                                                                                          [ 96%]
testing/test_compat.py s                                                                                                                                                                [ 96%]
testing/test_config.py ..                                                                                                                                                               [ 96%]
testing/test_debugging.py sssssssssss.ssssssssssssssss.sss....ssss.sss                                                                                                                  [ 97%]
testing/test_faulthandler.py ..s.                                                                                                                                                       [ 97%]
testing/test_helpconfig.py ..                                                                                                                                                           [ 97%]
testing/test_legacypath.py .                                                                                                                                                            [ 97%]
testing/test_meta.py ...................................................................                                                                                                [ 99%]
testing/test_pytester.py ....s                                                                                                                                                          [ 99%]
testing/test_recwarn.py .                                                                                                                                                               [ 99%]
testing/test_reports.py .                                                                                                                                                               [ 99%]
testing/test_terminal.py ss..                                                                                                                                                           [ 99%]
testing/test_unittest.py s.                                                                                                                                                             [ 99%]
testing/test_warnings.py .........                                                                                                                                                      [100%]

========================================================================================== FAILURES ===========================================================================================
_______________________________________________________________________________________ test_color_yes ________________________________________________________________________________________

pytester = <Pytester PosixPath('/tmp/pytest-of-<user>/pytest-19/test_color_yes0')>, color_mapping = <class 'conftest.color_mapping.<locals>.ColorMapping'>

    def test_color_yes(pytester: Pytester, color_mapping) -> None:
        p1 = pytester.makepyfile(
            """
            def fail():
                assert 0
    
            def test_this():
                fail()
            """
        )
        result = pytester.runpytest("--color=yes", str(p1))
>       result.stdout.fnmatch_lines(
            color_mapping.format_for_fnmatch(
                [
                    "{bold}=*= test session starts =*={reset}",
                    "collected 1 item",
                    "",
                    "test_color_yes.py {red}F{reset}{red} * [100%]{reset}",
                    "",
                    "=*= FAILURES =*=",
                    "{red}{bold}_*_ test_this _*_{reset}",
                    "",
                    "    {reset}{kw}def{hl-reset} {function}test_this{hl-reset}():{endline}",
                    ">       fail(){endline}",
                    "",
                    "{bold}{red}test_color_yes.py{reset}:5: ",
                    "_ _ * _ _*",
                    "",
                    "    {reset}{kw}def{hl-reset} {function}fail{hl-reset}():{endline}",
                    ">       {kw}assert{hl-reset} {number}0{hl-reset}{endline}",
                    "{bold}{red}E       assert 0{reset}",
                    "",
                    "{bold}{red}test_color_yes.py{reset}:2: AssertionError",
                    "{red}=*= {red}{bold}1 failed{reset}{red} in *s{reset}{red} =*={reset}",
                ]
            )
        )
E       Failed: fnmatch: '\x1b[[]1m=*= test session starts =*=\x1b[[]0m'
E          with: '\x1b[1m============================= test session starts ==============================\x1b[0m'
E       nomatch: 'collected 1 item'
E           and: 'platform linux -- Python 3.12.5, pytest-8.3.4, pluggy-1.5.0'
E           and: 'rootdir: /tmp/pytest-of-<user>/pytest-19/test_color_yes0'
E       exact match: 'collected 1 item'
E       exact match: ''
E       fnmatch: 'test_color_yes.py \x1b[[]31mF\x1b[[]0m\x1b[[]31m * [[]100%]\x1b[[]0m'
E          with: 'test_color_yes.py \x1b[31mF\x1b[0m\x1b[31m                                                      [100%]\x1b[0m'
E       exact match: ''
E       fnmatch: '=*= FAILURES =*='
E          with: '=================================== FAILURES ==================================='
E       fnmatch: '\x1b[[]31m\x1b[[]1m_*_ test_this _*_\x1b[[]0m'
E          with: '\x1b[31m\x1b[1m__________________________________ test_this ___________________________________\x1b[0m'
E       exact match: ''
E       nomatch: '    \x1b[[]0m\x1b[[]94mdef\x1b[[]39;49;00m \x1b[[]92mtest_this\x1b[[]39;49;00m():\x1b[[]90m\x1b[[]39;49;00m'
E           and: '    \x1b[0m\x1b[94mdef\x1b[39;49;00m\x1b[90m \x1b[39;49;00m\x1b[92mtest_this\x1b[39;49;00m():\x1b[90m\x1b[39;49;00m'
E           and: '>       fail()\x1b[90m\x1b[39;49;00m'
E           and: ''
E           and: '\x1b[1m\x1b[31mtest_color_yes.py\x1b[0m:5: '
E           and: '_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ '
E           and: ''
E           and: '    \x1b[0m\x1b[94mdef\x1b[39;49;00m\x1b[90m \x1b[39;49;00m\x1b[92mfail\x1b[39;49;00m():\x1b[90m\x1b[39;49;00m'
E           and: '>       \x1b[94massert\x1b[39;49;00m \x1b[94m0\x1b[39;49;00m\x1b[90m\x1b[39;49;00m'
E           and: '\x1b[1m\x1b[31mE       assert 0\x1b[0m'
E           and: ''
E           and: '\x1b[1m\x1b[31mtest_color_yes.py\x1b[0m:2: AssertionError'
E           and: '\x1b[36m\x1b[1m=========================== short test summary info ============================\x1b[0m'
E           and: '\x1b[31mFAILED\x1b[0m test_color_yes.py::\x1b[1mtest_this\x1b[0m - assert 0'
E           and: '\x1b[31m============================== \x1b[31m\x1b[1m1 failed\x1b[0m\x1b[31m in 0.01s\x1b[0m\x1b[31m ===============================\x1b[0m'
E       remains unmatched: '    \x1b[[]0m\x1b[[]94mdef\x1b[[]39;49;00m \x1b[[]92mtest_this\x1b[[]39;49;00m():\x1b[[]90m\x1b[[]39;49;00m'

/home/<user>/git-workspaces/pytest/testing/test_terminal.py:1291: Failed
------------------------------------------------------------------------------------ Captured stdout call -------------------------------------------------------------------------------------
============================= test session starts ==============================
platform linux -- Python 3.12.5, pytest-8.3.4, pluggy-1.5.0
rootdir: /tmp/pytest-of-<user>/pytest-19/test_color_yes0
collected 1 item

test_color_yes.py F                                                      [100%]

=================================== FAILURES ===================================
__________________________________ test_this ___________________________________

    def test_this():
>       fail()

test_color_yes.py:5: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

    def fail():
>       assert 0
E       assert 0

test_color_yes.py:2: AssertionError
=========================== short test summary info ============================
FAILED test_color_yes.py::test_this - assert 0
============================== 1 failed in 0.01s ===============================
________________________________________________________________________ TestCodeHighlight.test_code_highlight_simple _________________________________________________________________________

self = <test_terminal.TestCodeHighlight object at 0x7fa80e0ecfe0>, pytester = <Pytester PosixPath('/tmp/pytest-of-<user>/pytest-19/test_code_highlight_simple0')>
color_mapping = <class 'conftest.color_mapping.<locals>.ColorMapping'>

    def test_code_highlight_simple(self, pytester: Pytester, color_mapping) -> None:
        pytester.makepyfile(
            """
            def test_foo():
                assert 1 == 10
        """
        )
        result = pytester.runpytest("--color=yes")
>       result.stdout.fnmatch_lines(
            color_mapping.format_for_fnmatch(
                [
                    "    {reset}{kw}def{hl-reset} {function}test_foo{hl-reset}():{endline}",
                    ">       {kw}assert{hl-reset} {number}1{hl-reset} == {number}10{hl-reset}{endline}",
                    "{bold}{red}E       assert 1 == 10{reset}",
                ]
            )
        )
E       Failed: nomatch: '    \x1b[[]0m\x1b[[]94mdef\x1b[[]39;49;00m \x1b[[]92mtest_foo\x1b[[]39;49;00m():\x1b[[]90m\x1b[[]39;49;00m'
E           and: '\x1b[1m============================= test session starts ==============================\x1b[0m'
E           and: 'platform linux -- Python 3.12.5, pytest-8.3.4, pluggy-1.5.0'
E           and: 'rootdir: /tmp/pytest-of-<user>/pytest-19/test_code_highlight_simple0'
E           and: 'collected 1 item'
E           and: ''
E           and: 'test_code_highlight_simple.py \x1b[31mF\x1b[0m\x1b[31m                                          [100%]\x1b[0m'
E           and: ''
E           and: '=================================== FAILURES ==================================='
E           and: '\x1b[31m\x1b[1m___________________________________ test_foo ___________________________________\x1b[0m'
E           and: ''
E           and: '    \x1b[0m\x1b[94mdef\x1b[39;49;00m\x1b[90m \x1b[39;49;00m\x1b[92mtest_foo\x1b[39;49;00m():\x1b[90m\x1b[39;49;00m'
E           and: '>       \x1b[94massert\x1b[39;49;00m \x1b[94m1\x1b[39;49;00m == \x1b[94m10\x1b[39;49;00m\x1b[90m\x1b[39;49;00m'
E           and: '\x1b[1m\x1b[31mE       assert 1 == 10\x1b[0m'
E           and: ''
E           and: '\x1b[1m\x1b[31mtest_code_highlight_simple.py\x1b[0m:2: AssertionError'
E           and: '\x1b[36m\x1b[1m=========================== short test summary info ============================\x1b[0m'
E           and: '\x1b[31mFAILED\x1b[0m test_code_highlight_simple.py::\x1b[1mtest_foo\x1b[0m - assert 1 == 10'
E           and: '\x1b[31m============================== \x1b[31m\x1b[1m1 failed\x1b[0m\x1b[31m in 0.01s\x1b[0m\x1b[31m ===============================\x1b[0m'
E       remains unmatched: '    \x1b[[]0m\x1b[[]94mdef\x1b[[]39;49;00m \x1b[[]92mtest_foo\x1b[[]39;49;00m():\x1b[[]90m\x1b[[]39;49;00m'

/home/<user>/git-workspaces/pytest/testing/test_terminal.py:2585: Failed
------------------------------------------------------------------------------------ Captured stdout call -------------------------------------------------------------------------------------
============================= test session starts ==============================
platform linux -- Python 3.12.5, pytest-8.3.4, pluggy-1.5.0
rootdir: /tmp/pytest-of-<user>/pytest-19/test_code_highlight_simple0
collected 1 item

test_code_highlight_simple.py F                                          [100%]

=================================== FAILURES ===================================
___________________________________ test_foo ___________________________________

    def test_foo():
>       assert 1 == 10
E       assert 1 == 10

test_code_highlight_simple.py:2: AssertionError
=========================== short test summary info ============================
FAILED test_code_highlight_simple.py::test_foo - assert 1 == 10
============================== 1 failed in 0.01s ===============================
_____________________________________________________________________ TestCodeHighlight.test_code_highlight_continuation ______________________________________________________________________

self = <test_terminal.TestCodeHighlight object at 0x7fa80e0ed1f0>, pytester = <Pytester PosixPath('/tmp/pytest-of-<user>/pytest-19/test_code_highlight_continuation0')>
color_mapping = <class 'conftest.color_mapping.<locals>.ColorMapping'>

    def test_code_highlight_continuation(
        self, pytester: Pytester, color_mapping
    ) -> None:
        pytester.makepyfile(
            """
            def test_foo():
                print('''
                '''); assert 0
        """
        )
        result = pytester.runpytest("--color=yes")
    
>       result.stdout.fnmatch_lines(
            color_mapping.format_for_fnmatch(
                [
                    "    {reset}{kw}def{hl-reset} {function}test_foo{hl-reset}():{endline}",
                    "        {print}print{hl-reset}({str}'''{hl-reset}{str}{hl-reset}",
                    ">   {str}    {hl-reset}{str}'''{hl-reset}); {kw}assert{hl-reset} {number}0{hl-reset}{endline}",
                    "{bold}{red}E       assert 0{reset}",
                ]
            )
        )
E       Failed: nomatch: '    \x1b[[]0m\x1b[[]94mdef\x1b[[]39;49;00m \x1b[[]92mtest_foo\x1b[[]39;49;00m():\x1b[[]90m\x1b[[]39;49;00m'
E           and: '\x1b[1m============================= test session starts ==============================\x1b[0m'
E           and: 'platform linux -- Python 3.12.5, pytest-8.3.4, pluggy-1.5.0'
E           and: 'rootdir: /tmp/pytest-of-<user>/pytest-19/test_code_highlight_continuation0'
E           and: 'collected 1 item'
E           and: ''
E           and: 'test_code_highlight_continuation.py \x1b[31mF\x1b[0m\x1b[31m                                    [100%]\x1b[0m'
E           and: ''
E           and: '=================================== FAILURES ==================================='
E           and: '\x1b[31m\x1b[1m___________________________________ test_foo ___________________________________\x1b[0m'
E           and: ''
E           and: '    \x1b[0m\x1b[94mdef\x1b[39;49;00m\x1b[90m \x1b[39;49;00m\x1b[92mtest_foo\x1b[39;49;00m():\x1b[90m\x1b[39;49;00m'
E           and: "        \x1b[96mprint\x1b[39;49;00m(\x1b[33m'''\x1b[39;49;00m\x1b[33m\x1b[39;49;00m"
E           and: ">   \x1b[33m    \x1b[39;49;00m\x1b[33m'''\x1b[39;49;00m); \x1b[94massert\x1b[39;49;00m \x1b[94m0\x1b[39;49;00m\x1b[90m\x1b[39;49;00m"
E           and: '\x1b[1m\x1b[31mE       assert 0\x1b[0m'
E           and: ''
E           and: '\x1b[1m\x1b[31mtest_code_highlight_continuation.py\x1b[0m:3: AssertionError'
E           and: '----------------------------- Captured stdout call -----------------------------'
E           and: ''
E           and: '    '
E           and: '\x1b[36m\x1b[1m=========================== short test summary info ============================\x1b[0m'
E           and: '\x1b[31mFAILED\x1b[0m test_code_highlight_continuation.py::\x1b[1mtest_foo\x1b[0m - assert 0'
E           and: '\x1b[31m============================== \x1b[31m\x1b[1m1 failed\x1b[0m\x1b[31m in 0.01s\x1b[0m\x1b[31m ===============================\x1b[0m'
E       remains unmatched: '    \x1b[[]0m\x1b[[]94mdef\x1b[[]39;49;00m \x1b[[]92mtest_foo\x1b[[]39;49;00m():\x1b[[]90m\x1b[[]39;49;00m'

/home/<user>/git-workspaces/pytest/testing/test_terminal.py:2607: Failed
------------------------------------------------------------------------------------ Captured stdout call -------------------------------------------------------------------------------------
============================= test session starts ==============================
platform linux -- Python 3.12.5, pytest-8.3.4, pluggy-1.5.0
rootdir: /tmp/pytest-of-<user>/pytest-19/test_code_highlight_continuation0
collected 1 item

test_code_highlight_continuation.py F                                    [100%]

=================================== FAILURES ===================================
___________________________________ test_foo ___________________________________

    def test_foo():
        print('''
>       '''); assert 0
E       assert 0

test_code_highlight_continuation.py:3: AssertionError
----------------------------- Captured stdout call -----------------------------

    
=========================== short test summary info ============================
FAILED test_code_highlight_continuation.py::test_foo - assert 0
============================== 1 failed in 0.01s ===============================
_____________________________________________________________________ TestCodeHighlight.test_code_highlight_custom_theme ______________________________________________________________________

self = <test_terminal.TestCodeHighlight object at 0x7fa80e0ed430>, pytester = <Pytester PosixPath('/tmp/pytest-of-<user>/pytest-19/test_code_highlight_custom_theme0')>
color_mapping = <class 'conftest.color_mapping.<locals>.ColorMapping'>, monkeypatch = <_pytest.monkeypatch.MonkeyPatch object at 0x7fa80c3dda90>

    def test_code_highlight_custom_theme(
        self, pytester: Pytester, color_mapping, monkeypatch: MonkeyPatch
    ) -> None:
        pytester.makepyfile(
            """
            def test_foo():
                assert 1 == 10
        """
        )
        monkeypatch.setenv("PYTEST_THEME", "solarized-dark")
        monkeypatch.setenv("PYTEST_THEME_MODE", "dark")
        result = pytester.runpytest("--color=yes")
>       result.stdout.fnmatch_lines(
            color_mapping.format_for_fnmatch(
                [
                    "    {reset}{kw}def{hl-reset} {function}test_foo{hl-reset}():{endline}",
                    ">       {kw}assert{hl-reset} {number}1{hl-reset} == {number}10{hl-reset}{endline}",
                    "{bold}{red}E       assert 1 == 10{reset}",
                ]
            )
        )
E       Failed: nomatch: '    \x1b[[]0m\x1b[[]94mdef\x1b[[]39;49;00m \x1b[[]92mtest_foo\x1b[[]39;49;00m():\x1b[[]90m\x1b[[]39;49;00m'
E           and: '\x1b[1m============================= test session starts ==============================\x1b[0m'
E           and: 'platform linux -- Python 3.12.5, pytest-8.3.4, pluggy-1.5.0'
E           and: 'rootdir: /tmp/pytest-of-<user>/pytest-19/test_code_highlight_custom_theme0'
E           and: 'collected 1 item'
E           and: ''
E           and: 'test_code_highlight_custom_theme.py \x1b[31mF\x1b[0m\x1b[31m                                    [100%]\x1b[0m'
E           and: ''
E           and: '=================================== FAILURES ==================================='
E           and: '\x1b[31m\x1b[1m___________________________________ test_foo ___________________________________\x1b[0m'
E           and: ''
E           and: '    \x1b[0m\x1b[94mdef\x1b[39;49;00m\x1b[90m \x1b[39;49;00m\x1b[92mtest_foo\x1b[39;49;00m():\x1b[90m\x1b[39;49;00m'
E           and: '>       \x1b[94massert\x1b[39;49;00m \x1b[94m1\x1b[39;49;00m == \x1b[94m10\x1b[39;49;00m\x1b[90m\x1b[39;49;00m'
E           and: '\x1b[1m\x1b[31mE       assert 1 == 10\x1b[0m'
E           and: ''
E           and: '\x1b[1m\x1b[31mtest_code_highlight_custom_theme.py\x1b[0m:2: AssertionError'
E           and: '\x1b[36m\x1b[1m=========================== short test summary info ============================\x1b[0m'
E           and: '\x1b[31mFAILED\x1b[0m test_code_highlight_custom_theme.py::\x1b[1mtest_foo\x1b[0m - assert 1 == 10'
E           and: '\x1b[31m============================== \x1b[31m\x1b[1m1 failed\x1b[0m\x1b[31m in 0.01s\x1b[0m\x1b[31m ===============================\x1b[0m'
E       remains unmatched: '    \x1b[[]0m\x1b[[]94mdef\x1b[[]39;49;00m \x1b[[]92mtest_foo\x1b[[]39;49;00m():\x1b[[]90m\x1b[[]39;49;00m'

/home/<user>/git-workspaces/pytest/testing/test_terminal.py:2630: Failed
------------------------------------------------------------------------------------ Captured stdout call -------------------------------------------------------------------------------------
============================= test session starts ==============================
platform linux -- Python 3.12.5, pytest-8.3.4, pluggy-1.5.0
rootdir: /tmp/pytest-of-<user>/pytest-19/test_code_highlight_custom_theme0
collected 1 item

test_code_highlight_custom_theme.py F                                    [100%]

=================================== FAILURES ===================================
___________________________________ test_foo ___________________________________

    def test_foo():
>       assert 1 == 10
E       assert 1 == 10

test_code_highlight_custom_theme.py:2: AssertionError
=========================== short test summary info ============================
FAILED test_code_highlight_custom_theme.py::test_foo - assert 1 == 10
============================== 1 failed in 0.01s ===============================
=========================================================================================== XPASSES ===========================================================================================
=================================================================================== short test summary info ===================================================================================
FAILED testing/test_terminal.py::test_color_yes - Failed: fnmatch: '\x1b[[]1m=*= test session starts =*=\x1b[[]0m'
FAILED testing/test_terminal.py::TestCodeHighlight::test_code_highlight_simple - Failed: nomatch: '    \x1b[[]0m\x1b[[]94mdef\x1b[[]39;49;00m \x1b[[]92mtest_foo\x1b[[]39;49;00m():\x1b[[]90m\x1b[[]39;49;00m'
FAILED testing/test_terminal.py::TestCodeHighlight::test_code_highlight_continuation - Failed: nomatch: '    \x1b[[]0m\x1b[[]94mdef\x1b[[]39;49;00m \x1b[[]92mtest_foo\x1b[[]39;49;00m():\x1b[[]90m\x1b[[]39;49;00m'
FAILED testing/test_terminal.py::TestCodeHighlight::test_code_highlight_custom_theme - Failed: nomatch: '    \x1b[[]0m\x1b[[]94mdef\x1b[[]39;49;00m \x1b[[]92mtest_foo\x1b[[]39;49;00m():\x1b[[]90m\x1b[[]39;49;00m'
XPASS testing/_py/test_local.py::TestLocalPath::test_make_numbered_dir_multiprocess_safe - #11603
======================================================= 4 failed, 3627 passed, 110 skipped, 11 xfailed, 1 xpassed in 147.69s (0:02:27) ========================================================
py312: exit 1 (148.30 seconds) /home/<user>/git-workspaces/pytest> pytest pid=2583682
  py312: FAIL code 1 (163.94=setup[15.64]+cmd[148.30] seconds)
  evaluation failed :( (164.06 seconds)

(Edited by @The-Compiler to fold away the lengthy output)

Bisected to this Pygments change: Improve whitespace handling in the Python lexer. · pygments/pygments@d24e272

In the test, we match {reset}{kw}def{hl-reset} {function}test_this{hl-reset}():{endline}. With the strings aligned by inserting some ! chars, the failure looks like this:

nomatch: '    \x1b[[]0m\x1b[[]94mdef\x1b[[]39;49;00m \x1b[[]!!!!!!!!!!!!!!!!!!!!!!!92mtest_this\x1b[[]39;49;00m():\x1b[[]90m\x1b[[]39;49;00m'       
    and: '    \x1b[!!0m\x1b[!!94mdef\x1b[!!39;49;00m!\x1b[!!90m!\x1b[39;49;00m\x1b[92mtest_this\x1b[!!39;49;00m():\x1b[!!90m\x1b[!!39;49;00m'

The [] in there are bogus (a [[] matches a literal [ with the fnmatch syntax), so the difference is in the space between def and test_this.

Old:

  def             # literal
  \x1b[39;49;00m  # reset fg/bg/formatting
  ␣               # literal space
  \x1b[92m        # bright green color for function name
  test_this       # literal

New:

  def             # literal
  \x1b[39;49;00m  # reset fg/bg/formatting
! \x1b[90m        # text color to "bright black"
  ␣               # literal space
! \x1b[39;49;00m  # reset fg/bg/formatting
  \x1b[92m        # bright green color for function name
  test_this       # literal

From what I understand, Pygments introduced a new Token.Text.Whitespace, and seems to be adjusting lexers to emit those rather than just Text:

WIP - Whitespace lexing after Keywords - collecting issue pygments/pygments#1905

Not sure why our style/formatter then decides to highlight that in "bright black" since it's space anyways, but from what I understand, Pygments isn't doing anything wrong here and we probably just need to adjust our expected output?

Fix selftests with Pygments >= 2.19.0

Fix: #13113

Sorry, I misread this. Thought it was a regression in Pygments, but it looks like a test on your side got tripped up by more precise whitespace handling?

Yes, exactly. We have integration tests for pytest's output including the color codes, in order to ensure that pytest correctly shows Pygments-highlighted Python source code as part of its tracebacks.

With Pygments now lexing the space in def something(): as Whitespace instead of Text, it seems to get highlighted as a "bright black space" rather than a "unformatted space", which caused the pytest selftests to fail.

I don't think there is anything Pygments does wrong here. I'm somewhat surprised by spaces being colorized in the output as the foreground color would be invisible anyways, but I suppose there is some good reason for that.

Thanks for getting back quickly. Yeah, we've been spending quite some time making sure that whitespace is correctly tagged. I'm also surprised there's a style which has a specific color for whitespace, I assume that "bright black space" is actually the background color and that's by mistake. Do you happen to know which style the tests use?

Just the default style with dark terminal background, here is a reproducer:

import pygments
from pygments.formatters.terminal import TerminalFormatter
from pygments.lexers import PythonLexer

source = "def func(): pass"
lexer = PythonLexer()
formatter = TerminalFormatter(bg="dark")
output = pygments.highlight(source, lexer, formatter)

print(output)
print(repr(output))

Or with Pygments 2.18:

$ echo "def func(): pass" | pygmentize -l python -f terminal -O bg=dark | cat -v
^[[94mdef^[[39;49;00m ^[[92mfunc^[[39;49;00m(): ^[[94mpass^[[39;49;00m^[[90m^[[39;49;00m

but 2.19:

$ echo "def func(): pass" | pygmentize -l python -f terminal -O bg=dark | cat -v
^[[94mdef^[[39;49;00m^[[90m ^[[39;49;00m^[[92mfunc^[[39;49;00m(): ^[[94mpass^[[39;49;00m^[[90m^[[39;49;00m

(I tried passing -O bg=dark to pygmentize in the hope that'd pass that setting to the formatter and result in the same exact output as pytest, but for some reason that doesn't seem to change anything and I still get light-background output) (I was missing a -f terminal as the default seems to be terminal256 in my case)

Fix selftests with Pygments >= 2.19.0

chore(deps): update dependency pytest to v9 (#305)

Navigation Menu

Repository navigation

Pytest's self tests are failing with pygments v2.19.0 #13112

Description

How to recreate

Tox output

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Pytest's self tests are failing with pygments v2.19.0 #13112

Description

Description

How to recreate

Tox output

Activity

The-Compiler commented on Jan 7, 2025

The-Compiler commented on Jan 7, 2025

The-Compiler commented on Jan 7, 2025

Anteru commented on Jan 7, 2025

The-Compiler commented on Jan 7, 2025

Anteru commented on Jan 7, 2025

The-Compiler commented on Jan 7, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions