[Pytest tests] Multi-color stats and 'dumb' terminal

[stanislavlevin]

I run the Pytest's test suite (5.3.0/5.3.1) during the RPM build.
With recent multi-color stats feature the following tests fail in dumb terminal environments:

FAILED testing/test_pdb.py::TestPDB::test_pdb_unittest_skip - pexpect.excepti...
FAILED testing/test_pdb.py::TestPDB::test_pdb_print_captured_stdout_and_stderr
FAILED testing/test_pdb.py::TestPDB::test_doctest_set_trace_quit - AssertionE...
FAILED testing/test_pdb.py::TestPDB::test_pdb_continue_with_recursive_debug[]
FAILED testing/test_pdb.py::TestPDB::test_pdb_continue_with_recursive_debug[-s]
FAILED testing/test_pdb.py::TestPDB::test_pdb_continue_with_recursive_debug[-p no:capture]
FAILED testing/test_pdb.py::TestTraceOption::test_trace_sets_breakpoint - Ass...
FAILED testing/test_pdb.py::TestTraceOption::test_trace_with_parametrize_handles_shared_fixtureinfo
FAILED testing/test_pdb.py::test_pdb_suspends_fixture_capturing[capfd] - Asse...
FAILED testing/test_pdb.py::test_pdb_suspends_fixture_capturing[capsys] - Ass...
FAILED testing/test_terminal.py::TestTerminal::test_report_collect_after_half_a_second

        rest = child.read().decode("utf8")
        assert "Exit: Quitting debugger" in rest
>       assert "= \x1b[31m\x1b[1m1 failed\x1b[0m\x1b[31m in" in rest
E       AssertionError: assert '= \x1b[31m\x1b[1m1 failed\x1b[0m\x1b[31m in' in ') \r\n\r\n\r\n!!!!!!!!!!!!!!!!!!! _pytest.outcomes.Exit: Quitting debugger !!!!!!!!!!!!!!!!!!!\r\n============================== 1 failed in 0.20s ===============================\r\n'

/usr/src/RPM/BUILD/python3-module-pytest-5.3.1/testing/test_pdb.py:225: AssertionError
_____________________ TestPDB.test_doctest_set_trace_quit ______________________

Sure, if I set TERM=xterm all tests pass.
This is because of the expectations are unconditionally hardcoded.

[harishkesavarao]

@Zac-HD is this still a problem?

[Zac-HD]

Closing as resolved: the self-test failures under TERM=dumb were fixed by #12244, which made the affected tests set PY_COLORS=1 explicitly instead of relying on the ambient terminal type.