Skip to content

ExceptionGroup ergonomics in pytest.param(..., marks=xfail(raises=...)) #12504

Description

@Zac-HD

Suppose you have a parameterized test, some params of which are expected to raise (e.g.) IndexError:

return pytest.param(
    ...,
    marks=pytest.mark.xfail(raises=IndexError, strict=True),
)

If your test is async though, you'll actually raise an ExceptionGroup(..., [IndexError]) (maybe with even more nesting), so the tests will still fail. Can we make this more ergonomic? What would a parametrize-aware RaisesGroup-like thing look like?

See also #11538 and #11671.

Activity

  1. added
    type: enhancementnew feature or API change, should be merged into features branch
    on Jun 20, 2024
  2. jakkdl commented on Jul 16, 2024

    @jakkdl
    Member

    I might be seeing nails everywhere, but I would do

    return pytest.param(
        ...,
        marks=pytest.mark.xfail(raises=RaisesGroup(IndexError), strict=True),
    )

    and afaict the only code doing the matching is

    if raises is not None and not isinstance(call.excinfo.value, raises):
    rep.outcome = "failed"

    where it would be straightforward to add

    if isinstance(raises, RaisesGroup) and not raises.matches(call.excinfo.value):
      rep.outcome = "failed"
    elif raises is not None [...]
      ...

    This of course depends on #11671 (or a version that fully implements trio.RaisesGroup functionality).

  3. jakkdl commented on Sep 17, 2024

    @jakkdl
    Member

    Actually, I don't see why this is about pytest.param at all. If pytest.mark.xfail(raises=...) got support for RaisesGroup (or equivalent) it should Just Work ™️

    @pytest.mark.xfail(raises=RaisesGroup(ValueError))
    def test_foo():
        raise ExceptionGroup("foo", [ValueError])

    #11671 has fallen behind the trio implementation, and is stalled by people wanting to have opinions on the interface design, otherwise I'd resolve this issue in it right away.

  4. jakkdl commented on Mar 13, 2025

    @jakkdl
    Member

    Oh this also got fixed in #13192

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    topic: parametrizerelated to @pytest.mark.parametrizetype: enhancementnew feature or API change, should be merged into features branch

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions