Skip to content

ENH: mechanism for failing when warnings are issued without conversion to exceptions? #11518

Closed

Description

@tylerjereddy

Maybe this is rather niche, or more of a usage query, but I think it has confused us a little bit. One conundrum we occasionally face in SciPy is that the heavily-used NumPy library sometimes decides to issue warnings for some function calls, but the same code paths can also have try/except guards for various reasons.

So, when we try to proactively turn warnings into errors in filterwarnings in our pytest.ini, what we end up with is a case where a warning becomes invisible to our testsuite/CI testing, and we miss out on i.e., deprecation warnings by the very mechanism we are trying to use to discover them, because the try/except block is triggered via pytest and then handled gracefully by NumPy to end up in a warning-free scenario!

Do you think there's a valid feature request in here? Or should we just resign ourselves to i.e., a CI job that runs the testsuite with the "warnings->errors" disabled and then greps on the pytest summary and errors out on non-zero warnings accounting?

Related:

Activity

  1. RonnyPfannschmidt commented on Oct 18, 2023

    @RonnyPfannschmidt
    Member

    This is a dupe, let me find the one I made

  2. RonnyPfannschmidt commented on Oct 18, 2023

    @RonnyPfannschmidt
    Member

    #10098 is not quite a dupe but similar enough

    I propose post test warnings failure as a feature

  3. Zac-HD commented on Oct 21, 2023

    @Zac-HD
    Member

    I'd consider this a bug in Numpy: the code described in scipy/scipy#18781 (comment) is going to give the wrong answer if users have warnings-as-errors, and for that matter if they try to ctrl-c it'll misbehave badly too. Instead of

    try:
       return np.float64(float(obj))
    except:
       return np.array(obj, dtype=np.float64)[()]

    you should do something like

    try:
       return np.float64(float(obj))
    except Warning:
        raise
    except Exception:
       return np.array(obj, dtype=np.float64)[()]
  4. Zac-HD commented on Jul 22, 2026

    @Zac-HD
    Member

    Closing as resolved: pytest now has a --max-warnings/max_warnings option (#14372, closing #14371) that fails the run based on the post-test warnings summary count without requiring filterwarnings = error, which is exactly the feature discussed here and avoids the try/except-swallowing problem described above.

    (I do also think that hiding warnings when end-users set -Werror is regrettable though)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    plugin: warningsrelated to the warnings builtin plugin

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions