[roberfi]

Fixes #16788

Description

The fix consists in discarding one possible type item of the union if it is an argument of other item that is subtype of the actual type.

Code examples

Example 1

from typing import List, Iterable, TypeVar, Union, reveal_type

T = TypeVar("T")


def coerce_list(x: Union[T, Iterable[T]]) -> List[T]:
    reveal_type(x)
    raise NotImplementedError()


def test() -> None:
    reveal_type(coerce_list(1))
    reveal_type(coerce_list([1, 2]))

Output before the fix:

file.py:7: note: Revealed type is "Union[T`-1, typing.Iterable[T`-1]]"
file.py:12: note: Revealed type is "builtins.list[builtins.int]"
file.py:13: note: Revealed type is "builtins.list[Never]"
file.py:13: error: Argument 1 to "coerce_list" has incompatible type "list[int]"; expected "Iterable[Never]"

Output after the fix:

file.py:7: note: Revealed type is "Union[T`-1, typing.Iterable[T`-1]]"
file.py:12: note: Revealed type is "builtins.list[builtins.int]"
file.py:13: note: Revealed type is "builtins.list[builtins.int]"

Example 2

from typing import Generic, TypeVar, Union, reveal_type

T = TypeVar("T")


class GenericClass(Generic[T]):
    def __init__(self, value: T) -> None:
        self.value = value


def method_with_union(arg: Union[GenericClass[T], T]) -> GenericClass[T]:
    if not isinstance(arg, GenericClass):
        arg = GenericClass(arg)
    return arg


result_1 = method_with_union(GenericClass("test"))
reveal_type(result_1)

result_2 = method_with_union("test")
reveal_type(result_2)

Output before the fix:

file.py:17: error: Need type annotation for "result_1"  [var-annotated]
file.py:17: error: Argument 1 to "method_with_union" has incompatible type "GenericClass[str]"; expected "GenericClass[Never]"  [arg-type]
file.py:18: note: Revealed type is "file.GenericClass[Any]"
file.py:21: note: Revealed type is "file.GenericClass[builtins.str]"

Output after the fix:

file.py:18: note: Revealed type is "file.GenericClass[builtins.str]"
file.py:21: note: Revealed type is "file.GenericClass[builtins.str]"

Known limitations

• Only direct type arguments are considered. For Union[T, list[list[T]]], passing list[list[int]] still infers Never, because T is not a direct argument of list[list[T]]. Handling nested occurrences would widen the scope of the change considerably for a rare pattern.
• Union[type[T], type[Box[T]]] still fails when a class object expression such as Box[int] is passed, because mypy represents it as a callable (def () -> Box[int]) rather than type[Box[int]]. Passing a value of type type[Box[int]] works. This is unrelated to the heuristic and is better addressed separately.

[LevBernstein]

Bumping this for visibility--I'm seeing the same error in my own project, caused by calling a nextcord.Command with a keyword argument:

bb_test.py:2276: error: Argument "words" to "__call__" of "Command" has incompatible type "str"; expected "Never"  [arg-type]
...
ch = MockChannel(guild=MockGuild())
ctx = MockContext(Bot.BeardlessBot, channel=ch)
assert await Bot.cmdDefine(ctx, words="f") == 1
...
@BeardlessBot.command(name="define")  # type: ignore[arg-type]
async def cmdDefine(ctx: misc.BotContext, *, words: str = "") -> int:
if misc.ctxCreatedThread(ctx):
  return -1
emb = await misc.define(words)
await ctx.send(embed=emb)
return 1

The command decorator in that snippet also has a type-ignore comment, because every use of the decorator sees the same "Never" issue. I would love for this to be merged.

[github-actions]

Diff from mypy_primer, showing the effect of this PR on open source code:

prefect (https://github.com/PrefectHQ/prefect)
- src/prefect/states.py:373: error: Argument 1 to "StateGroup" has incompatible type "Collection[Never]"; expected "list[State[Any]]"  [arg-type]
+ src/prefect/states.py:373: error: Argument 1 to "StateGroup" has incompatible type "Collection[State[Any] | Any]"; expected "list[State[Any]]"  [arg-type]
- src/prefect/_internal/states.py:243: error: Argument 1 to "StateGroup" has incompatible type "Collection[Never]"; expected "list[State[Any]]"  [arg-type]
+ src/prefect/_internal/states.py:243: error: Argument 1 to "StateGroup" has incompatible type "Collection[State[Any] | Any]"; expected "list[State[Any]]"  [arg-type]
- src/prefect/task_runners.py:110: error: Overloaded function implementation does not accept all possible parameters of signature 1  [misc]
+ src/prefect/task_runners.py:110: error: Overloaded function implementation cannot satisfy signature 1 due to inconsistencies in how they use type variables  [misc]
+ src/prefect/tasks.py:1211: error: Overloaded function implementation cannot satisfy signature 5 due to inconsistencies in how they use type variables  [misc]

discord.py (https://github.com/Rapptz/discord.py)
- discord/app_commands/commands.py:1013: error: Argument 1 to "maybe_coroutine" has incompatible type "Callable[[Interaction[ClientT]], Coroutine[Any, Any, bool]]"; expected "Callable[[Interaction[Client]], Never | Awaitable[Never]]"  [arg-type]
- discord/ext/commands/cog.py:710: error: Argument 1 to "maybe_coroutine" has incompatible type "Callable[[], Coroutine[Any, Any, None]]"; expected "Callable[[], Never | Awaitable[Never]]"  [arg-type]
- discord/ext/commands/cog.py:727: error: Argument 1 to "maybe_coroutine" has incompatible type "Callable[[], Coroutine[Any, Any, None]]"; expected "Callable[[], Never | Awaitable[Never]]"  [arg-type]
- discord/ext/commands/cog.py:780: error: Argument 1 to "maybe_coroutine" has incompatible type "Callable[[], Coroutine[Any, Any, None]]"; expected "Callable[[], Never | Awaitable[Never]]"  [arg-type]
- discord/ext/commands/hybrid.py:470: error: Argument 1 to "maybe_coroutine" has incompatible type "Callable[[Interaction[ClientT]], Coroutine[Any, Any, bool]]"; expected "Callable[[Interaction[Client]], Never | Awaitable[Never]]"  [arg-type]

Tanjun (https://github.com/FasterSpeeding/Tanjun)
- tanjun/dependencies/data.py:220: error: Argument "callback" to "inject" has incompatible type "Callable[..., Coroutine[Any, Any, _T]]"; expected "Callable[..., Coroutine[Any, Any, Never] | Never]"  [arg-type]
- tanjun/dependencies/data.py:347: error: Argument "callback" to "inject" has incompatible type "Callable[..., Coroutine[Any, Any, _T]]"; expected "Callable[..., Coroutine[Any, Any, Never] | Never]"  [arg-type]
- tanjun/schedules.py:347: error: Argument 1 to "call_with_async_di" of "Client" has incompatible type "_CallbackSigT"; expected "Callable[..., Coroutine[Any, Any, Never] | Never]"  [arg-type]
- tanjun/schedules.py:373: error: Argument 1 to "call_with_async_di" of "Client" has incompatible type "Callable[..., Coroutine[Any, Any, None]]"; expected "Callable[..., Coroutine[Any, Any, Never] | Never]"  [arg-type]
- tanjun/schedules.py:396: error: Argument 1 to "call_with_async_di" of "Client" has incompatible type "Callable[..., Coroutine[Any, Any, None]]"; expected "Callable[..., Coroutine[Any, Any, Never] | Never]"  [arg-type]
- tanjun/schedules.py:1067: error: Argument 1 to "call_with_async_di" of "Client" has incompatible type "_CallbackSigT"; expected "Callable[..., Coroutine[Any, Any, Never] | Never]"  [arg-type]
- tanjun/commands/slash.py:3191: error: Argument 1 to "call_with_async_di" of "Context" has incompatible type "_SlashCallbackSigT"; expected "Callable[..., Coroutine[Any, Any, Never] | Never]"  [arg-type]
- tanjun/commands/slash.py:3235: error: Argument 1 to "call_with_async_di" of "Context" has incompatible type "def (AutocompleteContext, str, /, *Any, **Any) -> Coroutine[Any, Any, None] | def (AutocompleteContext, float, /, *Any, **Any) -> Coroutine[Any, Any, None] | def (AutocompleteContext, int, /, *Any, **Any) -> Coroutine[Any, Any, None]"; expected "Callable[..., Coroutine[Any, Any, Never] | Never]"  [arg-type]
- tanjun/commands/message.py:346: error: Argument 1 to "call_with_async_di" of "Context" has incompatible type "_MessageCallbackSigT"; expected "Callable[..., Coroutine[Any, Any, Never] | Never]"  [arg-type]
- tanjun/commands/menu.py:689: error: Argument 1 to "call_with_async_di" of "Context" has incompatible type "_AnyMenuCallbackSigT"; expected "Callable[..., Coroutine[Any, Any, Never] | Never]"  [arg-type]
- tanjun/clients.py:2294: error: Argument 1 to "call_with_async_di" of "Context" has incompatible type "def (MessageContext, /, *Any, **Any) -> Coroutine[Any, Any, Iterable[str]]"; expected "Callable[..., Coroutine[Any, Any, Never] | Never]"  [arg-type]

pywin32 (https://github.com/mhammond/pywin32)
  ./pythonwin/pywin/test/_dbgscript.py: error: INTERNAL ERROR -- Please try using mypy master on GitHub:
- version: 2.5.0+dev.a0dda1c08e861018029fea6aaa2140e4d6c58333
+ version: 2.5.0+dev.f65f73992bac1237ea1f7cf717ad201500309c45

ibis (https://github.com/ibis-project/ibis)
+ ibis/common/egraph.py:733: error: Item "function" of "Callable[..., Any] | Pattern | Variable" has no attribute "substitute"  [union-attr]
- ibis/common/egraph.py:731: error: Need type annotation for "rewrite"  [var-annotated]
- ibis/common/egraph.py:731: error: Argument 1 to "promote_list" has incompatible type "list[Rewrite]"; expected "Iterable[Never]"  [arg-type]
- ibis/expr/types/strings.py:973: error: Need type annotation for "pattern"  [var-annotated]
- ibis/expr/types/strings.py:975: error: Argument 1 to "promote_list" has incompatible type "str | StringValue | Deferred | Iterable[str | StringValue | Deferred]"; expected "Iterable[Never]"  [arg-type]
+ ibis/expr/types/strings.py:974: error: Argument 2 to "StringSQLLike" has incompatible type "str | StringValue | Deferred"; expected "Value[String, Any]"  [arg-type]
- ibis/expr/types/strings.py:1019: error: Need type annotation for "pattern"  [var-annotated]
- ibis/expr/types/strings.py:1021: error: Argument 1 to "promote_list" has incompatible type "str | StringValue | Deferred | Iterable[str | StringValue | Deferred]"; expected "Iterable[Never]"  [arg-type]
+ ibis/expr/types/strings.py:1020: error: Argument 2 to "StringSQLILike" has incompatible type "str | StringValue | Deferred"; expected "Value[String, Any]"  [arg-type]
- ibis/expr/types/relations.py:1141: error: Need type annotation for "arg"  [var-annotated]
- ibis/expr/types/relations.py:1143: error: Argument 1 to "promote_list" has incompatible type "Sequence[str | int]"; expected "Iterable[Never]"  [arg-type]
- ibis/backends/sql/__init__.py:589: error: Argument 1 to "promote_tuple" has incompatible type "str | Iterable[str]"; expected "Iterable[Never]"  [arg-type]

static-frame (https://github.com/static-frame/static-frame)
+ static_frame/core/archive_npy.py:451: error: Unused "type: ignore" comment  [unused-ignore]

scipy-stubs (https://github.com/scipy/scipy-stubs)
+ tests/sparse/test_bsr.pyi:71: error: Unused "type: ignore" comment  [unused-ignore]
+ tests/sparse/test_bsr.pyi:72: error: Unused "type: ignore" comment  [unused-ignore]
+ tests/sparse/test_bsr.pyi:73: error: Unused "type: ignore" comment  [unused-ignore]
+ tests/sparse/test_bsr.pyi:74: error: Unused "type: ignore" comment  [unused-ignore]
+ tests/sparse/test_bsr.pyi:76: error: Unused "type: ignore" comment  [unused-ignore]
+ tests/sparse/test_bsr.pyi:77: error: Unused "type: ignore" comment  [unused-ignore]
+ tests/sparse/test_bsr.pyi:78: error: Unused "type: ignore" comment  [unused-ignore]
+ tests/sparse/test_bsr.pyi:79: error: Unused "type: ignore" comment  [unused-ignore]
+ tests/sparse/test_bsr.pyi:176: error: Unused "type: ignore" comment  [unused-ignore]
+ tests/sparse/test_bsr.pyi:178: error: Unused "type: ignore" comment  [unused-ignore]