From 97437e5ca9daa216784e8c5429923d779c639ec9 Mon Sep 17 00:00:00 2001 From: Jukka Lehtosalo Date: Wed, 14 Aug 2019 15:43:56 +0300 Subject: [PATCH 01/13] Add documentation for error codes --- docs/source/error_code_list.rst | 124 ++++++++++++++++++++++++++++++++ docs/source/error_codes.rst | 47 ++++++++++++ docs/source/index.rst | 2 + 3 files changed, 173 insertions(+) create mode 100644 docs/source/error_code_list.rst create mode 100644 docs/source/error_codes.rst diff --git a/docs/source/error_code_list.rst b/docs/source/error_code_list.rst new file mode 100644 index 0000000000000..e7f5b1c3ce223 --- /dev/null +++ b/docs/source/error_code_list.rst @@ -0,0 +1,124 @@ +List of error codes +=================== + +This section documents various errors codes that mypy can generate. +See :ref:`error-codes` for general documentation about error codes. + + +Checking that attribute exists [attr-defined] +--------------------------------------------- + +Mypy that an attribute is defined in the target class or module. This +applies to reading an attribute and setting an attribute. Attribute +assignments in a class body or through the ``self`` argument are +considered to define new attributes. Mypy doesn't allow defining +attributes outside a class definition. + +Example: + +.. code-block:: python + + class Resource: + def __init__(self, name: str) -> None: + self.name = name + + r = Resouce('x') + print(r.name) # OK + print(r.id) # "Resource" has no attribute "id" [attr-defined] + r.id = 5 # "Resource" has no attribute "id" [attr-defined] + + +Checking that name is defined [name-defined] +-------------------------------------------- + +Mypy expects that all name references contain a definitinon, such as +an assignment, function definition or an import. This can catch missing +definitions, missing imports, and typos. + +Example: + +.. code-block:: python + + x = func(1) # Name 'func' is not defined [name-defined] + +Checking arguments in calls [call-arg] +-------------------------------------- + +Mypy expects that the number and names of arguments match the called function. +Note that argument type checks have a separate error code ``arg-type``. + +Example: + +.. code-block:: python + + from typing import Sequence + + def greet(name: str) -> None: + print('hello', name) + + greet('jack') # OK + greet('hi', 'jack') # Too many arguments for "greet" [call-arg] + +Checking argument types [arg-type] +---------------------------------- + +Mypy checks that argument types in a call match the declared argument +types in the signature. + +Example: + +.. code-block:: python + + from typing import List, Optional + + def first(x: List[int]) -> Optional[int]: + return x[0] if x else 0 + + t = (5, 4) + # Argument 1 to "first" has incompatible type "Tuple[int, int]"; + # expected "List[int]" [arg-type] + print(first(t)) + + +Checking validity of overrides [override] +----------------------------------------- + +Mypy checks that an overridden method or attribute is compatible with +the base class. A method in a subclass must accept all arguments +that the base class method accepts, and the return type must conform +to the return type in the base class. + +Argument typess can be more general is a subclass (i.e., they can vary +contravariantly). Return type can be narrowed in a subclass (i.e., it +can vary covariantly). It's okay to define additional arguments in +a subclass method, as long all extra arguments can be left out. + +Example: + +.. code-block:: python + + from typing import Optional, Union + + class Base: + def method(self, + arg: int) -> Optional[int]: + ... + + class Derived(Base): + def method(self, + arg: Union[int, str]) -> int: # OK + ... + + class DerivedBad(Base): + # Argument 1 of "method" is incompatible with "Base" [override] + def method(self, + arg: bool) -> int: + ... + +Miscellaneous checks [misc] +--------------------------- + +Mypy performs numerous other, more rarely failing checks that don't +have a specific error codes. These use the ``misc`` error code. This +error code is not special. For example, you can ignore all errors in +this category by using ``# type: ignore[misc]`` comment. diff --git a/docs/source/error_codes.rst b/docs/source/error_codes.rst new file mode 100644 index 0000000000000..f1a3fad09aaed --- /dev/null +++ b/docs/source/error_codes.rst @@ -0,0 +1,47 @@ +.. _error-codes: + +Error codes +=========== + +Mypy can optionally display an error code after a message. Error +codes serve two purposes: + +1. It's possible to silence specific error codes on a line using + ``# type: ignore[code]``. +2. The error code can be looked up here in the documentation to provide + additional context. + +Displaying error codes +---------------------- + +Error codes are not displayed by default. Use ``--show-error-codes`` +to display error codes. Error codes are shown inside square brackets: + +.. code-block:: text + + $ mypy --show-error-codes prog.py + prog.py:1: error: "str" has no attribute "trim" [attr-defined] + +Silencing errors based on error codes +------------------------------------- + +You can use ``# type: ignore[code, ...]`` to ignore all errors with +specific codes on a line. This can be used even if you have not +configured mypy to show error codes. Currently it's only possible to +disable arbitrary error codes on individual lines using this comment. + +.. note:: + + There are command-line flags and config file settings for enabling + certain optional error codes, such as ``--disallow-untype-defs``, + which enables the ``no-untyped-def`` error code. + +Here is how we ignore an error about an imported name mypy thinks is +undefined: + +.. code-block:: python + + # Assume 'foo' is defined in 'foolib', even though mypy + # can't see the definition. + + from foolib import foo # type: ignore[attr-defined] diff --git a/docs/source/index.rst b/docs/source/index.rst index f5a3e2b628080..dd09795dc4589 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -64,6 +64,8 @@ Mypy is a static type checker for Python 3 and Python 2.7. :maxdepth: 2 :caption: Miscellaneous + error_codes + error_code_list common_issues supported_python_features python36 From 90b4e732eab4a50871e14b6779455b8b435e9f74 Mon Sep 17 00:00:00 2001 From: Jukka Lehtosalo Date: Mon, 2 Sep 2019 17:32:53 +0100 Subject: [PATCH 02/13] Add empty sections for various error codes --- docs/source/error_code_list.rst | 136 +++++++++++++++++++++++++++++--- 1 file changed, 126 insertions(+), 10 deletions(-) diff --git a/docs/source/error_code_list.rst b/docs/source/error_code_list.rst index e7f5b1c3ce223..92a5753826abc 100644 --- a/docs/source/error_code_list.rst +++ b/docs/source/error_code_list.rst @@ -4,9 +4,11 @@ List of error codes This section documents various errors codes that mypy can generate. See :ref:`error-codes` for general documentation about error codes. +Checks enabled by default +************************* -Checking that attribute exists [attr-defined] ---------------------------------------------- +Check that attribute exists [attr-defined] +------------------------------------------ Mypy that an attribute is defined in the target class or module. This applies to reading an attribute and setting an attribute. Attribute @@ -27,9 +29,13 @@ Example: print(r.id) # "Resource" has no attribute "id" [attr-defined] r.id = 5 # "Resource" has no attribute "id" [attr-defined] +Check that attribute exists in each union item [union-attr] +----------------------------------------------------------- -Checking that name is defined [name-defined] --------------------------------------------- +TODO + +Check that name is defined [name-defined] +----------------------------------------- Mypy expects that all name references contain a definitinon, such as an assignment, function definition or an import. This can catch missing @@ -41,8 +47,8 @@ Example: x = func(1) # Name 'func' is not defined [name-defined] -Checking arguments in calls [call-arg] --------------------------------------- +Check arguments in calls [call-arg] +----------------------------------- Mypy expects that the number and names of arguments match the called function. Note that argument type checks have a separate error code ``arg-type``. @@ -59,8 +65,8 @@ Example: greet('jack') # OK greet('hi', 'jack') # Too many arguments for "greet" [call-arg] -Checking argument types [arg-type] ----------------------------------- +Check argument types [arg-type] +------------------------------- Mypy checks that argument types in a call match the declared argument types in the signature. @@ -79,9 +85,23 @@ Example: # expected "List[int]" [arg-type] print(first(t)) +Check calls to overloaded functions [call-overload] +--------------------------------------------------- -Checking validity of overrides [override] ------------------------------------------ +TODO + +Check validity of type annotations [valid-type] +----------------------------------------------- + +TODO + +Require annotation if variable type is unclear [var-annotated] +-------------------------------------------------------------- + +TODO + +Check validity of overrides [override] +-------------------------------------- Mypy checks that an overridden method or attribute is compatible with the base class. A method in a subclass must accept all arguments @@ -115,6 +135,92 @@ Example: arg: bool) -> int: ... +Check that function returns a value [return] +-------------------------------------------- + +TODO + + +Check that return value is compatible [return-value] +---------------------------------------------------- + +TODO + +Check compatibility of assignment statement [assignment] +-------------------------------------------------------- + +TODO + +Check that type arguments exist [type-arg] +------------------------------------------ + +TODO + +Check type variable values [type-var] +------------------------------------- + +TODO + +Check indexing operations [index] +--------------------------------- + +TODO + +Check uses of various operators [operator] +------------------------------------------ + +TODO + +Check list items [list-item] +---------------------------- + +TODO + +Check dict items [dict-item] +---------------------------- + +TODO + +Check TypedDict items [typeddict-item] +-------------------------------------- + +TODO + +Check that type of target is known [has-type] +--------------------------------------------- + +TODO + +Check that import target can be found [import] +---------------------------------------------- + +TODO + +Check that each name is defined once [no-redef] +----------------------------------------------- + +TODO + +Check that called functions return a value [func-returns-value] +--------------------------------------------------------------- + +TODO + +Check instantiation of abstract classes [abstract] +-------------------------------------------------- + +TODO + +Check the target of NewType [valid-newtype] +------------------------------------------- + +TODO + +Report syntax errors [syntax] +----------------------------- + +TODO + Miscellaneous checks [misc] --------------------------- @@ -122,3 +228,13 @@ Mypy performs numerous other, more rarely failing checks that don't have a specific error codes. These use the ``misc`` error code. This error code is not special. For example, you can ignore all errors in this category by using ``# type: ignore[misc]`` comment. + +Optional checks +*************** + +* no-untyped-def +* no-untyped-call +* redundant-cast +* comparison-overlap +* no-any-unimported +* no-any-return From 831dbeca3ddf28cb1625fb7853a34e1b28b2e7d1 Mon Sep 17 00:00:00 2001 From: Jukka Lehtosalo Date: Mon, 2 Sep 2019 17:45:11 +0100 Subject: [PATCH 03/13] Split error code list into two pages and other updates --- docs/source/error_code_list.rst | 23 ++++++------------- docs/source/error_code_list2.rst | 39 ++++++++++++++++++++++++++++++++ docs/source/error_codes.rst | 22 ++++++++++-------- docs/source/index.rst | 1 + 4 files changed, 59 insertions(+), 26 deletions(-) create mode 100644 docs/source/error_code_list2.rst diff --git a/docs/source/error_code_list.rst b/docs/source/error_code_list.rst index 92a5753826abc..e56e29b301147 100644 --- a/docs/source/error_code_list.rst +++ b/docs/source/error_code_list.rst @@ -1,11 +1,12 @@ -List of error codes -=================== +.. _error-code-list: -This section documents various errors codes that mypy can generate. -See :ref:`error-codes` for general documentation about error codes. +List of error codes (default checks) +==================================== -Checks enabled by default -************************* +This section documents various errors codes that mypy can generate +with default options. See :ref:`error-codes` for general documentation +about error codes. See :ref:`error-codes-optional` for additional +error codes that you can enable. Check that attribute exists [attr-defined] ------------------------------------------ @@ -228,13 +229,3 @@ Mypy performs numerous other, more rarely failing checks that don't have a specific error codes. These use the ``misc`` error code. This error code is not special. For example, you can ignore all errors in this category by using ``# type: ignore[misc]`` comment. - -Optional checks -*************** - -* no-untyped-def -* no-untyped-call -* redundant-cast -* comparison-overlap -* no-any-unimported -* no-any-return diff --git a/docs/source/error_code_list2.rst b/docs/source/error_code_list2.rst new file mode 100644 index 0000000000000..0201c41ed0782 --- /dev/null +++ b/docs/source/error_code_list2.rst @@ -0,0 +1,39 @@ +.. _error-codes-optional: + +List of error codes (optional checks) +===================================== + +This section documents various errors codes that mypy can generate if +you enable certain options. See :ref:`error-codes` for general +documentation about error codes. :ref:`error-code-list` documents +error codes that are enabled by default. + +Check that every function has an annotation [no-untyped-def] +------------------------------------------------------------ + +TODO + +Check that now untyped functions are called [no-untyped-call] +------------------------------------------------------------- + +TODO + +Check that cast is not redundant [redundant-cast] +------------------------------------------------- + +TODO + +Check that comparisons can be true [comparison-overlap] +------------------------------------------------------- + +TODO + +Check that unimported Any values are not used [no-any-unimported] +----------------------------------------------------------------- + +TODO + +Check that function does not return Any value [no-any-return] +------------------------------------------------------------- + +TODO diff --git a/docs/source/error_codes.rst b/docs/source/error_codes.rst index f1a3fad09aaed..dea888b6620aa 100644 --- a/docs/source/error_codes.rst +++ b/docs/source/error_codes.rst @@ -3,13 +3,14 @@ Error codes =========== -Mypy can optionally display an error code after a message. Error -codes serve two purposes: +Mypy can optionally display an error code such as ``[attr-defined]`` +after error messages. Error codes serve two purposes: 1. It's possible to silence specific error codes on a line using ``# type: ignore[code]``. -2. The error code can be looked up here in the documentation to provide - additional context. +2. The error code can be used to find documentation about the error. + +The next two topics document the various error codes mypy can report. Displaying error codes ---------------------- @@ -25,10 +26,11 @@ to display error codes. Error codes are shown inside square brackets: Silencing errors based on error codes ------------------------------------- -You can use ``# type: ignore[code, ...]`` to ignore all errors with -specific codes on a line. This can be used even if you have not -configured mypy to show error codes. Currently it's only possible to -disable arbitrary error codes on individual lines using this comment. +You can use a special comment ``# type: ignore[code, ...]`` to only +ignore errors with a specific error code (or codes) on a particular +line. This can be used even if you have not configured mypy to show +error codes. Currently it's only possible to disable arbitrary error +codes on individual lines using this comment. .. note:: @@ -36,8 +38,8 @@ disable arbitrary error codes on individual lines using this comment. certain optional error codes, such as ``--disallow-untype-defs``, which enables the ``no-untyped-def`` error code. -Here is how we ignore an error about an imported name mypy thinks is -undefined: +This example shows how to ignore an error about an imported name mypy +thinks is undefined: .. code-block:: python diff --git a/docs/source/index.rst b/docs/source/index.rst index dd09795dc4589..7080a496df2d4 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -66,6 +66,7 @@ Mypy is a static type checker for Python 3 and Python 2.7. error_codes error_code_list + error_code_list2 common_issues supported_python_features python36 From 7055ea3bc0f05f9a775e8a9fdfbec2d37363ae56 Mon Sep 17 00:00:00 2001 From: Jukka Lehtosalo Date: Mon, 2 Sep 2019 18:22:05 +0100 Subject: [PATCH 04/13] Document additional error codes --- docs/source/error_code_list.rst | 135 ++++++++++++++++++++++++++++---- 1 file changed, 121 insertions(+), 14 deletions(-) diff --git a/docs/source/error_code_list.rst b/docs/source/error_code_list.rst index e56e29b301147..1e3c7f1ad1ff3 100644 --- a/docs/source/error_code_list.rst +++ b/docs/source/error_code_list.rst @@ -33,7 +33,29 @@ Example: Check that attribute exists in each union item [union-attr] ----------------------------------------------------------- -TODO +If you access the attribute of a value with a union type, mypy checks +that the attribute is defined for every union item. Otherwise the +operation can fail at runtime. + +Example: + +.. code-block:: python + + from typing import Union + + class Cat: + def sleep(self) -> None: ... + def miaow(self) -> None: ... + + class Dog: + def sleep(self) -> None: ... + def follow_me(self) -> None: ... + + def f(animal: Union[Cat, Dog]) -> None: + # OK: 'sleep' is defined for both Cat and Dog + animal.sleep() + # Error: Item "Cat" of "Union[Cat, Dog]" has no attribute "follow_me" [union-attr] + animal.follow_me() Check that name is defined [name-defined] ----------------------------------------- @@ -89,17 +111,51 @@ Example: Check calls to overloaded functions [call-overload] --------------------------------------------------- -TODO +When you call an overloaded function, mypy checks that at least one of +the signatures of the overload items match the argument types in the +call. -Check validity of type annotations [valid-type] ------------------------------------------------ +Check validity of types [valid-type] +------------------------------------ -TODO +Mypy checks that each type annotation and any expression that +represents a type is a valid type. Examples of valid types include +classes, union types, callable types, type aliases, and literal types. +Examples of invalid types include bare integer literals, functions, +variables, and undefined names. Require annotation if variable type is unclear [var-annotated] -------------------------------------------------------------- -TODO +In some cases mypy can't infer the type of a variable without an +explicit annotation. Mypy treats this as an error. This often happens +when you initialize a variable with an empty collection, and mypy +can't infer the collection item type. Mypy replaces any parts of the +type it couldn't infer with ``Any``. + +Example with an error: + +.. code-block:: python + + class Bundle: + def __init__(self) -> None: + # Error: Need type annotation for 'items' + # (hint: "items: List[] = ...") [var-annotated] + self.items = [] + + reveal_type(Bundle().items) # list[Any] + +In this example we have an explicit annotation to silence the error: + +.. code-block:: python + + from typing import List + + class Bundle: + def __init__(self) -> None: + self.items: List[str] = [] # OK + + reveal_type(Bundle().items) # list[str] Check validity of overrides [override] -------------------------------------- @@ -205,27 +261,78 @@ TODO Check that called functions return a value [func-returns-value] --------------------------------------------------------------- -TODO +Mypy reports an error if you call a function with a ``None`` +return type and don't ignore the return value, as this is +usually (but not always) a programming error. For example, +the ``if f()`` check is always false since ``f`` returns +``None``: + +.. code-block:: python + + def f() -> None: + ... + + # "f" does not return a value [func-returns-value] + if f(): + print("not false") Check instantiation of abstract classes [abstract] -------------------------------------------------- -TODO +Mypy generates an error if you try to instantiate an abstract base +class (ABC). An abtract base class is a class with at least once +abstract method or attribute. (See also `Python +abc module documentation `_.) + +Sometimes a class is accidentally abstract, due to an +unimplemented abstract method, for example. In a case like this you +need to provide an implementation for the method to make the class +concrete (non-abstract). + +Example: + +.. code-block:: python + + from abc import ABCMeta, abstractmethod + + class Persistable(metaclass=ABCMeta): + @abstractmethod + def save(self) -> None: ... + + class Thing(Persistable): + def __init__(self) -> None: + ... + + ... # No "save" method + + # Cannot instantiate abstract class 'Thing' with abstract attribute 'save' [abstract] + t = Thing() Check the target of NewType [valid-newtype] ------------------------------------------- -TODO +The target of a ``NewType`` definition must be a class type. It can't +be a union type, ``Any``, or various other special types. + +You can also get this error also if the target has been imported from +a module mypy can't find the source for, since any such definitions +are treated by mypy as values with ``Any`` types. Report syntax errors [syntax] ----------------------------- -TODO +If the code being checked is not syntactically valid, mypy issues a +syntax error. Most, but not all, syntax errors are *blocking errors*: +they can't be ignored with a ``# type: ignore`` comment. Miscellaneous checks [misc] --------------------------- -Mypy performs numerous other, more rarely failing checks that don't -have a specific error codes. These use the ``misc`` error code. This -error code is not special. For example, you can ignore all errors in -this category by using ``# type: ignore[misc]`` comment. +Mypy performs numerous other, less commonly failing checks that don't +have specific error codes. These use the ``misc`` error code. Other +than being used for multiple unrelated errors, the ``misc`` error code +is not special in other ways. For example, you can ignore all errors +in this category by using ``# type: ignore[misc]`` comment. Since these +errors are not expected to be common, it's unlikely that you'll see +two *different* errors with the ``misc`` code on a single line -- though this +can certainly happen once in a while. From 4a1ea5d4bed1ca2d5c0670bc5ef24746c4d6e198 Mon Sep 17 00:00:00 2001 From: Jukka Lehtosalo Date: Tue, 3 Sep 2019 12:25:51 +0100 Subject: [PATCH 05/13] Document additional error codes --- docs/source/error_code_list.rst | 180 +++++++++++++++++++++++++++---- docs/source/error_code_list2.rst | 5 + 2 files changed, 162 insertions(+), 23 deletions(-) diff --git a/docs/source/error_code_list.rst b/docs/source/error_code_list.rst index 1e3c7f1ad1ff3..ba45f9b30eb26 100644 --- a/docs/source/error_code_list.rst +++ b/docs/source/error_code_list.rst @@ -27,8 +27,18 @@ Example: r = Resouce('x') print(r.name) # OK - print(r.id) # "Resource" has no attribute "id" [attr-defined] - r.id = 5 # "Resource" has no attribute "id" [attr-defined] + print(r.id) # Error: "Resource" has no attribute "id" [attr-defined] + r.id = 5 # Error: "Resource" has no attribute "id" [attr-defined] + +This error code is also generated if the imported name is not defined +in a ``from ... import`` statement (as long as the target module can +be found): + +.. code-block:: python + + # Error: Module 'os' has no attribute 'non_existent' [attr-defined] + from os import non_existent + Check that attribute exists in each union item [union-attr] ----------------------------------------------------------- @@ -195,68 +205,192 @@ Example: Check that function returns a value [return] -------------------------------------------- -TODO +If a function has a non-``None`` return type, mypy expects that the +function always explicitly returns a value (or raises an exception). +The function should not fall off the end of the function, since this +is often a bug. + +Example: + +.. code-block:: python + # Error: Missing return statement [return] + def show(x: int) -> int: + print(x) + + # Error: Missing return statement [return] + def pred1(x: int) -> int: + if x > 0: + return x - 1 + + # OK + def pred2(x: int) -> int: + if x > 0: + return x - 1 + else: + raise ValueError('not defined for zero') Check that return value is compatible [return-value] ---------------------------------------------------- -TODO +Mypy checks that the returned value is compatible with the type +signature of the function. -Check compatibility of assignment statement [assignment] --------------------------------------------------------- +Example: -TODO +.. code-block:: python -Check that type arguments exist [type-arg] ------------------------------------------- + def func(x: int) -> str: + # Error: Incompatible return value type (got "int", expected "str") [return-value] + return x + 1 -TODO +Check types in assignment statement [assignment] +------------------------------------------------ -Check type variable values [type-var] -------------------------------------- +Mypy checks that the assigned expression (or expressions) is compatible +with the assignment target (or targets). -TODO +Example: -Check indexing operations [index] ---------------------------------- +.. code-block:: python + + class Resource: + def __init__(self, name: str) -> None: + self.name = name + + r = Resource('a') + + r.name = 'b' # OK + + # Error: Incompatible types in assignment (expression has type "int", + # variable has type "str") [assignment] + r.name = 5 + +Check type variable values [type-var] +------------------------------------- -TODO +Mypy checks that value of a type variable is compatible with a value +restriction or the upper bound type. Check uses of various operators [operator] ------------------------------------------ -TODO +Mypy checks that operands support each binary or unary operation. Indexing +operations are so common that they have their own error code ``[index]`` +(see below). + +Check indexing operations [index] +--------------------------------- + +Mypy checks that the value in indexing operation such as ``x[y]`` supports +indexing, and that the index expression has a valid type. Check list items [list-item] ---------------------------- -TODO +When constructing a list using ``[item, ...]``, mypy checks that each item +is compatible with the list type that is inferred from the surrounding +context. + +Example: + +.. code-block:: python + + from typing import List + + # Error: List item 0 has incompatible type "int"; expected "str" [list-item] + a: List[str] = [0] Check dict items [dict-item] ---------------------------- -TODO +When constructing a dictionary using ``{key: value, ...]`` or ``dict(key=value, ...)``, +mypy checks that each key and value is compatible with the dictionary type that is +inferred from the surrounding context. + +Example: + +.. code-block:: python + + from typing import Dict + + # Error: Dict entry 0 has incompatible type "str": "str"; expected "str": "int" [dict-item] + d: Dict[str, int] = {'key': 'value'} Check TypedDict items [typeddict-item] -------------------------------------- -TODO +When constructing a TypedDict object, mypy checks that each key and value is compatible +with the TypedDict type that is inferred from the surrounding context. + +Example: + +.. code-block:: python + + from typing_extensions import TypedDict + + class Point(TypedDict): + x: int + y: int + + # Error: Incompatible types (expression has type "float", + # TypedDict item "x" has type "int") [typeddict-item] + p: Point = {'x': 1.2, 'y': 4} + Check that type of target is known [has-type] --------------------------------------------- -TODO +Mypy may generate an error when mypy hasn't inferred any type for a +variable being referenced. This can happen for forward references and +references across modules that form an import cycle. When this +happens, the reference gets an implicit ``Any`` type. + +To work around this issue, you can add an explicit type annotation to +the target variable or attribute. Sometimes you can also reorganize the +code so that the definition of the variable is placed earlier than the +reference in a source file. Untangling cyclic imports may also help. Check that import target can be found [import] ---------------------------------------------- -TODO +Mypy generates an error if it can't find the source code or a stub file +of an imported module. + +Example: + +.. code-block:: python + + # Error: Cannot find module named 'acme' [import] + import acme + +See :ref:`ignore-missing-imports` for how to work around these errors. Check that each name is defined once [no-redef] ----------------------------------------------- -TODO +Mypy may generate an error if you have multiple definitions for a name. +The reason for this that this is often an error (the second definition +may overwrite the first one). Also, mypy often can't be able to determine +whether references point to the first or the second definition, which +would compromise type checking. + +If you ignore this error, each reference to the defined name refers to +the *first* definition. + +Example: + +.. code-block:: python + + class A: + def __init__(self, x: int) -> None: ... + + class A: # Error: Name 'A' already defined on line 1 [no-redef] + def __init__(self, x: str) -> None: ... + + # Argument 1 to "A" has incompatible type "str"; expected "int" + # (the first definition wins!) + A('x') Check that called functions return a value [func-returns-value] --------------------------------------------------------------- diff --git a/docs/source/error_code_list2.rst b/docs/source/error_code_list2.rst index 0201c41ed0782..9e4e2c849c342 100644 --- a/docs/source/error_code_list2.rst +++ b/docs/source/error_code_list2.rst @@ -8,6 +8,11 @@ you enable certain options. See :ref:`error-codes` for general documentation about error codes. :ref:`error-code-list` documents error codes that are enabled by default. +Check that type arguments exist [type-arg] +------------------------------------------ + +TODO + Check that every function has an annotation [no-untyped-def] ------------------------------------------------------------ From 153466c34917b6cbb1527451633e6dbba6ec8f64 Mon Sep 17 00:00:00 2001 From: Jukka Lehtosalo Date: Tue, 3 Sep 2019 13:44:11 +0100 Subject: [PATCH 06/13] Document more error codes --- docs/source/error_code_list2.rst | 137 ++++++++++++++++++++++++++++--- 1 file changed, 124 insertions(+), 13 deletions(-) diff --git a/docs/source/error_code_list2.rst b/docs/source/error_code_list2.rst index 9e4e2c849c342..aa50d189e564c 100644 --- a/docs/source/error_code_list2.rst +++ b/docs/source/error_code_list2.rst @@ -3,42 +3,153 @@ List of error codes (optional checks) ===================================== -This section documents various errors codes that mypy can generate if -you enable certain options. See :ref:`error-codes` for general +This section documents various errors codes that mypy generates only +if you enable certain options. See :ref:`error-codes` for general documentation about error codes. :ref:`error-code-list` documents error codes that are enabled by default. Check that type arguments exist [type-arg] ------------------------------------------ -TODO +If you use ``--disallow-any-generics``, mypy requires that each generic +type has values for each type argument. For example, the types ``List`` or +``dict`` would be rejected. You should instead use types like ``List[int]`` or +``Dict[str, int]``. Any omitted generic type arguments get implicit ``Any`` +values. The type ``List`` is equivalent to ``List[Any]``, and so on. + +Example: + +.. code-block:: python + + # mypy: disallow-any-generics + from typing import List + + # Error: Missing type parameters for generic type "List" [type-arg] + def remove_dups(items: List) -> List: + ... Check that every function has an annotation [no-untyped-def] ------------------------------------------------------------ -TODO +If you use ``--disallow-untyped-defs``, mypy requires that each function +has an annotation (either a Python 3 annotation or a type comment). -Check that now untyped functions are called [no-untyped-call] -------------------------------------------------------------- +Example: + +.. code-block:: python + + # mypy: disallow-untyped-defs + + def inc(x): # Error: Function is missing a type annotation [no-untyped-def] + return x + 1 + + def inc(x: int) -> int: # OK + return x + 1 + + class Counter: + # Error: Function is missing a type annotation [no-untyped-def] + def __init__(self): + self.value = 0 -TODO + class Counter2: + # OK: An explicit "-> None" is needed if "__init__" takes no arguments + def __init__(self) -> None: + self.value = 0 Check that cast is not redundant [redundant-cast] ------------------------------------------------- -TODO +If you use ``--warn-redundant-casts``, mypy will generate an error if the source +type of a cast is the same as the target type. + +Example: + +.. code-block:: python + + # mypy: warn-redundant-casts + from typing import cast + + Count = int + + def example(x: Count) -> int: + # Error: Redundant cast to "int" [redundant-cast] + return cast(int, x) Check that comparisons can be true [comparison-overlap] ------------------------------------------------------- -TODO +If you use ``--strict-equality``, mypy will generate an error if it thinks that +a comparison operation is always true or false. These are often bugs. Sometimes +mypy is too picky and the test can actually succeed. Instead of disabling strict +equality checking everywhere, you can use ``# type: ignore[comparison-overlap]`` +to ignore the issue on a particular line only. + +Example: + +.. code-block:: python + + # mypy: strict-equality + + def is_magic(x: int) -> bool: + # Error: Non-overlapping equality check (left operand type: "int", + # right operand type: "str") [comparison-overlap] + return x == 'magic' -Check that unimported Any values are not used [no-any-unimported] ------------------------------------------------------------------ +Check that no untyped functions are called [no-untyped-call] +------------------------------------------------------------ + +If you use ``--disallow-untyped-calls``, mypy generates an error when you +calle an unannotated function in an annotated function. + +Example: + +.. code-block:: python + + # mypy: disallow-untyped-calls + + def do_it() -> None: + # Error: Call to untyped function "bad" in typed context [no-untyped-call] + bad() + + def bad(): + ... -TODO Check that function does not return Any value [no-any-return] ------------------------------------------------------------- -TODO +If you use ``--warn-return-any``, mypy generates an error if you return a +value with an ``Any`` type in a function that is annotated to return a +non-``Any`` value. + +Example: + +.. code-block:: python + + # mypy: warn-return-any + + def fields(s): + return s.split(',') + + def first_field(x: str) -> str: + # Error: Returning Any from function declared to return "str" [no-any-return] + return fields(x)[0] + +Check that types have no Any components due to missing imports [no-any-unimported] +---------------------------------------------------------------------------------- + +If you use ``--disallow-any-unimported``, mypy generates an error if a component of +a type becomes ``Any`` because mypy couldn't resolve an import. + +In this example, we assume that mypy can't find module ``animals``, which means +that ``Cat`` falls back to ``Any`` in a type annotation: + +.. code-block:: python + + # mypy: disallow-any-unimported + + from animals import Cat # type: ignore + + # Error: Argument 1 to "feed" becomes "Any" due to an unfollowed import [no-any-unimported] + def feed(cat: Cat) -> None: + ... From 007b230ec77c5949ae9b6af79ec4a500a8ed361a Mon Sep 17 00:00:00 2001 From: Jukka Lehtosalo Date: Tue, 3 Sep 2019 14:26:50 +0100 Subject: [PATCH 07/13] Various updates --- docs/source/error_code_list.rst | 78 +++++++++++++++++----------- docs/source/error_code_list2.rst | 4 +- test-data/unit/check-errorcodes.test | 4 +- 3 files changed, 53 insertions(+), 33 deletions(-) diff --git a/docs/source/error_code_list.rst b/docs/source/error_code_list.rst index ba45f9b30eb26..8e0b6c029cafa 100644 --- a/docs/source/error_code_list.rst +++ b/docs/source/error_code_list.rst @@ -1,20 +1,21 @@ .. _error-code-list: -List of error codes (default checks) -==================================== +Error codes enabled by default +============================== This section documents various errors codes that mypy can generate with default options. See :ref:`error-codes` for general documentation -about error codes. See :ref:`error-codes-optional` for additional +about error codes. :ref:`error-codes-optional` documents additional error codes that you can enable. Check that attribute exists [attr-defined] ------------------------------------------ -Mypy that an attribute is defined in the target class or module. This -applies to reading an attribute and setting an attribute. Attribute -assignments in a class body or through the ``self`` argument are -considered to define new attributes. Mypy doesn't allow defining +Mypy checks that an attribute is defined in the target class or module +when using the dot operator. This applies to both getting and setting +an attribute. Attribute assignments in the class body or through the +``self`` argument in a method define new attributes, and they don't +generate ``attr-defined`` errors. Mypy doesn't allow defining attributes outside a class definition. Example: @@ -30,22 +31,21 @@ Example: print(r.id) # Error: "Resource" has no attribute "id" [attr-defined] r.id = 5 # Error: "Resource" has no attribute "id" [attr-defined] -This error code is also generated if the imported name is not defined -in a ``from ... import`` statement (as long as the target module can -be found): +This error code is also generated if an imported name is not defined +in the module in a ``from ... import`` statement (as long as the +target module can be found): .. code-block:: python # Error: Module 'os' has no attribute 'non_existent' [attr-defined] from os import non_existent - Check that attribute exists in each union item [union-attr] ----------------------------------------------------------- If you access the attribute of a value with a union type, mypy checks -that the attribute is defined for every union item. Otherwise the -operation can fail at runtime. +that the attribute is defined for *every* union item type. Otherwise the +operation can fail at runtime. This also applies to optional types. Example: @@ -61,24 +61,41 @@ Example: def sleep(self) -> None: ... def follow_me(self) -> None: ... - def f(animal: Union[Cat, Dog]) -> None: + def func(animal: Union[Cat, Dog]) -> None: # OK: 'sleep' is defined for both Cat and Dog animal.sleep() # Error: Item "Cat" of "Union[Cat, Dog]" has no attribute "follow_me" [union-attr] animal.follow_me() +You can often work around these errors by using ``assert isinstance(obj, ClassName)`` +or ``assert obj is not None`` to tell mypy that you know that the type is more specific +than what mypy thinks. Alternatively, you can cast to the target type: + +Example: + +.. code-block:: python + + from typing import Union, cast + + ... # Definitions of Cat and Dog omitted + + def func(animal: Union[Cat, Dog]) -> None: + animal.sleep() # OK + cast(Dog, animal).follow_me() # OK + Check that name is defined [name-defined] ----------------------------------------- -Mypy expects that all name references contain a definitinon, such as -an assignment, function definition or an import. This can catch missing -definitions, missing imports, and typos. +Mypy expects that all references to names have a corresponding +definition in an active scope, such as an assignment, function +definition or an import. This can catch missing definitions, missing +imports, and typos. -Example: +This example accidentally calls ``sort()`` instead of ``sorted()``: .. code-block:: python - x = func(1) # Name 'func' is not defined [name-defined] + x = sort([3, 2, 4]) # Error: Name 'sort' is not defined [name-defined] Check arguments in calls [call-arg] ----------------------------------- @@ -96,13 +113,13 @@ Example: print('hello', name) greet('jack') # OK - greet('hi', 'jack') # Too many arguments for "greet" [call-arg] + greet('jill', 'jack') # Error: Too many arguments for "greet" [call-arg] Check argument types [arg-type] ------------------------------- Mypy checks that argument types in a call match the declared argument -types in the signature. +types in the signature of the called function (if one exists). Example: @@ -114,8 +131,8 @@ Example: return x[0] if x else 0 t = (5, 4) - # Argument 1 to "first" has incompatible type "Tuple[int, int]"; - # expected "List[int]" [arg-type] + # Error: Argument 1 to "first" has incompatible type "Tuple[int, int]"; + # expected "List[int]" [arg-type] print(first(t)) Check calls to overloaded functions [call-overload] @@ -132,7 +149,7 @@ Mypy checks that each type annotation and any expression that represents a type is a valid type. Examples of valid types include classes, union types, callable types, type aliases, and literal types. Examples of invalid types include bare integer literals, functions, -variables, and undefined names. +variables, and modules. Require annotation if variable type is unclear [var-annotated] -------------------------------------------------------------- @@ -150,7 +167,7 @@ Example with an error: class Bundle: def __init__(self) -> None: # Error: Need type annotation for 'items' - # (hint: "items: List[] = ...") [var-annotated] + # (hint: "items: List[] = ...") [var-annotated] self.items = [] reveal_type(Bundle().items) # list[Any] @@ -173,12 +190,13 @@ Check validity of overrides [override] Mypy checks that an overridden method or attribute is compatible with the base class. A method in a subclass must accept all arguments that the base class method accepts, and the return type must conform -to the return type in the base class. +to the return type in the base class (Liskov substitution principle). Argument typess can be more general is a subclass (i.e., they can vary -contravariantly). Return type can be narrowed in a subclass (i.e., it -can vary covariantly). It's okay to define additional arguments in -a subclass method, as long all extra arguments can be left out. +contravariantly). The return type can be narrowed in a subclass +(i.e., it can vary covariantly). It's okay to define additional +arguments in a subclass method, as long all extra arguments have default +values or can be left out. Example: @@ -197,7 +215,7 @@ Example: ... class DerivedBad(Base): - # Argument 1 of "method" is incompatible with "Base" [override] + # Error: Argument 1 of "method" is incompatible with "Base" [override] def method(self, arg: bool) -> int: ... diff --git a/docs/source/error_code_list2.rst b/docs/source/error_code_list2.rst index aa50d189e564c..65981d6228172 100644 --- a/docs/source/error_code_list2.rst +++ b/docs/source/error_code_list2.rst @@ -1,7 +1,7 @@ .. _error-codes-optional: -List of error codes (optional checks) -===================================== +Error codes for optional checks +=============================== This section documents various errors codes that mypy generates only if you enable certain options. See :ref:`error-codes` for general diff --git a/test-data/unit/check-errorcodes.test b/test-data/unit/check-errorcodes.test index fef3290fc42a1..fcbd495c43935 100644 --- a/test-data/unit/check-errorcodes.test +++ b/test-data/unit/check-errorcodes.test @@ -266,6 +266,7 @@ class B(A): a: A a.x = '' # E: Incompatible types in assignment (expression has type "str", variable has type "int") [assignment] + [case testErrorCodeMissingTypeArg] # flags: --disallow-any-generics from typing import List, TypeVar @@ -400,7 +401,8 @@ c = {} # E: Expected TypedDict key 'x' but found no keys [typeddict-item] [builtins fixtures/dict.pyi] [case testErrorCodeCannotDetermineType] -x # E: Cannot determine type of 'x' [has-type] +y = x # E: Cannot determine type of 'x' [has-type] +reveal_type(y) # N: Revealed type is 'Any' x = None [case testErrorCodeRedundantCast] From 6233b7915d26c7f3da3b44445cf543827373bfc3 Mon Sep 17 00:00:00 2001 From: Jukka Lehtosalo Date: Tue, 3 Sep 2019 16:06:23 +0100 Subject: [PATCH 08/13] Various updates --- docs/source/error_code_list.rst | 96 ++++++++++++++++++-------------- docs/source/error_code_list2.rst | 35 ++++++++---- 2 files changed, 77 insertions(+), 54 deletions(-) diff --git a/docs/source/error_code_list.rst b/docs/source/error_code_list.rst index 8e0b6c029cafa..56a08eb274d3d 100644 --- a/docs/source/error_code_list.rst +++ b/docs/source/error_code_list.rst @@ -265,8 +265,8 @@ Example: Check types in assignment statement [assignment] ------------------------------------------------ -Mypy checks that the assigned expression (or expressions) is compatible -with the assignment target (or targets). +Mypy checks that the assigned expression is compatible with the +assignment target (or targets). Example: @@ -276,12 +276,12 @@ Example: def __init__(self, name: str) -> None: self.name = name - r = Resource('a') + r = Resource('A') - r.name = 'b' # OK + r.name = 'B' # OK # Error: Incompatible types in assignment (expression has type "int", - # variable has type "str") [assignment] + # variable has type "str") [assignment] r.name = 5 Check type variable values [type-var] @@ -293,15 +293,16 @@ restriction or the upper bound type. Check uses of various operators [operator] ------------------------------------------ -Mypy checks that operands support each binary or unary operation. Indexing -operations are so common that they have their own error code ``[index]`` -(see below). +Mypy checks that operands support a binary or unary operation, such as +``+`` or ``~``. Indexing operations are so common that they have their +own error code ``index`` (see below). Check indexing operations [index] --------------------------------- -Mypy checks that the value in indexing operation such as ``x[y]`` supports -indexing, and that the index expression has a valid type. +Mypy checks that the indexed value in indexing operation such as +``x[y]`` supports indexing, and that the index expression has a valid +type. Check list items [list-item] ---------------------------- @@ -322,7 +323,7 @@ Example: Check dict items [dict-item] ---------------------------- -When constructing a dictionary using ``{key: value, ...]`` or ``dict(key=value, ...)``, +When constructing a dictionary using ``{key: value, ...}`` or ``dict(key=value, ...)``, mypy checks that each key and value is compatible with the dictionary type that is inferred from the surrounding context. @@ -359,21 +360,23 @@ Example: Check that type of target is known [has-type] --------------------------------------------- -Mypy may generate an error when mypy hasn't inferred any type for a -variable being referenced. This can happen for forward references and -references across modules that form an import cycle. When this -happens, the reference gets an implicit ``Any`` type. +Mypy sometimes generates an error when it hasn't inferred any type for +a variable being referenced. This can happen for references to +variables that have not been initialized yet, and for references +across modules that form an import cycle. When this happens, the +reference gets an implicit ``Any`` type. -To work around this issue, you can add an explicit type annotation to -the target variable or attribute. Sometimes you can also reorganize the -code so that the definition of the variable is placed earlier than the -reference in a source file. Untangling cyclic imports may also help. +To work around this error, you can add an explicit type annotation to +the target variable or attribute. Sometimes you can also reorganize +the code so that the definition of the variable is placed earlier than +the reference to the variable in a source file. Untangling cyclic +imports may also help. Check that import target can be found [import] ---------------------------------------------- Mypy generates an error if it can't find the source code or a stub file -of an imported module. +for an imported module. Example: @@ -387,11 +390,11 @@ See :ref:`ignore-missing-imports` for how to work around these errors. Check that each name is defined once [no-redef] ----------------------------------------------- -Mypy may generate an error if you have multiple definitions for a name. -The reason for this that this is often an error (the second definition -may overwrite the first one). Also, mypy often can't be able to determine -whether references point to the first or the second definition, which -would compromise type checking. +Mypy may generate an error if you have multiple definitions for a name +in the same namespace. The reason is that this is often an error, as +the second definition may overwrite the first one. Also, mypy often +can't be able to determine whether references point to the first or +the second definition, which would compromise type checking. If you ignore this error, each reference to the defined name refers to the *first* definition. @@ -406,8 +409,8 @@ Example: class A: # Error: Name 'A' already defined on line 1 [no-redef] def __init__(self, x: str) -> None: ... - # Argument 1 to "A" has incompatible type "str"; expected "int" - # (the first definition wins!) + # Error: Argument 1 to "A" has incompatible type "str"; expected "int" + # (the first definition wins!) A('x') Check that called functions return a value [func-returns-value] @@ -415,16 +418,20 @@ Check that called functions return a value [func-returns-value] Mypy reports an error if you call a function with a ``None`` return type and don't ignore the return value, as this is -usually (but not always) a programming error. For example, -the ``if f()`` check is always false since ``f`` returns -``None``: +usually (but not always) a programming error. + +In this example, the ``if f()`` check is always false since ``f`` +returns ``None``: .. code-block:: python def f() -> None: ... - # "f" does not return a value [func-returns-value] + # OK: we don't do anything with the return value + f() + + # Error: "f" does not return a value [func-returns-value] if f(): print("not false") @@ -432,14 +439,14 @@ Check instantiation of abstract classes [abstract] -------------------------------------------------- Mypy generates an error if you try to instantiate an abstract base -class (ABC). An abtract base class is a class with at least once +class (ABC). An abtract base class is a class with at least one abstract method or attribute. (See also `Python abc module documentation `_.) -Sometimes a class is accidentally abstract, due to an -unimplemented abstract method, for example. In a case like this you -need to provide an implementation for the method to make the class -concrete (non-abstract). +Sometimes a class is made accidentally abstract, often due to an +unimplemented abstract method. In a case like this you need to provide +an implementation for the method to make the class concrete +(non-abstract). Example: @@ -447,17 +454,17 @@ Example: from abc import ABCMeta, abstractmethod - class Persistable(metaclass=ABCMeta): + class Persistent(metaclass=ABCMeta): @abstractmethod def save(self) -> None: ... - class Thing(Persistable): + class Thing(Persistent): def __init__(self) -> None: ... ... # No "save" method - # Cannot instantiate abstract class 'Thing' with abstract attribute 'save' [abstract] + # Error: Cannot instantiate abstract class 'Thing' with abstract attribute 'save' [abstract] t = Thing() Check the target of NewType [valid-newtype] @@ -466,9 +473,9 @@ Check the target of NewType [valid-newtype] The target of a ``NewType`` definition must be a class type. It can't be a union type, ``Any``, or various other special types. -You can also get this error also if the target has been imported from -a module mypy can't find the source for, since any such definitions -are treated by mypy as values with ``Any`` types. +You can also get this error if the target has been imported from a +module mypy can't find the source for, since any such definitions are +treated by mypy as values with ``Any`` types. Report syntax errors [syntax] ----------------------------- @@ -488,3 +495,8 @@ in this category by using ``# type: ignore[misc]`` comment. Since these errors are not expected to be common, it's unlikely that you'll see two *different* errors with the ``misc`` code on a single line -- though this can certainly happen once in a while. + +.. note:: + + Future mypy versions will likely add new error codes for some errors + that currently use the ``misc`` error code. diff --git a/docs/source/error_code_list2.rst b/docs/source/error_code_list2.rst index 65981d6228172..84eca92cb5cea 100644 --- a/docs/source/error_code_list2.rst +++ b/docs/source/error_code_list2.rst @@ -8,6 +8,13 @@ if you enable certain options. See :ref:`error-codes` for general documentation about error codes. :ref:`error-code-list` documents error codes that are enabled by default. +.. note:: + + The examples in this section use :ref:`inline configuration + ` to specify mypy options. You can also set the same + options by using a :ref:`configuration file ` or + :ref:`command-line options `. + Check that type arguments exist [type-arg] ------------------------------------------ @@ -22,6 +29,7 @@ Example: .. code-block:: python # mypy: disallow-any-generics + from typing import List # Error: Missing type parameters for generic type "List" [type-arg] @@ -43,7 +51,7 @@ Example: def inc(x): # Error: Function is missing a type annotation [no-untyped-def] return x + 1 - def inc(x: int) -> int: # OK + def inc_ok(x: int) -> int: # OK return x + 1 class Counter: @@ -51,7 +59,7 @@ Example: def __init__(self): self.value = 0 - class Counter2: + class CounterOk: # OK: An explicit "-> None" is needed if "__init__" takes no arguments def __init__(self) -> None: self.value = 0 @@ -67,6 +75,7 @@ Example: .. code-block:: python # mypy: warn-redundant-casts + from typing import cast Count = int @@ -75,14 +84,15 @@ Example: # Error: Redundant cast to "int" [redundant-cast] return cast(int, x) -Check that comparisons can be true [comparison-overlap] -------------------------------------------------------- +Check that comparisons are overlapping [comparison-overlap] +----------------------------------------------------------- -If you use ``--strict-equality``, mypy will generate an error if it thinks that -a comparison operation is always true or false. These are often bugs. Sometimes -mypy is too picky and the test can actually succeed. Instead of disabling strict -equality checking everywhere, you can use ``# type: ignore[comparison-overlap]`` -to ignore the issue on a particular line only. +If you use ``--strict-equality``, mypy will generate an error if it +thinks that a comparison operation is always true or false. These are +often bugs. Sometimes mypy is too picky and the comparison can +actually be useful. Instead of disabling strict equality checking +everywhere, you can use ``# type: ignore[comparison-overlap]`` to +ignore the issue on a particular line only. Example: @@ -99,7 +109,7 @@ Check that no untyped functions are called [no-untyped-call] ------------------------------------------------------------ If you use ``--disallow-untyped-calls``, mypy generates an error when you -calle an unannotated function in an annotated function. +call an unannotated function in an annotated function. Example: @@ -139,9 +149,10 @@ Check that types have no Any components due to missing imports [no-any-unimporte ---------------------------------------------------------------------------------- If you use ``--disallow-any-unimported``, mypy generates an error if a component of -a type becomes ``Any`` because mypy couldn't resolve an import. +a type becomes ``Any`` because mypy couldn't resolve an import. These "stealth" +``Any`` types can be surprising and accidentally cause imprecise type checking. -In this example, we assume that mypy can't find module ``animals``, which means +In this example, we assume that mypy can't find the module ``animals``, which means that ``Cat`` falls back to ``Any`` in a type annotation: .. code-block:: python From 8435d9281abb3c7333956a671c987616828482ce Mon Sep 17 00:00:00 2001 From: Jukka Lehtosalo Date: Wed, 4 Sep 2019 13:16:42 +0100 Subject: [PATCH 09/13] Respond to feedback --- docs/source/command_line.rst | 11 ++++- docs/source/config_file.rst | 3 ++ docs/source/error_code_list.rst | 73 +++++++++++++++++--------------- docs/source/error_code_list2.rst | 4 +- docs/source/error_codes.rst | 10 +++-- 5 files changed, 59 insertions(+), 42 deletions(-) diff --git a/docs/source/command_line.rst b/docs/source/command_line.rst index f2c810a254f72..c8b1e6eaa2c2c 100644 --- a/docs/source/command_line.rst +++ b/docs/source/command_line.rst @@ -497,12 +497,19 @@ in error messages. main.py:3: error: Unsupported operand types for + ("int" and "str") ``--show-column-numbers`` - This flag will add column offsets to error messages, - for example, the following indicates an error in line 12, column 9 + This flag will add column offsets to error messages. + For example, the following indicates an error in line 12, column 9 (note that column offsets are 0-based):: main.py:12:9: error: Unsupported operand types for / ("int" and "str") +``--show-error-codes`` + This flag will add an error code ``[]`` to error messages. The error + code is shown after each error message:: + + prog.py:1: error: "str" has no attribute "trim" [attr-defined] + + See :ref:`error-codes` for more information. .. _incremental: diff --git a/docs/source/config_file.rst b/docs/source/config_file.rst index 27019b0b7e825..1c08a3d0c74d4 100644 --- a/docs/source/config_file.rst +++ b/docs/source/config_file.rst @@ -441,6 +441,9 @@ section of the command line docs. ``show_column_numbers`` (bool, default False) Shows column numbers in error messages. +``show_error_codes`` (bool, default False) + Shows error codes in error messages. See :ref:`error-codes` for more information. + Advanced options ---------------- diff --git a/docs/source/error_code_list.rst b/docs/source/error_code_list.rst index 56a08eb274d3d..63683a7aee9dd 100644 --- a/docs/source/error_code_list.rst +++ b/docs/source/error_code_list.rst @@ -13,10 +13,9 @@ Check that attribute exists [attr-defined] Mypy checks that an attribute is defined in the target class or module when using the dot operator. This applies to both getting and setting -an attribute. Attribute assignments in the class body or through the -``self`` argument in a method define new attributes, and they don't -generate ``attr-defined`` errors. Mypy doesn't allow defining -attributes outside a class definition. +an attribute. New attributes are defined by assignments in the class +body, or assignments to ``self.x`` in methods. These assignments don't +generate ``attr-defined`` errors. Example: @@ -44,8 +43,9 @@ Check that attribute exists in each union item [union-attr] ----------------------------------------------------------- If you access the attribute of a value with a union type, mypy checks -that the attribute is defined for *every* union item type. Otherwise the -operation can fail at runtime. This also applies to optional types. +that the attribute is defined for *every* type in that +union. Otherwise the operation can fail at runtime. This also applies +to optional types. Example: @@ -69,19 +69,7 @@ Example: You can often work around these errors by using ``assert isinstance(obj, ClassName)`` or ``assert obj is not None`` to tell mypy that you know that the type is more specific -than what mypy thinks. Alternatively, you can cast to the target type: - -Example: - -.. code-block:: python - - from typing import Union, cast - - ... # Definitions of Cat and Dog omitted - - def func(animal: Union[Cat, Dog]) -> None: - animal.sleep() # OK - cast(Dog, animal).follow_me() # OK +than what mypy thinks. Check that name is defined [name-defined] ----------------------------------------- @@ -155,10 +143,11 @@ Require annotation if variable type is unclear [var-annotated] -------------------------------------------------------------- In some cases mypy can't infer the type of a variable without an -explicit annotation. Mypy treats this as an error. This often happens -when you initialize a variable with an empty collection, and mypy -can't infer the collection item type. Mypy replaces any parts of the -type it couldn't infer with ``Any``. +explicit annotation. Mypy treats this as an error. This typically +happens when you initialize a variable with an empty collection or +``None``. If mypy can't infer the collection item type, mypy replaces +any parts of the type it couldn't infer with ``Any`` and generates an +error. Example with an error: @@ -172,7 +161,7 @@ Example with an error: reveal_type(Bundle().items) # list[Any] -In this example we have an explicit annotation to silence the error: +To address this, we add an explicit annotation: .. code-block:: python @@ -192,11 +181,11 @@ the base class. A method in a subclass must accept all arguments that the base class method accepts, and the return type must conform to the return type in the base class (Liskov substitution principle). -Argument typess can be more general is a subclass (i.e., they can vary +Argument types can be more general is a subclass (i.e., they can vary contravariantly). The return type can be narrowed in a subclass (i.e., it can vary covariantly). It's okay to define additional arguments in a subclass method, as long all extra arguments have default -values or can be left out. +values or can be left out (``*args``, for example). Example: @@ -290,6 +279,22 @@ Check type variable values [type-var] Mypy checks that value of a type variable is compatible with a value restriction or the upper bound type. +Example: + +.. code-block:: python + + from typing import TypeVar + + T1 = TypeVar('T1', int, float) + + def add(x: T1, y: T1) -> T1: + return x + y + + add(4, 5.5) # OK + + # Error: Value of type variable "T1" of "add" cannot be "str" [type-var] + add('x', 'y') + Check uses of various operators [operator] ------------------------------------------ @@ -396,7 +401,7 @@ the second definition may overwrite the first one. Also, mypy often can't be able to determine whether references point to the first or the second definition, which would compromise type checking. -If you ignore this error, each reference to the defined name refers to +If you silence this error, all references to the defined name refer to the *first* definition. Example: @@ -413,7 +418,7 @@ Example: # (the first definition wins!) A('x') -Check that called functions return a value [func-returns-value] +Check that called function returns a value [func-returns-value] --------------------------------------------------------------- Mypy reports an error if you call a function with a ``None`` @@ -474,7 +479,7 @@ The target of a ``NewType`` definition must be a class type. It can't be a union type, ``Any``, or various other special types. You can also get this error if the target has been imported from a -module mypy can't find the source for, since any such definitions are +module whose source mypy cannot find, since any such definitions are treated by mypy as values with ``Any`` types. Report syntax errors [syntax] @@ -490,11 +495,11 @@ Miscellaneous checks [misc] Mypy performs numerous other, less commonly failing checks that don't have specific error codes. These use the ``misc`` error code. Other than being used for multiple unrelated errors, the ``misc`` error code -is not special in other ways. For example, you can ignore all errors -in this category by using ``# type: ignore[misc]`` comment. Since these -errors are not expected to be common, it's unlikely that you'll see -two *different* errors with the ``misc`` code on a single line -- though this -can certainly happen once in a while. +is not special. For example, you can ignore all errors in this +category by using ``# type: ignore[misc]`` comment. Since these errors +are not expected to be common, it's unlikely that you'll see two +*different* errors with the ``misc`` code on a single line -- though +this can certainly happen once in a while. .. note:: diff --git a/docs/source/error_code_list2.rst b/docs/source/error_code_list2.rst index 84eca92cb5cea..034b407dbc158 100644 --- a/docs/source/error_code_list2.rst +++ b/docs/source/error_code_list2.rst @@ -39,8 +39,8 @@ Example: Check that every function has an annotation [no-untyped-def] ------------------------------------------------------------ -If you use ``--disallow-untyped-defs``, mypy requires that each function -has an annotation (either a Python 3 annotation or a type comment). +If you use ``--disallow-untyped-defs``, mypy requires that all functions +have annotations (either a Python 3 annotation or a type comment). Example: diff --git a/docs/source/error_codes.rst b/docs/source/error_codes.rst index dea888b6620aa..c9033bc3c5480 100644 --- a/docs/source/error_codes.rst +++ b/docs/source/error_codes.rst @@ -4,12 +4,15 @@ Error codes =========== Mypy can optionally display an error code such as ``[attr-defined]`` -after error messages. Error codes serve two purposes: +after each error message. Error codes serve two purposes: 1. It's possible to silence specific error codes on a line using ``# type: ignore[code]``. 2. The error code can be used to find documentation about the error. +Most error codes are shared between multiple related error messages. +Error codes may change in future mypy releases. + The next two topics document the various error codes mypy can report. Displaying error codes @@ -43,7 +46,6 @@ thinks is undefined: .. code-block:: python - # Assume 'foo' is defined in 'foolib', even though mypy - # can't see the definition. - + # 'foo' is defined in 'foolib', even though mypy can't see the + # definition. from foolib import foo # type: ignore[attr-defined] From 812ec6f33c43a44c596a3357e5bbb52f050f6f29 Mon Sep 17 00:00:00 2001 From: Jukka Lehtosalo Date: Wed, 4 Sep 2019 13:18:57 +0100 Subject: [PATCH 10/13] Add note about type of missing attribute reference --- docs/source/error_code_list.rst | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/docs/source/error_code_list.rst b/docs/source/error_code_list.rst index 63683a7aee9dd..e8da4859ab73a 100644 --- a/docs/source/error_code_list.rst +++ b/docs/source/error_code_list.rst @@ -39,6 +39,10 @@ target module can be found): # Error: Module 'os' has no attribute 'non_existent' [attr-defined] from os import non_existent +A reference to a missing attribute is given the ``Any`` type. In the +above example, the type of ``non_existent`` will be ``Any``, which can +be important if you silence the error. + Check that attribute exists in each union item [union-attr] ----------------------------------------------------------- From e5fbd85190d8050206faff19976205d046766ab1 Mon Sep 17 00:00:00 2001 From: Jukka Lehtosalo Date: Wed, 4 Sep 2019 14:03:13 +0100 Subject: [PATCH 11/13] Add and update examples --- docs/source/error_code_list.rst | 80 ++++++++++++++++++++++++++++++-- docs/source/error_code_list2.rst | 14 +++++- 2 files changed, 88 insertions(+), 6 deletions(-) diff --git a/docs/source/error_code_list.rst b/docs/source/error_code_list.rst index e8da4859ab73a..e6d940fa00656 100644 --- a/docs/source/error_code_list.rst +++ b/docs/source/error_code_list.rst @@ -134,6 +134,30 @@ When you call an overloaded function, mypy checks that at least one of the signatures of the overload items match the argument types in the call. +Example: + +.. code-block:: python + + from typing import overload, Optional + + @overload + def inc_maybe(x: None) -> None: ... + + @overload + def inc_maybe(x: int) -> int: ... + + def inc_maybe(x: Optional[int]) -> Optional[int]: + if x is None: + return None + else: + return x + 1 + + inc_maybe(None) # OK + inc_maybe(5) # OK + + # Error: No overload variant of "inc_maybe" matches argument type "float" [call-overload] + inc_maybe(1.2) + Check validity of types [valid-type] ------------------------------------ @@ -143,6 +167,31 @@ classes, union types, callable types, type aliases, and literal types. Examples of invalid types include bare integer literals, functions, variables, and modules. +This example incorrectly uses the function ``log`` as a type: + +.. code-block:: python + + from typing import List + + def log(x: object) -> None: + print('log:', repr(x)) + + # Error: Function "t.log" is not valid as a type [valid-type] + def log_all(objs: List[object], f: log) -> None: + for x in objs: + f(x) + +You can use ``Callable`` as the type for callable objects: + +.. code-block:: python + + from typing import List, Callable + + # OK + def log_all(objs: List[object], f: Callable[[object], None]) -> None: + for x in objs: + f(x) + Require annotation if variable type is unclear [var-annotated] -------------------------------------------------------------- @@ -365,15 +414,26 @@ Example: # TypedDict item "x" has type "int") [typeddict-item] p: Point = {'x': 1.2, 'y': 4} - Check that type of target is known [has-type] --------------------------------------------- Mypy sometimes generates an error when it hasn't inferred any type for a variable being referenced. This can happen for references to -variables that have not been initialized yet, and for references -across modules that form an import cycle. When this happens, the -reference gets an implicit ``Any`` type. +variables that are initialized later in the source file, and for +references across modules that form an import cycle. When this +happens, the reference gets an implicit ``Any`` type. + +In this example the definitions of ``x`` and ``y`` are circular: + +.. code-block:: python + + class Problem: + def set_x(self) -> None: + # Error: Cannot determine type of 'y' [has-type] + self.x = self.y + + def set_y(self) -> None: + self.y = self.x To work around this error, you can add an explicit type annotation to the target variable or attribute. Sometimes you can also reorganize @@ -381,6 +441,18 @@ the code so that the definition of the variable is placed earlier than the reference to the variable in a source file. Untangling cyclic imports may also help. +We add an explicit annotation to the ``y`` attribute to work around +the issue: + +.. code-block:: python + + class Problem: + def set_x(self) -> None: + self.x = self.y # OK + + def set_y(self) -> None: + self.y: int = self.x # Added annotation here + Check that import target can be found [import] ---------------------------------------------- diff --git a/docs/source/error_code_list2.rst b/docs/source/error_code_list2.rst index 034b407dbc158..4cfd2f2613ab1 100644 --- a/docs/source/error_code_list2.rst +++ b/docs/source/error_code_list2.rst @@ -100,11 +100,21 @@ Example: # mypy: strict-equality - def is_magic(x: int) -> bool: - # Error: Non-overlapping equality check (left operand type: "int", + def is_magic(x: bytes) -> bool: + # Error: Non-overlapping equality check (left operand type: "bytes", # right operand type: "str") [comparison-overlap] return x == 'magic' +We can fix the error by changing the string literal to a bytes +literal: + +.. code-block:: python + + # mypy: strict-equality + + def is_magic(x: bytes) -> bool: + return x == b'magic' # OK + Check that no untyped functions are called [no-untyped-call] ------------------------------------------------------------ From aa27cbfc2c9b827ed7a0cb811aa644202dff2c97 Mon Sep 17 00:00:00 2001 From: Jukka Lehtosalo Date: Wed, 4 Sep 2019 14:28:03 +0100 Subject: [PATCH 12/13] Add more examples --- docs/source/error_code_list.rst | 37 ++++++++++++++++++++++++++++++++- 1 file changed, 36 insertions(+), 1 deletion(-) diff --git a/docs/source/error_code_list.rst b/docs/source/error_code_list.rst index e6d940fa00656..6948e51f7de50 100644 --- a/docs/source/error_code_list.rst +++ b/docs/source/error_code_list.rst @@ -355,6 +355,13 @@ Mypy checks that operands support a binary or unary operation, such as ``+`` or ``~``. Indexing operations are so common that they have their own error code ``index`` (see below). +Example: + +.. code-block:: python + + # Error: Unsupported operand types for + ("int" and "str") [operator] + 1 + 'x' + Check indexing operations [index] --------------------------------- @@ -362,6 +369,20 @@ Mypy checks that the indexed value in indexing operation such as ``x[y]`` supports indexing, and that the index expression has a valid type. +Example: + +.. code-block:: python + + a = {'x': 1, 'y': 2} + + a['x'] # OK + + # Error: Invalid index type "int" for "Dict[str, int]"; expected type "str" [index] + print(a[1]) + + # Error: Invalid index type "bytes" for "Dict[str, int]"; expected type "str" [index] + a[b'x'] = 4 + Check list items [list-item] ---------------------------- @@ -556,7 +577,21 @@ be a union type, ``Any``, or various other special types. You can also get this error if the target has been imported from a module whose source mypy cannot find, since any such definitions are -treated by mypy as values with ``Any`` types. +treated by mypy as values with ``Any`` types. Example: + +.. code-block:: python + + from typing import NewType + + # The source for "acme" is not available for mypy + from acme import Entity # type: ignore + + # Error: Argument 2 to NewType(...) must be subclassable (got "Any") [valid-newtype] + UserEntity = NewType('UserEntity', Entity) + +To work around the issue, you can either give mypy access to the sources +for ``acme`` or create a stub file for the module. See :ref:`ignore-missing-imports` +for more information. Report syntax errors [syntax] ----------------------------- From 06bae23e3da4eb1cf1d3e3b885914d15c2b9aade Mon Sep 17 00:00:00 2001 From: Jukka Lehtosalo Date: Wed, 4 Sep 2019 14:42:58 +0100 Subject: [PATCH 13/13] Misc updates --- docs/source/common_issues.rst | 8 ++++++++ docs/source/error_codes.rst | 11 ++++++++--- docs/source/index.rst | 4 ++-- 3 files changed, 18 insertions(+), 5 deletions(-) diff --git a/docs/source/common_issues.rst b/docs/source/common_issues.rst index c0c8d7e9d8d13..e4741d34ce1d8 100644 --- a/docs/source/common_issues.rst +++ b/docs/source/common_issues.rst @@ -124,6 +124,14 @@ error: The second line is now fine, since the ignore comment causes the name ``frobnicate`` to get an implicit ``Any`` type. +.. note:: + + You can use the form ``# type: ignore[]`` to only ignore + specific errors on the line. This way you are less likely to + silence unexpected errors that are not safe to ignore, and this + will also document what the purpose of the comment is. See + :ref:`error-codes` for more information. + .. note:: The ``# type: ignore`` comment will only assign the implicit ``Any`` diff --git a/docs/source/error_codes.rst b/docs/source/error_codes.rst index c9033bc3c5480..aafe95d0f289d 100644 --- a/docs/source/error_codes.rst +++ b/docs/source/error_codes.rst @@ -6,14 +6,19 @@ Error codes Mypy can optionally display an error code such as ``[attr-defined]`` after each error message. Error codes serve two purposes: -1. It's possible to silence specific error codes on a line using - ``# type: ignore[code]``. +1. It's possible to silence specific error codes on a line using ``# + type: ignore[code]``. This way you won't accidentally ignore other, + potentially more serious errors. + 2. The error code can be used to find documentation about the error. + The next two topics (:ref:`error-code-list` and + :ref:`error-codes-optional`) document the various error codes + mypy can report. Most error codes are shared between multiple related error messages. Error codes may change in future mypy releases. -The next two topics document the various error codes mypy can report. + Displaying error codes ---------------------- diff --git a/docs/source/index.rst b/docs/source/index.rst index 7080a496df2d4..42c3acd30eeca 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -64,11 +64,11 @@ Mypy is a static type checker for Python 3 and Python 2.7. :maxdepth: 2 :caption: Miscellaneous + common_issues + supported_python_features error_codes error_code_list error_code_list2 - common_issues - supported_python_features python36 additional_features faq