From 0339a334156941cf962f77b69a87a38f64f9b6b3 Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Mon, 3 Aug 2026 12:11:26 -0400 Subject: [PATCH 1/6] Pull in the reference implementation. MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Bartosz Sławecki --- Doc/library/exceptions.rst | 11 ++ Doc/reference/expressions.rst | 90 +++++++++ Doc/whatsnew/3.16.rst | 27 ++- Include/cpython/pyerrors.h | 5 + Include/internal/pycore_magic_number.h | 4 +- Include/internal/pycore_opcode_metadata.h | 9 +- Include/internal/pycore_uop_ids.h | 4 +- Include/opcode_ids.h | 221 +++++++++++----------- Lib/_opcode_metadata.py | 221 +++++++++++----------- Modules/_testinternalcapi/test_cases.c.h | 57 ++++++ Modules/_testinternalcapi/test_targets.h | 11 +- Objects/exceptions.c | 62 +++++- Objects/genobject.c | 59 +++--- Python/bytecodes.c | 17 ++ Python/codegen.c | 121 +++++++++++- Python/executor_cases.c.h | 2 + Python/generated_cases.c.h | 57 ++++++ Python/opcode_targets.h | 11 +- Python/optimizer_cases.c.h | 2 + 19 files changed, 729 insertions(+), 262 deletions(-) diff --git a/Doc/library/exceptions.rst b/Doc/library/exceptions.rst index ecf62fb6391b1b..f327239216b4b0 100644 --- a/Doc/library/exceptions.rst +++ b/Doc/library/exceptions.rst @@ -526,8 +526,19 @@ The following exceptions are the exceptions that are usually raised. Must be raised by :meth:`~object.__anext__` method of an :term:`asynchronous iterator` object to stop the iteration. + .. attribute:: StopAsyncIteration.value + + This is given as an argument when constructing the exception, and + defaults to :const:`None`. This is used for the result of + ``async yield from`` expressions (see :ref:`async-yield-from`). + + .. versionadded: next + .. versionadded:: 3.5 + .. versionchanged:: next + Added the ``value`` attribute. + .. exception:: SyntaxError(message, details) Raised when the parser encounters a syntax error. This may occur in an diff --git a/Doc/reference/expressions.rst b/Doc/reference/expressions.rst index af313f42f9bff6..29e0e6d1340137 100644 --- a/Doc/reference/expressions.rst +++ b/Doc/reference/expressions.rst @@ -1159,6 +1159,10 @@ the yield expression. It can be either set explicitly when raising .. versionchanged:: 3.3 Added ``yield from `` to delegate control flow to a subiterator. +.. versionchanged:: next + ``yield from`` is now allowed to be used in an async generator. + Previously, it would raise a :class:`SyntaxError`. + The parentheses may be omitted when the yield expression is the sole expression on the right hand side of an assignment statement. @@ -1179,6 +1183,10 @@ on the right hand side of an assignment statement. The proposal that expanded on :pep:`492` by adding generator capabilities to coroutine functions. + :pep:`828` - Supporting ``yield from`` in asynchronous generators + The proposal that expanded on :pep:`380` by adding subgenerator delegation + to asynchronous generators. + .. index:: pair: object; generator .. _generator-methods: @@ -1367,6 +1375,88 @@ of a *finalizer* method see the implementation of The expression ``yield from `` is a syntax error when used in an asynchronous generator function. +.. _async-yield-from: + +Asynchronous ``yield from`` +^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +In async generators, the ``yield from`` statement operates solely on +asynchronous constructs rather than synchronous ones. +In particular: + +.. list-table:: + :widths: auto + :header-rows: 1 + + * * ``yield from`` construct + * ``async yield from`` construct + * * :meth:`~object.__iter__` + * :meth:`~object.__aiter__` + * * :meth:`~generator.__next__` + * :meth:`~agen.__anext__` + * * :meth:`~generator.send` + * :meth:`~agen.asend` + * * :class:`StopIteration` + * :class:`StopAsyncIteration` + +To describe the above: + +* The object being delegated to must be asynchronously iterable (that is, it + must implement ``__aiter__`` instead of ``__iter__``). +* When ``anext`` is called on the parent generator (the one that contains + ``async yield from``), ``__anext__`` will be invoked on the subgenerator. + In contrast, a synchronous ``yield from`` would invoke ``__next__`` instead. + (Note that calling ``asend`` with a ``None`` value is equivalent to calling + ``anext()``, and thus applies here.) +* All calls to ``asend``, ``athrow``, and ``aclose`` are delegated to the + subgenerator (the object returned by ``__aiter__`` in this case). This means + that a call to ``parent_generator.asend(x)`` is semantically equivalent to + ``sub_generator.asend(x)``, where ``parent_generator`` is currently executing + an ``async yield from`` on ``sub_generator``. +* The result of the expression is retrieved through + :attr:`StopAsyncIteration.value` instead of :attr:`StopIteration.value`. + +An example of usage for ``yield from`` in async generator: + +.. code-block:: pycon + + >>> import asyncio + >>> async def sleepy_count(number): + ... for num in range(number): + ... await asyncio.sleep(1) + ... result = yield num + ... print(f"Got result: {result}") + ... + >>> async def counter(): + ... final_number = yield from sleepy_count(5) + ... yield final_number + ... + >>> await anext(ag) + 0 + >>> await anext(ag) + Got result: None + 1 + >>> await ag.asend(42) + Got result: 42 + 2 + >>> await ag.athrow(ValueError("Nobody expects the Spanish Inquisition")) + Traceback (most recent call last): + File "/home/python/cpython/Lib/concurrent/futures/_base.py", line 450, in result + return self.__get_result() + ~~~~~~~~~~~~~~~~~^^ + File "/home/python/cpython/Lib/concurrent/futures/_base.py", line 395, in __get_result + raise self._exception + File "", line 1, in + await ag.athrow(ValueError("Nobody expects the Spanish Inquisition")) + File "", line 8, in counter + final_number = async yield from sleepy_count(4) + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + File "", line 3, in sleepy_count + result = yield num + ^^^^^^^^^ + ValueError: Nobody expects the Spanish Inquisition + + .. index:: pair: object; asynchronous-generator .. _asynchronous-generator-methods: diff --git a/Doc/whatsnew/3.16.rst b/Doc/whatsnew/3.16.rst index c607e3c620572f..1c5877c8a99dd1 100644 --- a/Doc/whatsnew/3.16.rst +++ b/Doc/whatsnew/3.16.rst @@ -63,13 +63,38 @@ Summary --- release highlights Brevity is key. -.. PEP-sized items next. +* :pep:`828`: :ref:`'yield from' in async generators ` New features ============ +.. _whatsnew315-async-yield-from: + +:pep:`828`: Supporting ``yield from`` in asynchronous generators +---------------------------------------------------------------- + +Use of the :keyword:`yield from ` construct and the :keyword:`return` +statement with a non-``None`` value is now allowed in an +:term:`asynchronous generator function `. For example, +the following code would previously raise a :class:`SyntaxError`: + +.. code-block:: python + + async def asubgen(): + yield 2 + yield 3 + yield 4 + + async def agenerator(): + yield 1 + yield from asubgen() # Now allowed! + return 5 # Now allowed! + +.. seealso:: :pep:`828` for further details. + +(Contributed by Peter Bierma in :gh:`155126`.) Other language changes diff --git a/Include/cpython/pyerrors.h b/Include/cpython/pyerrors.h index be2e3b641c25cb..8bc91017fa2443 100644 --- a/Include/cpython/pyerrors.h +++ b/Include/cpython/pyerrors.h @@ -73,6 +73,11 @@ typedef struct { PyObject *value; } PyStopIterationObject; +typedef struct { + PyException_HEAD + PyObject *value; +} PyStopAsyncIterationObject; + typedef struct { PyException_HEAD PyObject *name; diff --git a/Include/internal/pycore_magic_number.h b/Include/internal/pycore_magic_number.h index e0c5cfc02bbc35..99b8d9e342b92b 100644 --- a/Include/internal/pycore_magic_number.h +++ b/Include/internal/pycore_magic_number.h @@ -302,7 +302,7 @@ Known values: Python 3.16a1 3702 (Replace DELETE_NAME with PUSH_NULL; STORE_NAME) Python 3.16a1 3703 (Replace DELETE_GLOBAL with PUSH_NULL; STORE_GLOBAL) Python 3.16a1 3704 (Replace DELETE_ATTR with PUSH_NULL; STORE_ATTR) - + Python 3.16a1 3705 (PEP 828: yield from for asyncgens) Python 3.17 will start with 3750 @@ -312,7 +312,7 @@ Known values: */ -#define PYC_MAGIC_NUMBER 3704 +#define PYC_MAGIC_NUMBER 3705 /* This is equivalent to converting PYC_MAGIC_NUMBER to 2 bytes (little-endian) and then appending b'\r\n'. */ #define PYC_MAGIC_NUMBER_TOKEN \ diff --git a/Include/internal/pycore_opcode_metadata.h b/Include/internal/pycore_opcode_metadata.h index 457e5c5bf20d2b..0b574257785154 100644 --- a/Include/internal/pycore_opcode_metadata.h +++ b/Include/internal/pycore_opcode_metadata.h @@ -156,6 +156,8 @@ int _PyOpcode_num_popped(int opcode, int oparg) { return 2; case CHECK_EXC_MATCH: return 2; + case CLEANUP_ASYNC_THROW: + return 2; case CLEANUP_THROW: return 4; case COMPARE_OP: @@ -651,6 +653,8 @@ int _PyOpcode_num_pushed(int opcode, int oparg) { return 2; case CHECK_EXC_MATCH: return 2; + case CLEANUP_ASYNC_THROW: + return 1; case CLEANUP_THROW: return 3; case COMPARE_OP: @@ -1159,6 +1163,7 @@ const struct opcode_metadata _PyOpcode_opcode_metadata[267] = { [CALL_TYPE_1] = { true, INSTR_FMT_IBC00, HAS_ARG_FLAG | HAS_EXIT_FLAG | HAS_ESCAPES_FLAG }, [CHECK_EG_MATCH] = { true, INSTR_FMT_IX, HAS_ERROR_FLAG | HAS_ESCAPES_FLAG }, [CHECK_EXC_MATCH] = { true, INSTR_FMT_IX, HAS_ERROR_FLAG | HAS_ERROR_NO_POP_FLAG | HAS_ESCAPES_FLAG }, + [CLEANUP_ASYNC_THROW] = { true, INSTR_FMT_IX, HAS_ERROR_FLAG | HAS_ERROR_NO_POP_FLAG | HAS_ESCAPES_FLAG }, [CLEANUP_THROW] = { true, INSTR_FMT_IX, HAS_ERROR_FLAG | HAS_ERROR_NO_POP_FLAG | HAS_ESCAPES_FLAG }, [COMPARE_OP] = { true, INSTR_FMT_IBC, HAS_ARG_FLAG | HAS_ERROR_FLAG | HAS_ESCAPES_FLAG }, [COMPARE_OP_FLOAT] = { true, INSTR_FMT_IBC, HAS_ARG_FLAG | HAS_EXIT_FLAG }, @@ -1611,6 +1616,7 @@ const char *_PyOpcode_OpName[267] = { [CALL_TYPE_1] = "CALL_TYPE_1", [CHECK_EG_MATCH] = "CHECK_EG_MATCH", [CHECK_EXC_MATCH] = "CHECK_EXC_MATCH", + [CLEANUP_ASYNC_THROW] = "CLEANUP_ASYNC_THROW", [CLEANUP_THROW] = "CLEANUP_THROW", [COMPARE_OP] = "COMPARE_OP", [COMPARE_OP_FLOAT] = "COMPARE_OP_FLOAT", @@ -1827,7 +1833,6 @@ const uint8_t _PyOpcode_Caches[256] = { PyAPI_DATA(const uint8_t) _PyOpcode_Deopt[256]; #ifdef NEED_OPCODE_METADATA const uint8_t _PyOpcode_Deopt[256] = { - [117] = 117, [118] = 118, [119] = 119, [120] = 120, @@ -1911,6 +1916,7 @@ const uint8_t _PyOpcode_Deopt[256] = { [CALL_TYPE_1] = CALL, [CHECK_EG_MATCH] = CHECK_EG_MATCH, [CHECK_EXC_MATCH] = CHECK_EXC_MATCH, + [CLEANUP_ASYNC_THROW] = CLEANUP_ASYNC_THROW, [CLEANUP_THROW] = CLEANUP_THROW, [COMPARE_OP] = COMPARE_OP, [COMPARE_OP_FLOAT] = COMPARE_OP, @@ -2088,7 +2094,6 @@ const uint8_t _PyOpcode_Deopt[256] = { #endif // NEED_OPCODE_METADATA #define EXTRA_CASES \ - case 117: \ case 118: \ case 119: \ case 120: \ diff --git a/Include/internal/pycore_uop_ids.h b/Include/internal/pycore_uop_ids.h index 97f934727a85a3..eb5bf45e5e4fd2 100644 --- a/Include/internal/pycore_uop_ids.h +++ b/Include/internal/pycore_uop_ids.h @@ -376,6 +376,7 @@ enum { #define _BUILD_TUPLE BUILD_TUPLE #define _CHECK_EG_MATCH CHECK_EG_MATCH #define _CHECK_EXC_MATCH CHECK_EXC_MATCH +#define _CLEANUP_ASYNC_THROW CLEANUP_ASYNC_THROW #define _CONVERT_VALUE CONVERT_VALUE #define _COPY_FREE_VARS COPY_FREE_VARS #define _DELETE_DEREF DELETE_DEREF @@ -619,6 +620,7 @@ enum { _CHECK_VALIDITY_r11, _CHECK_VALIDITY_r22, _CHECK_VALIDITY_r33, + _CLEANUP_ASYNC_THROW_r21, _COLD_DYNAMIC_EXIT_r00, _COLD_EXIT_r00, _COMPARE_OP_r21, @@ -1434,7 +1436,7 @@ enum { _WITH_EXCEPT_START_r33, _YIELD_VALUE_r11, }; -#define MAX_UOP_REGS_ID 1645 +#define MAX_UOP_REGS_ID 1646 #ifdef __cplusplus } diff --git a/Include/opcode_ids.h b/Include/opcode_ids.h index 11342ae451b9f6..959586326035e4 100644 --- a/Include/opcode_ids.h +++ b/Include/opcode_ids.h @@ -17,116 +17,117 @@ extern "C" { #define CALL_FUNCTION_EX 4 #define CHECK_EG_MATCH 5 #define CHECK_EXC_MATCH 6 -#define CLEANUP_THROW 7 -#define DELETE_SUBSCR 8 -#define END_FOR 9 -#define END_SEND 10 -#define EXIT_INIT_CHECK 11 -#define FORMAT_SIMPLE 12 -#define FORMAT_WITH_SPEC 13 -#define GET_AITER 14 -#define GET_ANEXT 15 -#define GET_LEN 16 +#define CLEANUP_ASYNC_THROW 7 +#define CLEANUP_THROW 8 +#define DELETE_SUBSCR 9 +#define END_FOR 10 +#define END_SEND 11 +#define EXIT_INIT_CHECK 12 +#define FORMAT_SIMPLE 13 +#define FORMAT_WITH_SPEC 14 +#define GET_AITER 15 +#define GET_ANEXT 16 #define RESERVED 17 -#define INTERPRETER_EXIT 18 -#define LOAD_BUILD_CLASS 19 -#define LOAD_LOCALS 20 -#define MAKE_FUNCTION 21 -#define MATCH_KEYS 22 -#define MATCH_MAPPING 23 -#define MATCH_SEQUENCE 24 -#define NOP 25 -#define NOT_TAKEN 26 -#define POP_EXCEPT 27 -#define POP_ITER 28 -#define POP_TOP 29 -#define PUSH_EXC_INFO 30 -#define PUSH_NULL 31 -#define RETURN_GENERATOR 32 -#define RETURN_VALUE 33 -#define SETUP_ANNOTATIONS 34 -#define STORE_SLICE 35 -#define STORE_SUBSCR 36 -#define TO_BOOL 37 -#define UNARY_INVERT 38 -#define UNARY_NEGATIVE 39 -#define UNARY_NOT 40 -#define WITH_EXCEPT_START 41 -#define BINARY_OP 42 -#define BUILD_INTERPOLATION 43 -#define BUILD_LIST 44 -#define BUILD_MAP 45 -#define BUILD_SET 46 -#define BUILD_SLICE 47 -#define BUILD_STRING 48 -#define BUILD_TUPLE 49 -#define CALL 50 -#define CALL_INTRINSIC_1 51 -#define CALL_INTRINSIC_2 52 -#define CALL_KW 53 -#define COMPARE_OP 54 -#define CONTAINS_OP 55 -#define CONVERT_VALUE 56 -#define COPY 57 -#define COPY_FREE_VARS 58 -#define DELETE_DEREF 59 -#define DELETE_FAST 60 -#define DICT_MERGE 61 -#define DICT_UPDATE 62 -#define END_ASYNC_FOR 63 -#define EXTENDED_ARG 64 -#define FOR_ITER 65 -#define GET_AWAITABLE 66 -#define GET_ITER 67 -#define IMPORT_FROM 68 -#define IMPORT_NAME 69 -#define IS_OP 70 -#define JUMP_BACKWARD 71 -#define JUMP_BACKWARD_NO_INTERRUPT 72 -#define JUMP_FORWARD 73 -#define LIST_APPEND 74 -#define LIST_EXTEND 75 -#define LOAD_ATTR 76 -#define LOAD_COMMON_CONSTANT 77 -#define LOAD_CONST 78 -#define LOAD_DEREF 79 -#define LOAD_FAST 80 -#define LOAD_FAST_AND_CLEAR 81 -#define LOAD_FAST_BORROW 82 -#define LOAD_FAST_BORROW_LOAD_FAST_BORROW 83 -#define LOAD_FAST_CHECK 84 -#define LOAD_FAST_LOAD_FAST 85 -#define LOAD_FROM_DICT_OR_DEREF 86 -#define LOAD_FROM_DICT_OR_GLOBALS 87 -#define LOAD_GLOBAL 88 -#define LOAD_NAME 89 -#define LOAD_SMALL_INT 90 -#define LOAD_SPECIAL 91 -#define LOAD_SUPER_ATTR 92 -#define MAKE_CELL 93 -#define MAP_ADD 94 -#define MATCH_CLASS 95 -#define POP_JUMP_IF_FALSE 96 -#define POP_JUMP_IF_NONE 97 -#define POP_JUMP_IF_NOT_NONE 98 -#define POP_JUMP_IF_TRUE 99 -#define RAISE_VARARGS 100 -#define RERAISE 101 -#define SEND 102 -#define SET_ADD 103 -#define SET_FUNCTION_ATTRIBUTE 104 -#define SET_UPDATE 105 -#define STORE_ATTR 106 -#define STORE_DEREF 107 -#define STORE_FAST 108 -#define STORE_FAST_LOAD_FAST 109 -#define STORE_FAST_STORE_FAST 110 -#define STORE_GLOBAL 111 -#define STORE_NAME 112 -#define SWAP 113 -#define UNPACK_EX 114 -#define UNPACK_SEQUENCE 115 -#define YIELD_VALUE 116 +#define GET_LEN 18 +#define INTERPRETER_EXIT 19 +#define LOAD_BUILD_CLASS 20 +#define LOAD_LOCALS 21 +#define MAKE_FUNCTION 22 +#define MATCH_KEYS 23 +#define MATCH_MAPPING 24 +#define MATCH_SEQUENCE 25 +#define NOP 26 +#define NOT_TAKEN 27 +#define POP_EXCEPT 28 +#define POP_ITER 29 +#define POP_TOP 30 +#define PUSH_EXC_INFO 31 +#define PUSH_NULL 32 +#define RETURN_GENERATOR 33 +#define RETURN_VALUE 34 +#define SETUP_ANNOTATIONS 35 +#define STORE_SLICE 36 +#define STORE_SUBSCR 37 +#define TO_BOOL 38 +#define UNARY_INVERT 39 +#define UNARY_NEGATIVE 40 +#define UNARY_NOT 41 +#define WITH_EXCEPT_START 42 +#define BINARY_OP 43 +#define BUILD_INTERPOLATION 44 +#define BUILD_LIST 45 +#define BUILD_MAP 46 +#define BUILD_SET 47 +#define BUILD_SLICE 48 +#define BUILD_STRING 49 +#define BUILD_TUPLE 50 +#define CALL 51 +#define CALL_INTRINSIC_1 52 +#define CALL_INTRINSIC_2 53 +#define CALL_KW 54 +#define COMPARE_OP 55 +#define CONTAINS_OP 56 +#define CONVERT_VALUE 57 +#define COPY 58 +#define COPY_FREE_VARS 59 +#define DELETE_DEREF 60 +#define DELETE_FAST 61 +#define DICT_MERGE 62 +#define DICT_UPDATE 63 +#define END_ASYNC_FOR 64 +#define EXTENDED_ARG 65 +#define FOR_ITER 66 +#define GET_AWAITABLE 67 +#define GET_ITER 68 +#define IMPORT_FROM 69 +#define IMPORT_NAME 70 +#define IS_OP 71 +#define JUMP_BACKWARD 72 +#define JUMP_BACKWARD_NO_INTERRUPT 73 +#define JUMP_FORWARD 74 +#define LIST_APPEND 75 +#define LIST_EXTEND 76 +#define LOAD_ATTR 77 +#define LOAD_COMMON_CONSTANT 78 +#define LOAD_CONST 79 +#define LOAD_DEREF 80 +#define LOAD_FAST 81 +#define LOAD_FAST_AND_CLEAR 82 +#define LOAD_FAST_BORROW 83 +#define LOAD_FAST_BORROW_LOAD_FAST_BORROW 84 +#define LOAD_FAST_CHECK 85 +#define LOAD_FAST_LOAD_FAST 86 +#define LOAD_FROM_DICT_OR_DEREF 87 +#define LOAD_FROM_DICT_OR_GLOBALS 88 +#define LOAD_GLOBAL 89 +#define LOAD_NAME 90 +#define LOAD_SMALL_INT 91 +#define LOAD_SPECIAL 92 +#define LOAD_SUPER_ATTR 93 +#define MAKE_CELL 94 +#define MAP_ADD 95 +#define MATCH_CLASS 96 +#define POP_JUMP_IF_FALSE 97 +#define POP_JUMP_IF_NONE 98 +#define POP_JUMP_IF_NOT_NONE 99 +#define POP_JUMP_IF_TRUE 100 +#define RAISE_VARARGS 101 +#define RERAISE 102 +#define SEND 103 +#define SET_ADD 104 +#define SET_FUNCTION_ATTRIBUTE 105 +#define SET_UPDATE 106 +#define STORE_ATTR 107 +#define STORE_DEREF 108 +#define STORE_FAST 109 +#define STORE_FAST_LOAD_FAST 110 +#define STORE_FAST_STORE_FAST 111 +#define STORE_GLOBAL 112 +#define STORE_NAME 113 +#define SWAP 114 +#define UNPACK_EX 115 +#define UNPACK_SEQUENCE 116 +#define YIELD_VALUE 117 #define RESUME 128 #define BINARY_OP_ADD_FLOAT 129 #define BINARY_OP_ADD_INT 130 @@ -253,7 +254,7 @@ extern "C" { #define SETUP_WITH 265 #define STORE_FAST_MAYBE_NULL 266 -#define HAVE_ARGUMENT 41 +#define HAVE_ARGUMENT 42 #define MIN_SPECIALIZED_OPCODE 129 #define MIN_INSTRUMENTED_OPCODE 233 diff --git a/Lib/_opcode_metadata.py b/Lib/_opcode_metadata.py index df92eae151d248..7a36f16968959a 100644 --- a/Lib/_opcode_metadata.py +++ b/Lib/_opcode_metadata.py @@ -238,115 +238,116 @@ CALL_FUNCTION_EX=4, CHECK_EG_MATCH=5, CHECK_EXC_MATCH=6, - CLEANUP_THROW=7, - DELETE_SUBSCR=8, - END_FOR=9, - END_SEND=10, - EXIT_INIT_CHECK=11, - FORMAT_SIMPLE=12, - FORMAT_WITH_SPEC=13, - GET_AITER=14, - GET_ANEXT=15, - GET_LEN=16, - INTERPRETER_EXIT=18, - LOAD_BUILD_CLASS=19, - LOAD_LOCALS=20, - MAKE_FUNCTION=21, - MATCH_KEYS=22, - MATCH_MAPPING=23, - MATCH_SEQUENCE=24, - NOP=25, - NOT_TAKEN=26, - POP_EXCEPT=27, - POP_ITER=28, - POP_TOP=29, - PUSH_EXC_INFO=30, - PUSH_NULL=31, - RETURN_GENERATOR=32, - RETURN_VALUE=33, - SETUP_ANNOTATIONS=34, - STORE_SLICE=35, - STORE_SUBSCR=36, - TO_BOOL=37, - UNARY_INVERT=38, - UNARY_NEGATIVE=39, - UNARY_NOT=40, - WITH_EXCEPT_START=41, - BINARY_OP=42, - BUILD_INTERPOLATION=43, - BUILD_LIST=44, - BUILD_MAP=45, - BUILD_SET=46, - BUILD_SLICE=47, - BUILD_STRING=48, - BUILD_TUPLE=49, - CALL=50, - CALL_INTRINSIC_1=51, - CALL_INTRINSIC_2=52, - CALL_KW=53, - COMPARE_OP=54, - CONTAINS_OP=55, - CONVERT_VALUE=56, - COPY=57, - COPY_FREE_VARS=58, - DELETE_DEREF=59, - DELETE_FAST=60, - DICT_MERGE=61, - DICT_UPDATE=62, - END_ASYNC_FOR=63, - EXTENDED_ARG=64, - FOR_ITER=65, - GET_AWAITABLE=66, - GET_ITER=67, - IMPORT_FROM=68, - IMPORT_NAME=69, - IS_OP=70, - JUMP_BACKWARD=71, - JUMP_BACKWARD_NO_INTERRUPT=72, - JUMP_FORWARD=73, - LIST_APPEND=74, - LIST_EXTEND=75, - LOAD_ATTR=76, - LOAD_COMMON_CONSTANT=77, - LOAD_CONST=78, - LOAD_DEREF=79, - LOAD_FAST=80, - LOAD_FAST_AND_CLEAR=81, - LOAD_FAST_BORROW=82, - LOAD_FAST_BORROW_LOAD_FAST_BORROW=83, - LOAD_FAST_CHECK=84, - LOAD_FAST_LOAD_FAST=85, - LOAD_FROM_DICT_OR_DEREF=86, - LOAD_FROM_DICT_OR_GLOBALS=87, - LOAD_GLOBAL=88, - LOAD_NAME=89, - LOAD_SMALL_INT=90, - LOAD_SPECIAL=91, - LOAD_SUPER_ATTR=92, - MAKE_CELL=93, - MAP_ADD=94, - MATCH_CLASS=95, - POP_JUMP_IF_FALSE=96, - POP_JUMP_IF_NONE=97, - POP_JUMP_IF_NOT_NONE=98, - POP_JUMP_IF_TRUE=99, - RAISE_VARARGS=100, - RERAISE=101, - SEND=102, - SET_ADD=103, - SET_FUNCTION_ATTRIBUTE=104, - SET_UPDATE=105, - STORE_ATTR=106, - STORE_DEREF=107, - STORE_FAST=108, - STORE_FAST_LOAD_FAST=109, - STORE_FAST_STORE_FAST=110, - STORE_GLOBAL=111, - STORE_NAME=112, - SWAP=113, - UNPACK_EX=114, - UNPACK_SEQUENCE=115, - YIELD_VALUE=116, + CLEANUP_ASYNC_THROW=7, + CLEANUP_THROW=8, + DELETE_SUBSCR=9, + END_FOR=10, + END_SEND=11, + EXIT_INIT_CHECK=12, + FORMAT_SIMPLE=13, + FORMAT_WITH_SPEC=14, + GET_AITER=15, + GET_ANEXT=16, + GET_LEN=18, + INTERPRETER_EXIT=19, + LOAD_BUILD_CLASS=20, + LOAD_LOCALS=21, + MAKE_FUNCTION=22, + MATCH_KEYS=23, + MATCH_MAPPING=24, + MATCH_SEQUENCE=25, + NOP=26, + NOT_TAKEN=27, + POP_EXCEPT=28, + POP_ITER=29, + POP_TOP=30, + PUSH_EXC_INFO=31, + PUSH_NULL=32, + RETURN_GENERATOR=33, + RETURN_VALUE=34, + SETUP_ANNOTATIONS=35, + STORE_SLICE=36, + STORE_SUBSCR=37, + TO_BOOL=38, + UNARY_INVERT=39, + UNARY_NEGATIVE=40, + UNARY_NOT=41, + WITH_EXCEPT_START=42, + BINARY_OP=43, + BUILD_INTERPOLATION=44, + BUILD_LIST=45, + BUILD_MAP=46, + BUILD_SET=47, + BUILD_SLICE=48, + BUILD_STRING=49, + BUILD_TUPLE=50, + CALL=51, + CALL_INTRINSIC_1=52, + CALL_INTRINSIC_2=53, + CALL_KW=54, + COMPARE_OP=55, + CONTAINS_OP=56, + CONVERT_VALUE=57, + COPY=58, + COPY_FREE_VARS=59, + DELETE_DEREF=60, + DELETE_FAST=61, + DICT_MERGE=62, + DICT_UPDATE=63, + END_ASYNC_FOR=64, + EXTENDED_ARG=65, + FOR_ITER=66, + GET_AWAITABLE=67, + GET_ITER=68, + IMPORT_FROM=69, + IMPORT_NAME=70, + IS_OP=71, + JUMP_BACKWARD=72, + JUMP_BACKWARD_NO_INTERRUPT=73, + JUMP_FORWARD=74, + LIST_APPEND=75, + LIST_EXTEND=76, + LOAD_ATTR=77, + LOAD_COMMON_CONSTANT=78, + LOAD_CONST=79, + LOAD_DEREF=80, + LOAD_FAST=81, + LOAD_FAST_AND_CLEAR=82, + LOAD_FAST_BORROW=83, + LOAD_FAST_BORROW_LOAD_FAST_BORROW=84, + LOAD_FAST_CHECK=85, + LOAD_FAST_LOAD_FAST=86, + LOAD_FROM_DICT_OR_DEREF=87, + LOAD_FROM_DICT_OR_GLOBALS=88, + LOAD_GLOBAL=89, + LOAD_NAME=90, + LOAD_SMALL_INT=91, + LOAD_SPECIAL=92, + LOAD_SUPER_ATTR=93, + MAKE_CELL=94, + MAP_ADD=95, + MATCH_CLASS=96, + POP_JUMP_IF_FALSE=97, + POP_JUMP_IF_NONE=98, + POP_JUMP_IF_NOT_NONE=99, + POP_JUMP_IF_TRUE=100, + RAISE_VARARGS=101, + RERAISE=102, + SEND=103, + SET_ADD=104, + SET_FUNCTION_ATTRIBUTE=105, + SET_UPDATE=106, + STORE_ATTR=107, + STORE_DEREF=108, + STORE_FAST=109, + STORE_FAST_LOAD_FAST=110, + STORE_FAST_STORE_FAST=111, + STORE_GLOBAL=112, + STORE_NAME=113, + SWAP=114, + UNPACK_EX=115, + UNPACK_SEQUENCE=116, + YIELD_VALUE=117, INSTRUMENTED_END_FOR=233, INSTRUMENTED_POP_ITER=234, INSTRUMENTED_END_SEND=235, @@ -380,5 +381,5 @@ STORE_FAST_MAYBE_NULL=266, ) -HAVE_ARGUMENT = 41 +HAVE_ARGUMENT = 42 MIN_INSTRUMENTED_OPCODE = 233 diff --git a/Modules/_testinternalcapi/test_cases.c.h b/Modules/_testinternalcapi/test_cases.c.h index a17648a33d4fe4..9a43b954a39136 100644 --- a/Modules/_testinternalcapi/test_cases.c.h +++ b/Modules/_testinternalcapi/test_cases.c.h @@ -5152,6 +5152,63 @@ DISPATCH(); } + TARGET(CLEANUP_ASYNC_THROW) { + #if _Py_TAIL_CALL_INTERP + int opcode = CLEANUP_ASYNC_THROW; + (void)(opcode); + #endif + _Py_CODEUNIT* const this_instr = next_instr; + (void)this_instr; + frame->instr_ptr = next_instr; + next_instr += 1; + INSTRUCTION_STATS(CLEANUP_ASYNC_THROW); + _PyStackRef iter; + _PyStackRef exc_value_st; + _PyStackRef value; + exc_value_st = stack_pointer[-1]; + iter = stack_pointer[-2]; + PyObject *exc_value = PyStackRef_AsPyObjectBorrow(exc_value_st); + assert(exc_value != NULL); + assert(PyExceptionInstance_Check(exc_value)); + _PyFrame_SetStackPointer(frame, stack_pointer); + _PyFrame_StackPointerValidate(frame); + int matches = PyErr_GivenExceptionMatches(exc_value, PyExc_StopAsyncIteration); + _PyFrame_StackPointerInvalidate(frame); + if (matches) { + value = PyStackRef_FromPyObjectNew(((PyStopAsyncIterationObject *)exc_value)->value); + assert(stack_pointer == _PyFrame_GetStackPointer(frame)); + _PyFrame_StackPointerValidate(frame); + _PyStackRef tmp = iter; + iter = value; + stack_pointer[-2] = iter; + PyStackRef_CLOSE(tmp); + _PyFrame_StackPointerInvalidate(frame); + assert(stack_pointer == _PyFrame_GetStackPointer(frame)); + _PyFrame_StackPointerValidate(frame); + tmp = exc_value_st; + exc_value_st = PyStackRef_NULL; + stack_pointer[-1] = exc_value_st; + PyStackRef_CLOSE(tmp); + _PyFrame_StackPointerInvalidate(frame); + stack_pointer += -1; + ASSERT_WITHIN_STACK_BOUNDS(__FILE__, __LINE__); + } + else { + assert(stack_pointer == _PyFrame_GetStackPointer(frame)); + _PyFrame_StackPointerValidate(frame); + _PyErr_SetRaisedException(tstate, Py_NewRef(exc_value)); + _PyFrame_StackPointerInvalidate(frame); + assert(stack_pointer == _PyFrame_GetStackPointer(frame)); + _PyFrame_StackPointerValidate(frame); + monitor_reraise(tstate, frame, this_instr); + _PyFrame_StackPointerInvalidate(frame); + assert(stack_pointer == _PyFrame_GetStackPointer(frame)); + _PyFrame_StackPointerValidate(frame); + JUMP_TO_LABEL(exception_unwind); + } + DISPATCH(); + } + TARGET(CLEANUP_THROW) { #if _Py_TAIL_CALL_INTERP int opcode = CLEANUP_THROW; diff --git a/Modules/_testinternalcapi/test_targets.h b/Modules/_testinternalcapi/test_targets.h index 91b424773224e4..5c506eb250afd9 100644 --- a/Modules/_testinternalcapi/test_targets.h +++ b/Modules/_testinternalcapi/test_targets.h @@ -7,6 +7,7 @@ static void *opcode_targets_table[256] = { &&TARGET_CALL_FUNCTION_EX, &&TARGET_CHECK_EG_MATCH, &&TARGET_CHECK_EXC_MATCH, + &&TARGET_CLEANUP_ASYNC_THROW, &&TARGET_CLEANUP_THROW, &&TARGET_DELETE_SUBSCR, &&TARGET_END_FOR, @@ -16,8 +17,8 @@ static void *opcode_targets_table[256] = { &&TARGET_FORMAT_WITH_SPEC, &&TARGET_GET_AITER, &&TARGET_GET_ANEXT, - &&TARGET_GET_LEN, &&TARGET_RESERVED, + &&TARGET_GET_LEN, &&TARGET_INTERPRETER_EXIT, &&TARGET_LOAD_BUILD_CLASS, &&TARGET_LOAD_LOCALS, @@ -127,7 +128,6 @@ static void *opcode_targets_table[256] = { &&_unknown_opcode, &&_unknown_opcode, &&_unknown_opcode, - &&_unknown_opcode, &&TARGET_RESUME, &&TARGET_BINARY_OP_ADD_FLOAT, &&TARGET_BINARY_OP_ADD_INT, @@ -376,7 +376,7 @@ static void *opcode_tracing_targets_table[256] = { &&TARGET_TRACE_RECORD, &&TARGET_TRACE_RECORD, &&TARGET_TRACE_RECORD, - &&_unknown_opcode, + &&TARGET_TRACE_RECORD, &&_unknown_opcode, &&_unknown_opcode, &&_unknown_opcode, @@ -590,6 +590,7 @@ static PyObject *Py_PRESERVE_NONE_CC _TAIL_CALL_CALL_TUPLE_1(TAIL_CALL_PARAMS); static PyObject *Py_PRESERVE_NONE_CC _TAIL_CALL_CALL_TYPE_1(TAIL_CALL_PARAMS); static PyObject *Py_PRESERVE_NONE_CC _TAIL_CALL_CHECK_EG_MATCH(TAIL_CALL_PARAMS); static PyObject *Py_PRESERVE_NONE_CC _TAIL_CALL_CHECK_EXC_MATCH(TAIL_CALL_PARAMS); +static PyObject *Py_PRESERVE_NONE_CC _TAIL_CALL_CLEANUP_ASYNC_THROW(TAIL_CALL_PARAMS); static PyObject *Py_PRESERVE_NONE_CC _TAIL_CALL_CLEANUP_THROW(TAIL_CALL_PARAMS); static PyObject *Py_PRESERVE_NONE_CC _TAIL_CALL_COMPARE_OP(TAIL_CALL_PARAMS); static PyObject *Py_PRESERVE_NONE_CC _TAIL_CALL_COMPARE_OP_FLOAT(TAIL_CALL_PARAMS); @@ -833,6 +834,7 @@ static py_tail_call_funcptr instruction_funcptr_handler_table[256] = { [CALL_TYPE_1] = _TAIL_CALL_CALL_TYPE_1, [CHECK_EG_MATCH] = _TAIL_CALL_CHECK_EG_MATCH, [CHECK_EXC_MATCH] = _TAIL_CALL_CHECK_EXC_MATCH, + [CLEANUP_ASYNC_THROW] = _TAIL_CALL_CLEANUP_ASYNC_THROW, [CLEANUP_THROW] = _TAIL_CALL_CLEANUP_THROW, [COMPARE_OP] = _TAIL_CALL_COMPARE_OP, [COMPARE_OP_FLOAT] = _TAIL_CALL_COMPARE_OP_FLOAT, @@ -1005,7 +1007,6 @@ static py_tail_call_funcptr instruction_funcptr_handler_table[256] = { [UNPACK_SEQUENCE_TWO_TUPLE] = _TAIL_CALL_UNPACK_SEQUENCE_TWO_TUPLE, [WITH_EXCEPT_START] = _TAIL_CALL_WITH_EXCEPT_START, [YIELD_VALUE] = _TAIL_CALL_YIELD_VALUE, - [117] = _TAIL_CALL_UNKNOWN_OPCODE, [118] = _TAIL_CALL_UNKNOWN_OPCODE, [119] = _TAIL_CALL_UNKNOWN_OPCODE, [120] = _TAIL_CALL_UNKNOWN_OPCODE, @@ -1091,6 +1092,7 @@ static py_tail_call_funcptr instruction_funcptr_tracing_table[256] = { [CALL_TYPE_1] = _TAIL_CALL_TRACE_RECORD, [CHECK_EG_MATCH] = _TAIL_CALL_TRACE_RECORD, [CHECK_EXC_MATCH] = _TAIL_CALL_TRACE_RECORD, + [CLEANUP_ASYNC_THROW] = _TAIL_CALL_TRACE_RECORD, [CLEANUP_THROW] = _TAIL_CALL_TRACE_RECORD, [COMPARE_OP] = _TAIL_CALL_TRACE_RECORD, [COMPARE_OP_FLOAT] = _TAIL_CALL_TRACE_RECORD, @@ -1263,7 +1265,6 @@ static py_tail_call_funcptr instruction_funcptr_tracing_table[256] = { [UNPACK_SEQUENCE_TWO_TUPLE] = _TAIL_CALL_TRACE_RECORD, [WITH_EXCEPT_START] = _TAIL_CALL_TRACE_RECORD, [YIELD_VALUE] = _TAIL_CALL_TRACE_RECORD, - [117] = _TAIL_CALL_UNKNOWN_OPCODE, [118] = _TAIL_CALL_UNKNOWN_OPCODE, [119] = _TAIL_CALL_UNKNOWN_OPCODE, [120] = _TAIL_CALL_UNKNOWN_OPCODE, diff --git a/Objects/exceptions.c b/Objects/exceptions.c index fb546ad2673576..317299e4cde6a9 100644 --- a/Objects/exceptions.c +++ b/Objects/exceptions.c @@ -723,8 +723,66 @@ SimpleExtendsException(PyExc_Exception, TypeError, /* * StopAsyncIteration extends Exception */ -SimpleExtendsException(PyExc_Exception, StopAsyncIteration, - "Signal the end from iterator.__anext__()."); +static PyMemberDef StopAsyncIteration_members[] = { + {"value", _Py_T_OBJECT, offsetof(PyStopAsyncIterationObject, value), 0, + PyDoc_STR("async generator return value")}, + {NULL} /* Sentinel */ +}; + +static inline PyStopAsyncIterationObject * +PyStopAsyncIterationObject_CAST(PyObject *self) +{ + assert(self != NULL); + assert(PyObject_TypeCheck(self, (PyTypeObject *)PyExc_StopAsyncIteration)); + return (PyStopAsyncIterationObject *)self; +} + +static int +StopAsyncIteration_init(PyObject *op, PyObject *args, PyObject *kwds) +{ + Py_ssize_t size = PyTuple_GET_SIZE(args); + + if (BaseException_init(op, args, kwds) < 0) { + return -1; + } + PyStopAsyncIterationObject *self = PyStopAsyncIterationObject_CAST(op); + Py_CLEAR(self->value); + if (size > 0) { + self->value = Py_NewRef(PyTuple_GET_ITEM(args, 0)); + } + else { + self->value = Py_None; + }; + return 0; +} + +static int +StopAsyncIteration_clear(PyObject *op) +{ + PyStopAsyncIterationObject *self = PyStopAsyncIterationObject_CAST(op); + Py_CLEAR(self->value); + return BaseException_clear(op); +} + +static void +StopAsyncIteration_dealloc(PyObject *self) +{ + _PyObject_GC_UNTRACK(self); + (void)StopAsyncIteration_clear(self); + Py_TYPE(self)->tp_free(self); +} + +static int +StopAsyncIteration_traverse(PyObject *op, visitproc visit, void *arg) +{ + PyStopAsyncIterationObject *self = PyStopAsyncIterationObject_CAST(op); + Py_VISIT(self->value); + return BaseException_traverse(op, visit, arg); +} + +ComplexExtendsException(PyExc_Exception, StopAsyncIteration, StopAsyncIteration, + 0, 0, StopAsyncIteration_members, 0, 0, 0, + "Signal the end from iterator.__anext__()."); /* diff --git a/Objects/genobject.c b/Objects/genobject.c index 3cdc06733363d3..18def6b62b3e02 100644 --- a/Objects/genobject.c +++ b/Objects/genobject.c @@ -168,7 +168,6 @@ gen_clear_frame(PyGenObject *gen) { assert(FT_ATOMIC_LOAD_INT8_RELAXED(gen->gi_frame_state) == FRAME_CLEARED); _PyInterpreterFrame *frame = &gen->gi_iframe; - _PyThreadState_UpdateLastProfiledFrame(_PyThreadState_GET(), frame, frame->previous); frame->previous = NULL; _PyFrame_ClearExceptCode(frame); _PyErr_ClearExcState(&gen->gi_exc_state); @@ -308,8 +307,7 @@ gen_send_ex2(PyGenObject *gen, PyObject *arg, PyObject **presult, int exc) /* If the generator just returned (as opposed to yielding), signal * that the generator is exhausted. */ if (result) { - assert(result == Py_None || !PyAsyncGen_CheckExact(gen)); - if (result == Py_None && !PyAsyncGen_CheckExact(gen) && !arg) { + if (result == Py_None && !arg) { /* Return NULL if called by gen_iternext() */ Py_CLEAR(result); } @@ -381,12 +379,14 @@ PyGen_am_send(PyObject *self, PyObject *arg, PyObject **result) return gen_send_ex(gen, arg, result); } +int +_PyAsyncGen_SetStopIterationValue(PyObject *value); + static PyObject * gen_set_stop_iteration(PyGenObject *gen, PyObject *result) { if (PyAsyncGen_CheckExact(gen)) { - assert(result == Py_None); - PyErr_SetNone(PyExc_StopAsyncIteration); + _PyAsyncGen_SetStopIterationValue(result); } else if (result == Py_None) { PyErr_SetNone(PyExc_StopIteration); @@ -426,7 +426,7 @@ gen_close_iter(PyObject *yf) { PyObject *retval = NULL; - if (PyGen_CheckExact(yf) || PyCoro_CheckExact(yf)) { + if (PyGen_CheckExact(yf) || PyCoro_CheckExact(yf) || PyAsyncGen_CheckExact(yf)) { retval = gen_close((PyObject *)yf, NULL); if (retval == NULL) return -1; @@ -622,7 +622,7 @@ the (type, val, tb) signature is deprecated, \n\ and may be removed in a future version of Python."); static PyObject * -_gen_throw(PyGenObject *gen, int close_on_genexit, +_gen_throw(PyGenObject *gen, PyObject *typ, PyObject *val, PyObject *tb) { int8_t frame_state = FT_ATOMIC_LOAD_INT8_RELAXED(gen->gi_frame_state); @@ -653,13 +653,18 @@ _gen_throw(PyGenObject *gen, int close_on_genexit, PyObject *yf = PyStackRef_AsPyObjectNew(_PyFrame_StackPeek(frame, 2)); PyObject *ret; int err; - if (PyErr_GivenExceptionMatches(typ, PyExc_GeneratorExit) && - close_on_genexit - ) { + if (PyErr_GivenExceptionMatches(typ, PyExc_GeneratorExit)) { /* Asynchronous generators *should not* be closed right away. We have to allow some awaits to work it through, hence the `close_on_genexit` parameter here. */ + // XXX: As of PEP 828, this doesn't seem to be true? + // In the above condition, there used to be a "&& close_on_genexit", + // where close_on_genexit was a parameter that was always zero when + // this was called from athrow(). This broke some tests/expected behavior + // for yield from in asyncgens. Removing the parameter didn't seem to cause + // any new test failures, nor could I reproduce any different behavior + // when experimenting with it, but we need to be careful. err = gen_close_iter(yf); Py_DECREF(yf); if (err < 0) { @@ -669,7 +674,7 @@ _gen_throw(PyGenObject *gen, int close_on_genexit, } PyThreadState *tstate = _PyThreadState_GET(); assert(tstate != NULL); - if (PyGen_CheckExact(yf) || PyCoro_CheckExact(yf)) { + if (PyGen_CheckExact(yf) || PyCoro_CheckExact(yf) || PyAsyncGen_CheckExact(yf)) { /* `yf` is a generator or a coroutine. */ /* Link frame into the stack to enable complete backtraces. */ @@ -680,9 +685,7 @@ _gen_throw(PyGenObject *gen, int close_on_genexit, tstate->current_frame = frame; /* Close the generator that we are currently iterating with 'yield from' or awaiting on with 'await'. */ - ret = _gen_throw((PyGenObject *)yf, close_on_genexit, - typ, val, tb); - _PyThreadState_UpdateLastProfiledFrame(tstate, frame, prev); + ret = _gen_throw((PyGenObject *)yf, typ, val, tb); tstate->current_frame = prev; frame->previous = NULL; } @@ -703,7 +706,6 @@ _gen_throw(PyGenObject *gen, int close_on_genexit, frame->previous = prev; tstate->current_frame = frame; ret = PyObject_CallFunctionObjArgs(meth, typ, val, tb, NULL); - _PyThreadState_UpdateLastProfiledFrame(tstate, frame, prev); tstate->current_frame = prev; frame->previous = NULL; Py_DECREF(meth); @@ -753,7 +755,7 @@ gen_throw(PyObject *op, PyObject *const *args, Py_ssize_t nargs) else if (nargs == 2) { val = args[1]; } - return _gen_throw(gen, 1, typ, val, tb); + return _gen_throw(gen, typ, val, tb); } @@ -780,8 +782,11 @@ gen_iternext(PyObject *self) * Returns 0 if StopIteration is set and -1 if any other exception is set. */ int -_PyGen_SetStopIterationValue(PyObject *value) +_PyAnyGen_SetStopIterationValue(PyObject *exc_class, PyObject *value) { + assert(exc_class != NULL); + assert(PyType_Check(exc_class)); + assert(value != NULL); assert(!PyErr_Occurred()); // Construct an exception instance manually with PyObject_CallOneArg() // but use PyErr_SetRaisedException() instead of PyErr_SetObject() as @@ -789,8 +794,8 @@ _PyGen_SetStopIterationValue(PyObject *value) // is a tuple, where the value of the StopIteration exception would be // set to 'value[0]' instead of 'value'. PyObject *exc = value == NULL - ? PyObject_CallNoArgs(PyExc_StopIteration) - : PyObject_CallOneArg(PyExc_StopIteration, value); + ? PyObject_CallNoArgs(exc_class) + : PyObject_CallOneArg(exc_class, value); if (exc == NULL) { return -1; } @@ -798,6 +803,18 @@ _PyGen_SetStopIterationValue(PyObject *value) return 0; } +int +_PyGen_SetStopIterationValue(PyObject *value) +{ + return _PyAnyGen_SetStopIterationValue(PyExc_StopIteration, value); +} + +int +_PyAsyncGen_SetStopIterationValue(PyObject *value) +{ + return _PyAnyGen_SetStopIterationValue(PyExc_StopAsyncIteration, value); +} + /* * If StopIteration exception is set, fetches its 'value' * attribute if any, otherwise sets pvalue to None. @@ -2352,8 +2369,6 @@ async_gen_athrow_send(PyObject *self, PyObject *arg) o->agt_gen->ag_closed = 1; retval = _gen_throw((PyGenObject *)gen, - 0, /* Do not close generator when - PyExc_GeneratorExit is passed */ PyExc_GeneratorExit, NULL, NULL); if (retval && _PyAsyncGenWrappedValue_CheckExact(retval)) { @@ -2362,8 +2377,6 @@ async_gen_athrow_send(PyObject *self, PyObject *arg) } } else { retval = _gen_throw((PyGenObject *)gen, - 0, /* Do not close generator when - PyExc_GeneratorExit is passed */ o->agt_typ, o->agt_val, o->agt_tb); retval = async_gen_unwrap_value(o->agt_gen, retval); } diff --git a/Python/bytecodes.c b/Python/bytecodes.c index 4d7b338e2dbd4c..5b8e77b092263c 100644 --- a/Python/bytecodes.c +++ b/Python/bytecodes.c @@ -1976,6 +1976,23 @@ dummy_func( } } + inst(CLEANUP_ASYNC_THROW, (iter, exc_value_st -- value)) { + PyObject *exc_value = PyStackRef_AsPyObjectBorrow(exc_value_st); + assert(exc_value != NULL); + assert(PyExceptionInstance_Check(exc_value)); + + int matches = PyErr_GivenExceptionMatches(exc_value, PyExc_StopAsyncIteration); + if (matches) { + value = PyStackRef_FromPyObjectNew(((PyStopAsyncIterationObject *)exc_value)->value); + DECREF_INPUTS(); + } + else { + _PyErr_SetRaisedException(tstate, Py_NewRef(exc_value)); + monitor_reraise(tstate, frame, this_instr); + goto exception_unwind; + } + } + inst(LOAD_COMMON_CONSTANT, ( -- value)) { // Keep in sync with _common_constants in opcode.py assert(oparg < NUM_COMMON_CONSTANTS); diff --git a/Python/codegen.c b/Python/codegen.c index f2c2b21d106fbd..760ff6bf5e10cb 100644 --- a/Python/codegen.c +++ b/Python/codegen.c @@ -514,6 +514,125 @@ codegen_add_yield_from(compiler *c, location loc, int await) return SUCCESS; } +static int +codegen_yield_from_async(compiler *c, location loc, expr_ty e) +{ + NEW_JUMP_TARGET_LABEL(c, send); + NEW_JUMP_TARGET_LABEL(c, exit); + NEW_JUMP_TARGET_LABEL(c, use_anext); + NEW_JUMP_TARGET_LABEL(c, got_coroutine); + + VISIT(c, expr, e->v.YieldFrom.value); + // Stack: [value] + + ADDOP_NAME(c, loc, LOAD_ATTR, &_Py_ID(__aiter__), names); + ADDOP(c, loc, PUSH_NULL); + ADDOP_I(c, loc, CALL, 0); + // Stack: [aiterator] + + ADDOP_LOAD_CONST(c, loc, Py_None); + // Stack: [aiterator, None] + + USE_LABEL(c, send); + + // Stack: [aiterator, asend_value] + ADDOP_I(c, loc, COPY, 1); + // Stack: [aiterator, asend_value, asend_value] + + ADDOP_LOAD_CONST(c, loc, Py_None); + // Stack: [aiterator, asend_value, asend_value, None] + + ADDOP_I(c, loc, IS_OP, 0); + // Stack: [aiterator, asend_value, bool] + + ADDOP_JUMP(c, loc, POP_JUMP_IF_TRUE, use_anext); + + ADDOP_I(c, loc, COPY, 2); + // Stack: [aiterator, asend_value, aiterator] + + ADDOP_NAME(c, loc, LOAD_ATTR, &_Py_ID(asend), names); + // Stack: [aiterator, asend_value, bound_method] + + ADDOP_I(c, loc, SWAP, 2); + // Stack: [aiterator, bound_method, asend_value] + + ADDOP(c, loc, PUSH_NULL); + // Stack: [aiterator, bound_method, asend_value, NULL] + + ADDOP_I(c, loc, SWAP, 2); + // Stack: [aiterator, bound_method, NULL, send_value] + + ADDOP_I(c, loc, CALL, 1); + // Stack: [aiterator, coroutine] + + ADDOP_JUMP(c, loc, JUMP_NO_INTERRUPT, got_coroutine); + + USE_LABEL(c, use_anext); + // Stack: [aiterator, asend_value] + + ADDOP(c, loc, POP_TOP); + // Stack: [aiterator] + + ADDOP_I(c, loc, COPY, 1); + // Stack: [aiterator, aiterator] + + ADDOP_NAME(c, loc, LOAD_ATTR, &_Py_ID(__anext__), names); + ADDOP(c, loc, PUSH_NULL); + ADDOP_I(c, loc, CALL, 0); + // Stack: [aiterator, coroutine] + + USE_LABEL(c, got_coroutine); + // Stack: [aiterator, coroutine] + + // Virtual try/except for the StopAsyncIteration + ADDOP_JUMP(c, loc, SETUP_FINALLY, exit); + + ADDOP(c, loc, PUSH_NULL); + ADDOP_LOAD_CONST(c, loc, Py_None); + // Stack: [aiterator, coroutine, NULL, None] + + ADD_YIELD_FROM(c, loc, 1); + // Stack: [aiterator, asend_result] + + ADDOP_I(c, loc, CALL_INTRINSIC_1, INTRINSIC_ASYNC_GEN_WRAP); + // Stack: [aiterator, wrapped_result] + + // Generators expect the iterable at stack_top[-2], so we have to make an + // extra copy. + ADDOP_I(c, loc, COPY, 2); + // Stack: [aiterator, wrapped_result, aiterator] + + ADDOP_I(c, loc, SWAP, 2); + // Stack: [aiterator, aiterator, wrapped_result] + + ADDOP_I(c, loc, YIELD_VALUE, 1); + // Stack: [aiterator, aiterator, resumed_value] + + ADDOP(c, NO_LOCATION, POP_BLOCK); + + ADDOP_I(c, loc, SWAP, 2); + // Stack: [aiterator, resumed_value, aiterator] + + ADDOP(c, loc, POP_TOP); + // Stack: [aiterator, resumed_value] + + ADDOP_JUMP(c, loc, JUMP_NO_INTERRUPT, send); + + USE_LABEL(c, exit); + // Stack: [aiterator, send_value, exc_value] (from SETUP_FINALLY) + + ADDOP_I(c, loc, SWAP, 2); + // Stack: [aiterator, exc_value, send_value] + + ADDOP(c, loc, POP_TOP); + // Stack: [aiterator, exc_value] + + ADDOP(c, loc, CLEANUP_ASYNC_THROW); + // Stack: [result] + + return SUCCESS; +} + static int codegen_pop_except_and_reraise(compiler *c, location loc) { @@ -5531,7 +5650,7 @@ codegen_visit_expr_impl(compiler *c, expr_ty e, bool result_is_unused) return _PyCompile_Error(c, loc, "'yield from' outside function"); } if (SCOPE_TYPE(c) == COMPILE_SCOPE_ASYNC_FUNCTION) { - return _PyCompile_Error(c, loc, "'yield from' inside async function"); + return codegen_yield_from_async(c, loc, e); } VISIT(c, expr, e->v.YieldFrom.value); ADDOP_I(c, loc, GET_ITER, GET_ITER_YIELD_FROM); diff --git a/Python/executor_cases.c.h b/Python/executor_cases.c.h index e45bbd7cceb295..620080db266cf2 100644 --- a/Python/executor_cases.c.h +++ b/Python/executor_cases.c.h @@ -9482,6 +9482,8 @@ break; } + /* _CLEANUP_ASYNC_THROW is not a viable micro-op for tier 2 because it uses the 'this_instr' variable */ + case _LOAD_COMMON_CONSTANT_r01: { CHECK_CURRENT_CACHED_VALUES(0); ASSERT_WITHIN_STACK_BOUNDS_IGNORING_CACHE(__FILE__, __LINE__); diff --git a/Python/generated_cases.c.h b/Python/generated_cases.c.h index 6178dc70c1b80e..ce2c691b941897 100644 --- a/Python/generated_cases.c.h +++ b/Python/generated_cases.c.h @@ -5152,6 +5152,63 @@ DISPATCH(); } + TARGET(CLEANUP_ASYNC_THROW) { + #if _Py_TAIL_CALL_INTERP + int opcode = CLEANUP_ASYNC_THROW; + (void)(opcode); + #endif + _Py_CODEUNIT* const this_instr = next_instr; + (void)this_instr; + frame->instr_ptr = next_instr; + next_instr += 1; + INSTRUCTION_STATS(CLEANUP_ASYNC_THROW); + _PyStackRef iter; + _PyStackRef exc_value_st; + _PyStackRef value; + exc_value_st = stack_pointer[-1]; + iter = stack_pointer[-2]; + PyObject *exc_value = PyStackRef_AsPyObjectBorrow(exc_value_st); + assert(exc_value != NULL); + assert(PyExceptionInstance_Check(exc_value)); + _PyFrame_SetStackPointer(frame, stack_pointer); + _PyFrame_StackPointerValidate(frame); + int matches = PyErr_GivenExceptionMatches(exc_value, PyExc_StopAsyncIteration); + _PyFrame_StackPointerInvalidate(frame); + if (matches) { + value = PyStackRef_FromPyObjectNew(((PyStopAsyncIterationObject *)exc_value)->value); + assert(stack_pointer == _PyFrame_GetStackPointer(frame)); + _PyFrame_StackPointerValidate(frame); + _PyStackRef tmp = iter; + iter = value; + stack_pointer[-2] = iter; + PyStackRef_CLOSE(tmp); + _PyFrame_StackPointerInvalidate(frame); + assert(stack_pointer == _PyFrame_GetStackPointer(frame)); + _PyFrame_StackPointerValidate(frame); + tmp = exc_value_st; + exc_value_st = PyStackRef_NULL; + stack_pointer[-1] = exc_value_st; + PyStackRef_CLOSE(tmp); + _PyFrame_StackPointerInvalidate(frame); + stack_pointer += -1; + ASSERT_WITHIN_STACK_BOUNDS(__FILE__, __LINE__); + } + else { + assert(stack_pointer == _PyFrame_GetStackPointer(frame)); + _PyFrame_StackPointerValidate(frame); + _PyErr_SetRaisedException(tstate, Py_NewRef(exc_value)); + _PyFrame_StackPointerInvalidate(frame); + assert(stack_pointer == _PyFrame_GetStackPointer(frame)); + _PyFrame_StackPointerValidate(frame); + monitor_reraise(tstate, frame, this_instr); + _PyFrame_StackPointerInvalidate(frame); + assert(stack_pointer == _PyFrame_GetStackPointer(frame)); + _PyFrame_StackPointerValidate(frame); + JUMP_TO_LABEL(exception_unwind); + } + DISPATCH(); + } + TARGET(CLEANUP_THROW) { #if _Py_TAIL_CALL_INTERP int opcode = CLEANUP_THROW; diff --git a/Python/opcode_targets.h b/Python/opcode_targets.h index 91b424773224e4..5c506eb250afd9 100644 --- a/Python/opcode_targets.h +++ b/Python/opcode_targets.h @@ -7,6 +7,7 @@ static void *opcode_targets_table[256] = { &&TARGET_CALL_FUNCTION_EX, &&TARGET_CHECK_EG_MATCH, &&TARGET_CHECK_EXC_MATCH, + &&TARGET_CLEANUP_ASYNC_THROW, &&TARGET_CLEANUP_THROW, &&TARGET_DELETE_SUBSCR, &&TARGET_END_FOR, @@ -16,8 +17,8 @@ static void *opcode_targets_table[256] = { &&TARGET_FORMAT_WITH_SPEC, &&TARGET_GET_AITER, &&TARGET_GET_ANEXT, - &&TARGET_GET_LEN, &&TARGET_RESERVED, + &&TARGET_GET_LEN, &&TARGET_INTERPRETER_EXIT, &&TARGET_LOAD_BUILD_CLASS, &&TARGET_LOAD_LOCALS, @@ -127,7 +128,6 @@ static void *opcode_targets_table[256] = { &&_unknown_opcode, &&_unknown_opcode, &&_unknown_opcode, - &&_unknown_opcode, &&TARGET_RESUME, &&TARGET_BINARY_OP_ADD_FLOAT, &&TARGET_BINARY_OP_ADD_INT, @@ -376,7 +376,7 @@ static void *opcode_tracing_targets_table[256] = { &&TARGET_TRACE_RECORD, &&TARGET_TRACE_RECORD, &&TARGET_TRACE_RECORD, - &&_unknown_opcode, + &&TARGET_TRACE_RECORD, &&_unknown_opcode, &&_unknown_opcode, &&_unknown_opcode, @@ -590,6 +590,7 @@ static PyObject *Py_PRESERVE_NONE_CC _TAIL_CALL_CALL_TUPLE_1(TAIL_CALL_PARAMS); static PyObject *Py_PRESERVE_NONE_CC _TAIL_CALL_CALL_TYPE_1(TAIL_CALL_PARAMS); static PyObject *Py_PRESERVE_NONE_CC _TAIL_CALL_CHECK_EG_MATCH(TAIL_CALL_PARAMS); static PyObject *Py_PRESERVE_NONE_CC _TAIL_CALL_CHECK_EXC_MATCH(TAIL_CALL_PARAMS); +static PyObject *Py_PRESERVE_NONE_CC _TAIL_CALL_CLEANUP_ASYNC_THROW(TAIL_CALL_PARAMS); static PyObject *Py_PRESERVE_NONE_CC _TAIL_CALL_CLEANUP_THROW(TAIL_CALL_PARAMS); static PyObject *Py_PRESERVE_NONE_CC _TAIL_CALL_COMPARE_OP(TAIL_CALL_PARAMS); static PyObject *Py_PRESERVE_NONE_CC _TAIL_CALL_COMPARE_OP_FLOAT(TAIL_CALL_PARAMS); @@ -833,6 +834,7 @@ static py_tail_call_funcptr instruction_funcptr_handler_table[256] = { [CALL_TYPE_1] = _TAIL_CALL_CALL_TYPE_1, [CHECK_EG_MATCH] = _TAIL_CALL_CHECK_EG_MATCH, [CHECK_EXC_MATCH] = _TAIL_CALL_CHECK_EXC_MATCH, + [CLEANUP_ASYNC_THROW] = _TAIL_CALL_CLEANUP_ASYNC_THROW, [CLEANUP_THROW] = _TAIL_CALL_CLEANUP_THROW, [COMPARE_OP] = _TAIL_CALL_COMPARE_OP, [COMPARE_OP_FLOAT] = _TAIL_CALL_COMPARE_OP_FLOAT, @@ -1005,7 +1007,6 @@ static py_tail_call_funcptr instruction_funcptr_handler_table[256] = { [UNPACK_SEQUENCE_TWO_TUPLE] = _TAIL_CALL_UNPACK_SEQUENCE_TWO_TUPLE, [WITH_EXCEPT_START] = _TAIL_CALL_WITH_EXCEPT_START, [YIELD_VALUE] = _TAIL_CALL_YIELD_VALUE, - [117] = _TAIL_CALL_UNKNOWN_OPCODE, [118] = _TAIL_CALL_UNKNOWN_OPCODE, [119] = _TAIL_CALL_UNKNOWN_OPCODE, [120] = _TAIL_CALL_UNKNOWN_OPCODE, @@ -1091,6 +1092,7 @@ static py_tail_call_funcptr instruction_funcptr_tracing_table[256] = { [CALL_TYPE_1] = _TAIL_CALL_TRACE_RECORD, [CHECK_EG_MATCH] = _TAIL_CALL_TRACE_RECORD, [CHECK_EXC_MATCH] = _TAIL_CALL_TRACE_RECORD, + [CLEANUP_ASYNC_THROW] = _TAIL_CALL_TRACE_RECORD, [CLEANUP_THROW] = _TAIL_CALL_TRACE_RECORD, [COMPARE_OP] = _TAIL_CALL_TRACE_RECORD, [COMPARE_OP_FLOAT] = _TAIL_CALL_TRACE_RECORD, @@ -1263,7 +1265,6 @@ static py_tail_call_funcptr instruction_funcptr_tracing_table[256] = { [UNPACK_SEQUENCE_TWO_TUPLE] = _TAIL_CALL_TRACE_RECORD, [WITH_EXCEPT_START] = _TAIL_CALL_TRACE_RECORD, [YIELD_VALUE] = _TAIL_CALL_TRACE_RECORD, - [117] = _TAIL_CALL_UNKNOWN_OPCODE, [118] = _TAIL_CALL_UNKNOWN_OPCODE, [119] = _TAIL_CALL_UNKNOWN_OPCODE, [120] = _TAIL_CALL_UNKNOWN_OPCODE, diff --git a/Python/optimizer_cases.c.h b/Python/optimizer_cases.c.h index 5e110360b81b44..b9a531d0aa0563 100644 --- a/Python/optimizer_cases.c.h +++ b/Python/optimizer_cases.c.h @@ -1988,6 +1988,8 @@ break; } + /* _CLEANUP_ASYNC_THROW is not a viable micro-op for tier 2 */ + case _LOAD_COMMON_CONSTANT: { JitOptRef value; assert(oparg < NUM_COMMON_CONSTANTS); From c6d0df22db8c5f6002c51b76bff10052b9cea6a1 Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Mon, 3 Aug 2026 12:19:32 -0400 Subject: [PATCH 2/6] Add tests and whatnot. --- Lib/test/test_asyncgen.py | 49 - Lib/test/test_coroutines.py | 4 - Lib/test/test_yield_from_async.py | 1722 +++++++++++++++++++++++++++++ Python/codegen.c | 3 - 4 files changed, 1722 insertions(+), 56 deletions(-) create mode 100644 Lib/test/test_yield_from_async.py diff --git a/Lib/test/test_asyncgen.py b/Lib/test/test_asyncgen.py index e9eecbc8341551..2ce8d4d442bbc0 100644 --- a/Lib/test/test_asyncgen.py +++ b/Lib/test/test_asyncgen.py @@ -82,55 +82,6 @@ async def anext_impl(): return anext_impl() -class AsyncGenSyntaxTest(unittest.TestCase): - - def test_async_gen_syntax_01(self): - code = '''async def foo(): - await abc - yield from 123 - ''' - - with self.assertRaisesRegex(SyntaxError, 'yield from.*inside async'): - exec(code, {}, {}) - - def test_async_gen_syntax_02(self): - code = '''async def foo(): - yield from 123 - ''' - - with self.assertRaisesRegex(SyntaxError, 'yield from.*inside async'): - exec(code, {}, {}) - - def test_async_gen_syntax_03(self): - code = '''async def foo(): - await abc - yield - return 123 - ''' - - with self.assertRaisesRegex(SyntaxError, 'return.*value.*async gen'): - exec(code, {}, {}) - - def test_async_gen_syntax_04(self): - code = '''async def foo(): - yield - return 123 - ''' - - with self.assertRaisesRegex(SyntaxError, 'return.*value.*async gen'): - exec(code, {}, {}) - - def test_async_gen_syntax_05(self): - code = '''async def foo(): - if 0: - yield - return 12 - ''' - - with self.assertRaisesRegex(SyntaxError, 'return.*value.*async gen'): - exec(code, {}, {}) - - class AsyncGenTest(unittest.TestCase): def compare_generators(self, sync_gen, async_gen): diff --git a/Lib/test/test_coroutines.py b/Lib/test/test_coroutines.py index 9d415238876c8f..6970097bd6afb0 100644 --- a/Lib/test/test_coroutines.py +++ b/Lib/test/test_coroutines.py @@ -90,10 +90,6 @@ def test_badsyntax_1(self): """await something()""", - """async def foo(): - yield from [] - """, - """async def foo(): await await fut """, diff --git a/Lib/test/test_yield_from_async.py b/Lib/test/test_yield_from_async.py new file mode 100644 index 00000000000000..cab44d8ba40414 --- /dev/null +++ b/Lib/test/test_yield_from_async.py @@ -0,0 +1,1722 @@ +""" +Test suite for PEP 828 (`yield from` in asyn cgenerator) + +Adapted from `test_yield_from`. Each adapted test mirrors its PEP 380 +counterpart by name with an `_ayf` suffix; `TestParityWithPEP380` enforces +the 1:1 mapping. Tests with no PEP 380 analogue go in `TestPEP828Extras`. +""" + +import unittest +import inspect +from functools import partial + +lazy from test import test_yield_from +from test.support import captured_stderr, disable_gc, gc_collect, run_yielding_async_fn, catch_unraisable_exception + +_async_test = partial(partial, run_yielding_async_fn) + +async def arange(*args): + for i in range(*args): + yield i + +class AsAsyncIterator: + def __init__(self, wrapped): + self._wrapped = iter(wrapped) + + def __aiter__(self): + return self + + async def __anext__(self): + try: + return self._wrapped.__next__() + except StopAsyncIteration as e: + raise RuntimeError("async generator raised StopAsyncIteration") from e + except StopIteration as e: + raise StopAsyncIteration(e.value) from e + +class TestPEP828Operation(unittest.TestCase): + """Test semantics. Mirrors `TestPEP380Operation` in `test_yield_from`.""" + + @_async_test + async def test_delegation_of_initial_next_to_subgenerator_ayf(self): + """ + Test delegation of initial anext() call to subgenerator + """ + trace = [] + async def g1(): + trace.append("Starting g1") + yield from g2() + trace.append("Finishing g1") + async def g2(): + trace.append("Starting g2") + yield 42 + trace.append("Finishing g2") + async for x in g1(): + trace.append("Yielded %s" % (x,)) + self.assertEqual(trace,[ + "Starting g1", + "Starting g2", + "Yielded 42", + "Finishing g2", + "Finishing g1", + ]) + + @_async_test + async def test_raising_exception_in_initial_next_call_ayf(self): + """ + Test raising exception in initial anext() call + """ + trace = [] + async def g1(): + try: + trace.append("Starting g1") + yield from g2() + finally: + trace.append("Finishing g1") + async def g2(): + try: + trace.append("Starting g2") + yield from AsAsyncIterator(()) + raise ValueError("spanish inquisition occurred") + finally: + trace.append("Finishing g2") + try: + async for x in g1(): + trace.append("Yielded %s" % (x,)) + except ValueError as e: + self.assertEqual(e.args[0], "spanish inquisition occurred") + else: + self.fail("subgenerator failed to raise ValueError") + self.assertEqual(trace, [ + "Starting g1", + "Starting g2", + "Finishing g2", + "Finishing g1", + ]) + + @_async_test + async def test_delegation_of_next_call_to_subgenerator_ayf(self): + """ + Test delegation of anext() call to subgenerator + """ + trace = [] + async def g1(): + trace.append("Starting g1") + yield "g1 ham" + yield from g2() + yield "g1 eggs" + trace.append("Finishing g1") + async def g2(): + trace.append("Starting g2") + yield "g2 spam" + yield "g2 more spam" + trace.append("Finishing g2") + async for x in g1(): + trace.append("Yielded %s" % (x,)) + self.assertEqual(trace,[ + "Starting g1", + "Yielded g1 ham", + "Starting g2", + "Yielded g2 spam", + "Yielded g2 more spam", + "Finishing g2", + "Yielded g1 eggs", + "Finishing g1", + ]) + + @_async_test + async def test_raising_exception_in_delegated_next_call_ayf(self): + """ + Test raising exception in delegated anext() call + """ + trace = [] + async def g1(): + try: + trace.append("Starting g1") + yield "g1 ham" + yield from g2() + yield "g1 eggs" + finally: + trace.append("Finishing g1") + async def g2(): + try: + trace.append("Starting g2") + yield "g2 spam" + raise ValueError("hovercraft is full of eels") + yield "g2 more spam" + finally: + trace.append("Finishing g2") + try: + async for x in g1(): + trace.append("Yielded %s" % (x,)) + except ValueError as e: + self.assertEqual(e.args[0], "hovercraft is full of eels") + else: + self.fail("subgenerator failed to raise ValueError") + self.assertEqual(trace,[ + "Starting g1", + "Yielded g1 ham", + "Starting g2", + "Yielded g2 spam", + "Finishing g2", + "Finishing g1", + ]) + + @_async_test + async def test_delegation_of_send_ayf(self): + """ + Test delegation of asend() + """ + trace = [] + async def g1(): + trace.append("Starting g1") + x = yield "g1 ham" + trace.append("g1 received %s" % (x,)) + yield from g2() + x = yield "g1 eggs" + trace.append("g1 received %s" % (x,)) + trace.append("Finishing g1") + async def g2(): + trace.append("Starting g2") + x = yield "g2 spam" + trace.append("g2 received %s" % (x,)) + x = yield "g2 more spam" + trace.append("g2 received %s" % (x,)) + trace.append("Finishing g2") + g = g1() + y = await anext(g) + x = 1 + try: + while 1: + y = await g.asend(x) + trace.append("Yielded %s" % (y,)) + x += 1 + except StopAsyncIteration: + pass + self.assertEqual(trace,[ + "Starting g1", + "g1 received 1", + "Starting g2", + "Yielded g2 spam", + "g2 received 2", + "Yielded g2 more spam", + "g2 received 3", + "Finishing g2", + "Yielded g1 eggs", + "g1 received 4", + "Finishing g1", + ]) + + @_async_test + async def test_handling_exception_while_delegating_send_ayf(self): + """ + Test handling exception while delegating 'asend' + """ + trace = [] + async def g1(): + trace.append("Starting g1") + x = yield "g1 ham" + trace.append("g1 received %s" % (x,)) + yield from g2() + x = yield "g1 eggs" + trace.append("g1 received %s" % (x,)) + trace.append("Finishing g1") + async def g2(): + trace.append("Starting g2") + x = yield "g2 spam" + trace.append("g2 received %s" % (x,)) + raise ValueError("hovercraft is full of eels") + x = yield "g2 more spam" + trace.append("g2 received %s" % (x,)) + trace.append("Finishing g2") + async def run(): + g = g1() + y = await anext(g) + x = 1 + try: + while 1: + y = await g.asend(x) + trace.append("Yielded %s" % (y,)) + x += 1 + except StopAsyncIteration: + trace.append("StopAsyncIteration") + with self.assertRaises(ValueError): + await run() + self.assertEqual(trace,[ + "Starting g1", + "g1 received 1", + "Starting g2", + "Yielded g2 spam", + "g2 received 2", + ]) + + @_async_test + async def test_delegating_close_ayf(self): + """ + Test delegating 'aclose' + """ + trace = [] + async def g1(): + try: + trace.append("Starting g1") + yield "g1 ham" + yield from g2() + yield "g1 eggs" + finally: + trace.append("Finishing g1") + async def g2(): + try: + trace.append("Starting g2") + yield "g2 spam" + yield "g2 more spam" + finally: + trace.append("Finishing g2") + g = g1() + for i in range(2): + x = await anext(g) + trace.append("Yielded %s" % (x,)) + await g.aclose() + self.assertEqual(trace,[ + "Starting g1", + "Yielded g1 ham", + "Starting g2", + "Yielded g2 spam", + "Finishing g2", + "Finishing g1" + ]) + + @_async_test + async def test_handing_exception_while_delegating_close_ayf(self): + """ + Test handling exception while delegating 'aclose' + """ + trace = [] + async def g1(): + try: + trace.append("Starting g1") + yield "g1 ham" + yield from g2() + yield "g1 eggs" + finally: + trace.append("Finishing g1") + async def g2(): + try: + trace.append("Starting g2") + yield "g2 spam" + yield "g2 more spam" + finally: + trace.append("Finishing g2") + raise ValueError("nybbles have exploded with delight") + try: + g = g1() + for i in range(2): + x = await anext(g) + trace.append("Yielded %s" % (x,)) + await g.aclose() + except ValueError as e: + self.assertEqual(e.args[0], "nybbles have exploded with delight") + self.assertIsInstance(e.__context__, GeneratorExit) + else: + self.fail("subgenerator failed to raise ValueError") + self.assertEqual(trace,[ + "Starting g1", + "Yielded g1 ham", + "Starting g2", + "Yielded g2 spam", + "Finishing g2", + "Finishing g1", + ]) + + @_async_test + async def test_delegating_throw_ayf(self): + """ + Test delegating 'athrow' + """ + trace = [] + async def g1(): + try: + trace.append("Starting g1") + yield "g1 ham" + yield from g2() + yield "g1 eggs" + finally: + trace.append("Finishing g1") + async def g2(): + try: + trace.append("Starting g2") + yield "g2 spam" + yield "g2 more spam" + finally: + trace.append("Finishing g2") + try: + g = g1() + for i in range(2): + x = await anext(g) + trace.append("Yielded %s" % (x,)) + e = ValueError("tomato ejected") + await g.athrow(e) + except ValueError as e: + self.assertEqual(e.args[0], "tomato ejected") + else: + self.fail("subgenerator failed to raise ValueError") + self.assertEqual(trace,[ + "Starting g1", + "Yielded g1 ham", + "Starting g2", + "Yielded g2 spam", + "Finishing g2", + "Finishing g1", + ]) + + @_async_test + async def test_value_attribute_of_StopIteration_exception_ayf(self): + """ + Test 'value' attribute of StopAsyncIteration exception + """ + trace = [] + async def pex(e): + trace.append("%s: %s" % (e.__class__.__name__, e)) + trace.append("value = %s" % (e.value,)) + e = StopAsyncIteration() + await pex(e) + e = StopAsyncIteration("spam") + await pex(e) + e.value = "eggs" + await pex(e) + self.assertEqual(trace,[ + "StopAsyncIteration: ", + "value = None", + "StopAsyncIteration: spam", + "value = spam", + "StopAsyncIteration: spam", + "value = eggs", + ]) + + @_async_test + async def test_exception_value_crash_ayf(self): + # There used to be a refcount error when the return value + # stored in the StopAsyncIteration has a refcount of 1. + async def g1(): + yield from g2() + async def g2(): + yield "g2" + return object() + self.assertEqual([x async for x in g1()], ["g2"]) + + @_async_test + async def test_generator_return_value_ayf(self): + """ + Test generator return value + """ + trace = [] + async def g1(): + trace.append("Starting g1") + yield "g1 ham" + ret = yield from g2() + trace.append("g2 returned %r" % (ret,)) + for v in 1, (2,), StopAsyncIteration(3): + ret = yield from g2(v) + trace.append("g2 returned %r" % (ret,)) + yield "g1 eggs" + trace.append("Finishing g1") + async def g2(v = None): + trace.append("Starting g2") + yield "g2 spam" + yield "g2 more spam" + trace.append("Finishing g2") + if v: + return v + async for x in g1(): + trace.append("Yielded %s" % (x,)) + self.assertEqual(trace,[ + "Starting g1", + "Yielded g1 ham", + "Starting g2", + "Yielded g2 spam", + "Yielded g2 more spam", + "Finishing g2", + "g2 returned None", + "Starting g2", + "Yielded g2 spam", + "Yielded g2 more spam", + "Finishing g2", + "g2 returned 1", + "Starting g2", + "Yielded g2 spam", + "Yielded g2 more spam", + "Finishing g2", + "g2 returned (2,)", + "Starting g2", + "Yielded g2 spam", + "Yielded g2 more spam", + "Finishing g2", + "g2 returned StopAsyncIteration(3)", + "Yielded g1 eggs", + "Finishing g1", + ]) + + @_async_test + async def test_delegation_of_next_to_non_generator_ayf(self): + """ + Test delegation of anext() to non-generator + """ + trace = [] + async def g(): + yield from arange(3) + async for x in g(): + trace.append("Yielded %s" % (x,)) + self.assertEqual(trace,[ + "Yielded 0", + "Yielded 1", + "Yielded 2", + ]) + + @_async_test + async def test_conversion_of_sendNone_to_next_ayf(self): + """ + Test conversion of asend(None) to anext() + """ + trace = [] + async def g(): + yield from arange(3) + gi = g() + for x in range(3): + y = await gi.asend(None) + trace.append("Yielded: %s" % (y,)) + self.assertEqual(trace,[ + "Yielded: 0", + "Yielded: 1", + "Yielded: 2", + ]) + + @_async_test + async def test_delegation_of_close_to_non_generator_ayf(self): + """ + Test delegation of aclose() to non-generator + """ + trace = [] + async def g(): + try: + trace.append("starting g") + yield from arange(3) + trace.append("g should not be here") + finally: + trace.append("finishing g") + gi = g() + await anext(gi) + with captured_stderr() as output: + await gi.aclose() + self.assertEqual(output.getvalue(), '') + self.assertEqual(trace,[ + "starting g", + "finishing g", + ]) + + @_async_test + async def test_delegating_throw_to_non_generator_ayf(self): + """ + Test delegating 'athrow' to non-generator + """ + trace = [] + async def g(): + try: + trace.append("Starting g") + yield from arange(10) + finally: + trace.append("Finishing g") + try: + gi = g() + for i in range(5): + x = await anext(gi) + trace.append("Yielded %s" % (x,)) + e = ValueError("tomato ejected") + await gi.athrow(e) + except ValueError as e: + self.assertEqual(e.args[0],"tomato ejected") + else: + self.fail("subgenerator failed to raise ValueError") + self.assertEqual(trace,[ + "Starting g", + "Yielded 0", + "Yielded 1", + "Yielded 2", + "Yielded 3", + "Yielded 4", + "Finishing g", + ]) + + @_async_test + async def test_attempting_to_send_to_non_generator_ayf(self): + """ + Test attempting to asend to non-generator + """ + trace = [] + async def g(): + try: + trace.append("starting g") + yield from AsAsyncIterator([1, 2, 3]) + trace.append("g should not be here") + finally: + trace.append("finishing g") + try: + gi = g() + await anext(gi) + for x in range(3): + y = await gi.asend(42) + trace.append("Should not have yielded: %s" % (y,)) + except AttributeError as e: + self.assertIn("send", e.args[0]) + else: + self.fail("was able to send into non-generator") + self.assertEqual(trace,[ + "starting g", + "finishing g", + ]) + + @_async_test + async def test_broken_getattr_handling_ayf(self): + """ + Test subiterator with a broken getattr implementation + """ + class Broken: + def __aiter__(self): + return self + async def __anext__(self): + return 1 + def __getattr__(self, attr): + 1/0 + + async def g(): + yield from Broken() + + with self.assertRaises(ZeroDivisionError): + gi = g() + self.assertEqual(await anext(gi), 1) + await gi.asend(1) + + with self.assertRaises(ZeroDivisionError): + gi = g() + self.assertEqual(await anext(gi), 1) + await gi.athrow(AttributeError) + + with catch_unraisable_exception() as cm: + gi = g() + self.assertEqual(await anext(gi), 1) + await gi.aclose() + + self.assertEqual(ZeroDivisionError, cm.unraisable.exc_type) + + @_async_test + async def test_exception_in_initial_next_call_ayf(self): + """ + Test exception in initial anext() call + """ + trace = [] + async def g1(): + trace.append("g1 about to yield from g2") + yield from g2() + trace.append("g1 should not be here") + async def g2(): + yield 1/0 + async def run(): + gi = g1() + await anext(gi) + with self.assertRaises(ZeroDivisionError): + await run() + self.assertEqual(trace,[ + "g1 about to yield from g2" + ]) + + @_async_test + async def test_attempted_yield_from_loop_ayf(self): + """ + Test attempted `yield from` loop + """ + trace = [] + async def g1(): + trace.append("g1: starting") + yield "y1" + trace.append("g1: about to yield from g2") + yield from g2() + trace.append("g1 should not be here") + + async def g2(): + trace.append("g2: starting") + yield "y2" + trace.append("g2: about to yield from g1") + yield from gi + trace.append("g2 should not be here") + try: + gi = g1() + async for y in gi: + trace.append("Yielded: %s" % (y,)) + except RuntimeError as e: + self.assertEqual(e.args[0],"anext(): asynchronous generator is already running") + else: + self.fail("subgenerator didn't raise RuntimeError") + self.assertEqual(trace,[ + "g1: starting", + "Yielded: y1", + "g1: about to yield from g2", + "g2: starting", + "Yielded: y2", + "g2: about to yield from g1", + ]) + + @_async_test + async def test_returning_value_from_delegated_throw_ayf(self): + """ + Test returning value from delegated 'athrow' + """ + trace = [] + async def g1(): + try: + trace.append("Starting g1") + yield "g1 ham" + yield from g2() + yield "g1 eggs" + finally: + trace.append("Finishing g1") + async def g2(): + try: + trace.append("Starting g2") + yield "g2 spam" + yield "g2 more spam" + except LunchError: + trace.append("Caught LunchError in g2") + yield "g2 lunch saved" + yield "g2 yet more spam" + class LunchError(Exception): + pass + g = g1() + for i in range(2): + x = await anext(g) + trace.append("Yielded %s" % (x,)) + e = LunchError("tomato ejected") + await g.athrow(e) + async for x in g: + trace.append("Yielded %s" % (x,)) + self.assertEqual(trace,[ + "Starting g1", + "Yielded g1 ham", + "Starting g2", + "Yielded g2 spam", + "Caught LunchError in g2", + "Yielded g2 yet more spam", + "Yielded g1 eggs", + "Finishing g1", + ]) + + @_async_test + async def test_next_and_return_with_value_ayf(self): + """ + Test anext and return with value + """ + trace = [] + async def f(r): + gi = g(r) + await anext(gi) + try: + trace.append("f resuming g") + await anext(gi) + trace.append("f SHOULD NOT BE HERE") + except StopAsyncIteration as e: + trace.append("f caught %r" % (e,)) + async def g(r): + trace.append("g starting") + yield + trace.append("g returning %r" % (r,)) + return r + await f(None) + await f(1) + await f((2,)) + await f(StopAsyncIteration(3)) + self.assertEqual(trace,[ + "g starting", + "f resuming g", + "g returning None", + "f caught StopAsyncIteration()", + "g starting", + "f resuming g", + "g returning 1", + "f caught StopAsyncIteration(1)", + "g starting", + "f resuming g", + "g returning (2,)", + "f caught StopAsyncIteration((2,))", + "g starting", + "f resuming g", + "g returning StopAsyncIteration(3)", + "f caught StopAsyncIteration(StopAsyncIteration(3))", + ]) + + @_async_test + async def test_send_and_return_with_value_ayf(self): + """ + Test asend and return with value + """ + trace = [] + async def f(r): + gi = g(r) + await anext(gi) + try: + trace.append("f sending spam to g") + await gi.asend("spam") + trace.append("f SHOULD NOT BE HERE") + except StopAsyncIteration as e: + trace.append("f caught %r" % (e,)) + async def g(r): + trace.append("g starting") + x = yield + trace.append("g received %r" % (x,)) + trace.append("g returning %r" % (r,)) + return r + await f(None) + await f(1) + await f((2,)) + await f(StopAsyncIteration(3)) + self.assertEqual(trace, [ + "g starting", + "f sending spam to g", + "g received 'spam'", + "g returning None", + "f caught StopAsyncIteration(None)", + "g starting", + "f sending spam to g", + "g received 'spam'", + "g returning 1", + 'f caught StopAsyncIteration(1)', + 'g starting', + 'f sending spam to g', + "g received 'spam'", + 'g returning (2,)', + 'f caught StopAsyncIteration((2,))', + 'g starting', + 'f sending spam to g', + "g received 'spam'", + 'g returning StopAsyncIteration(3)', + 'f caught StopAsyncIteration(StopAsyncIteration(3))' + ]) + + @_async_test + async def test_catching_exception_from_subgen_and_returning_ayf(self): + """ + Test catching an exception athrown into a + subgenerator and returning a value + """ + async def inner(): + try: + yield 1 + except ValueError: + trace.append("inner caught ValueError") + return value + + async def outer(): + v = yield from inner() + trace.append("inner returned %r to outer" % (v,)) + yield v + + for value in 2, (2,), StopAsyncIteration(2): + trace = [] + g = outer() + trace.append(await anext(g)) + trace.append(repr(await g.athrow(ValueError))) + self.assertEqual(trace, [ + 1, + "inner caught ValueError", + "inner returned %r to outer" % (value,), + repr(value), + ]) + + @_async_test + async def test_throwing_GeneratorExit_into_subgen_that_returns_ayf(self): + """ + Test athrow(GeneratorExit) into a subgenerator that + catches it and returns normally. + """ + trace = [] + async def f(): + try: + trace.append("Enter f") + yield + trace.append("Exit f") + except GeneratorExit: + return + async def g(): + trace.append("Enter g") + yield from f() + trace.append("Exit g") + try: + gi = g() + await anext(gi) + await gi.athrow(GeneratorExit) + except GeneratorExit: + pass + else: + self.fail("subgenerator failed to raise GeneratorExit") + self.assertEqual(trace,[ + "Enter g", + "Enter f", + ]) + + @_async_test + async def test_throwing_GeneratorExit_into_subgenerator_that_yields_ayf(self): + """ + Test athrow(GeneratorExit) into a subgenerator that + catches it and yields. + """ + trace = [] + async def f(): + try: + trace.append("Enter f") + yield + trace.append("Exit f") + except GeneratorExit: + yield + async def g(): + trace.append("Enter g") + yield from f() + trace.append("Exit g") + try: + gi = g() + await anext(gi) + await gi.athrow(GeneratorExit) + except RuntimeError as e: + self.assertEqual(e.args[0], "async generator ignored GeneratorExit") + else: + self.fail("subgenerator failed to raise GeneratorExit") + self.assertEqual(trace,[ + "Enter g", + "Enter f", + ]) + + @_async_test + async def test_throwing_GeneratorExit_into_subgen_that_raises_ayf(self): + """ + Test athrow(GeneratorExit) into a subgenerator that + catches it and raises a different exception. + """ + trace = [] + async def f(): + try: + trace.append("Enter f") + yield + trace.append("Exit f") + except GeneratorExit: + raise ValueError("Vorpal bunny encountered") + async def g(): + trace.append("Enter g") + yield from f() + trace.append("Exit g") + try: + gi = g() + await anext(gi) + await gi.athrow(GeneratorExit) + except ValueError as e: + self.assertEqual(e.args[0], "Vorpal bunny encountered") + self.assertIsInstance(e.__context__, GeneratorExit) + else: + self.fail("subgenerator failed to raise ValueError") + self.assertEqual(trace,[ + "Enter g", + "Enter f", + ]) + + @_async_test + async def test_yield_from_empty_ayf(self): + async def g(): + yield from AsAsyncIterator(()) + with self.assertRaises(StopAsyncIteration): + await anext(g()) + + @_async_test + async def test_delegating_generators_claim_to_be_running_ayf(self): + # Check with basic iteration + async def one(): + yield 0 + yield from two() + yield 3 + async def two(): + yield 1 + try: + yield from g1 + except RuntimeError: + pass + yield 2 + g1 = one() + self.assertEqual([e async for e in g1], [0, 1, 2, 3]) + + # Check with asend + g1 = one() + res = [await anext(g1)] + try: + while True: + res.append(await g1.asend(42)) + except StopAsyncIteration: + pass + self.assertEqual(res, [0, 1, 2, 3]) + + @_async_test + async def test_delegating_generators_claim_to_be_running_with_throw_ayf(self): + # Check with throw + class MyErr(Exception): + pass + async def one(): + try: + yield 0 + except MyErr: + pass + yield from two() + try: + yield 3 + except MyErr: + pass + async def two(): + try: + yield 1 + except MyErr: + pass + try: + yield from g1 + except RuntimeError: + pass + try: + yield 2 + except MyErr: + pass + g1 = one() + res = [await anext(g1)] + try: + while True: + res.append(await g1.athrow(MyErr)) + except StopAsyncIteration: + pass + except: + self.assertEqual(res, [0, 1, 2, 3]) + raise + + @_async_test + async def test_delegating_generators_claim_to_be_running_with_close_ayf(self): + # Check with close + class MyIt: + def __aiter__(self): + return self + async def __anext__(self): + return 42 + async def aclose(self_): + self.assertTrue(g1.gi_running) + with self.assertRaises(RuntimeError): + await anext(g1) + async def one(): + yield from MyIt() + g1 = one() + await anext(g1) + await g1.aclose() + + @_async_test + async def test_delegator_is_visible_to_debugger_ayf(self): + async def call_stack(): + return [f[3] for f in inspect.stack()] + + async def gen(): + yield await call_stack() + yield await call_stack() + yield await call_stack() + + async def spam(g): + yield from g + + async def eggs(g): + yield from g + + async for stack in spam(gen()): + self.assertTrue('spam' in stack) + + async for stack in spam(eggs(gen())): + self.assertTrue('spam' in stack and 'eggs' in stack) + + @_async_test + async def test_custom_iterator_return_ayf(self): + class MyIter: + def __aiter__(self): + return self + async def __anext__(self): + raise StopAsyncIteration(42) + async def gen(): + nonlocal ret + ret = yield from MyIter() + ret = None + [e async for e in gen()] + self.assertEqual(ret, 42) + + @_async_test + async def test_close_with_cleared_frame_ayf(self): + async def innermost(): + yield + async def inner(): + outer_gen = yield + yield from innermost() + async def outer(): + inner_gen = yield + yield from inner_gen + + with disable_gc(): + inner_gen = inner() + outer_gen = outer() + await outer_gen.asend(None) + await outer_gen.asend(inner_gen) + await outer_gen.asend(outer_gen) + + del outer_gen + del inner_gen + gc_collect() + + @_async_test + async def test_send_tuple_with_custom_generator_ayf(self): + class MyGen: + def __aiter__(self): + return self + async def __anext__(self): + return 42 + async def asend(self, what): + nonlocal v + v = what + return None + async def outer(): + v = yield from MyGen() + g = outer() + await anext(g) + v = None + await g.asend((1, 2, 3, 4)) + self.assertEqual(v, (1, 2, 3, 4)) + +class TestInterestingEdgeCases(unittest.TestCase): + """Interesting edge cases. Mirrors `TestInterestingEdgeCases` in `test_yield_from`.""" + + async def assert_stop_iteration(self, iterator): + with self.assertRaises(StopAsyncIteration) as caught: + await anext(iterator) + self.assertIsNone(caught.exception.value) + self.assertIsNone(caught.exception.__context__) + + def assert_generator_raised_stop_iteration(self): + return self.assertRaisesRegex(RuntimeError, r"^async generator raised StopAsyncIteration$") + + def assert_generator_ignored_generator_exit(self): + return self.assertRaisesRegex(RuntimeError, r"^async generator ignored GeneratorExit$") + + @_async_test + async def test_close_and_throw_work_ayf(self): + + yielded_first = sentinel("yielded_first") + yielded_second = sentinel("yielded_second") + returned = sentinel("returned") + + async def inner(): + yield yielded_first + yield yielded_second + return returned + + async def outer(): + return (yield from inner()) + + with self.subTest("aclose"): + g = outer() + self.assertIs(await anext(g), yielded_first) + await g.aclose() + await self.assert_stop_iteration(g) + + with self.subTest("athrow GeneratorExit"): + g = outer() + self.assertIs(await anext(g), yielded_first) + thrown = GeneratorExit() + with self.assertRaises(GeneratorExit) as caught: + await g.athrow(thrown) + self.assertIs(caught.exception, thrown) + self.assertIsNone(caught.exception.__context__) + await self.assert_stop_iteration(g) + + with self.subTest("athrow StopAsyncIteration"): + g = outer() + self.assertIs(await anext(g), yielded_first) + thrown = StopAsyncIteration() + with self.assert_generator_raised_stop_iteration() as caught: + await g.athrow(thrown) + self.assertIs(caught.exception.__context__, thrown) + self.assertIsNone(caught.exception.__context__.__context__) + await self.assert_stop_iteration(g) + + with self.subTest("athrow BaseException"): + g = outer() + self.assertIs(await anext(g), yielded_first) + thrown = BaseException() + with self.assertRaises(BaseException) as caught: + await g.athrow(thrown) + self.assertIs(caught.exception, thrown) + self.assertIsNone(caught.exception.__context__) + await self.assert_stop_iteration(g) + + with self.subTest("athrow Exception"): + g = outer() + self.assertIs(await anext(g), yielded_first) + thrown = Exception() + with self.assertRaises(Exception) as caught: + await g.athrow(thrown) + self.assertIs(caught.exception, thrown) + self.assertIsNone(caught.exception.__context__) + await self.assert_stop_iteration(g) + + @_async_test + async def test_close_and_throw_raise_generator_exit_ayf(self): + + yielded_first = sentinel("yielded_first") + yielded_second = sentinel("yielded_second") + returned = sentinel("returned") + + async def inner(): + try: + yield yielded_first + yield yielded_second + return returned + finally: + raise raised + + async def outer(): + return (yield from inner()) + + with self.subTest("aclose"): + g = outer() + self.assertIs(await anext(g), yielded_first) + raised = GeneratorExit() + # GeneratorExit is suppressed. This is analogous to PEP 342: + # https://peps.python.org/pep-0342/#new-generator-method-close + await g.aclose() + await self.assert_stop_iteration(g) + + with self.subTest("athrow GeneratorExit"): + g = outer() + self.assertIs(await anext(g), yielded_first) + raised = GeneratorExit() + thrown = GeneratorExit() + with self.assertRaises(GeneratorExit) as caught: + await g.athrow(thrown) + # The raised GeneratorExit is suppressed, but the thrown one + # propagates. This is analogous to PEP 380: + # https://peps.python.org/pep-0380/#proposal + self.assertIs(caught.exception, thrown) + self.assertIsNone(caught.exception.__context__) + await self.assert_stop_iteration(g) + + with self.subTest("athrow StopAsyncIteration"): + g = outer() + self.assertIs(await anext(g), yielded_first) + raised = GeneratorExit() + thrown = StopAsyncIteration() + with self.assertRaises(GeneratorExit) as caught: + await g.athrow(thrown) + self.assertIs(caught.exception, raised) + self.assertIs(caught.exception.__context__, thrown) + self.assertIsNone(caught.exception.__context__.__context__) + await self.assert_stop_iteration(g) + + with self.subTest("athrow BaseException"): + g = outer() + self.assertIs(await anext(g), yielded_first) + raised = GeneratorExit() + thrown = BaseException() + with self.assertRaises(GeneratorExit) as caught: + await g.athrow(thrown) + self.assertIs(caught.exception, raised) + self.assertIs(caught.exception.__context__, thrown) + self.assertIsNone(caught.exception.__context__.__context__) + await self.assert_stop_iteration(g) + + with self.subTest("athrow Exception"): + g = outer() + self.assertIs(await anext(g), yielded_first) + raised = GeneratorExit() + thrown = Exception() + with self.assertRaises(GeneratorExit) as caught: + await g.athrow(thrown) + self.assertIs(caught.exception, raised) + self.assertIs(caught.exception.__context__, thrown) + self.assertIsNone(caught.exception.__context__.__context__) + await self.assert_stop_iteration(g) + + @_async_test + async def test_close_and_throw_raise_stop_iteration_ayf(self): + + yielded_first = sentinel("yielded_first") + yielded_second = sentinel("yielded_second") + returned = sentinel("returned") + + async def inner(): + try: + yield yielded_first + yield yielded_second + return returned + finally: + raise raised + + async def outer(): + return (yield from inner()) + + with self.subTest("aclose"): + g = outer() + self.assertIs(await anext(g), yielded_first) + raised = StopAsyncIteration() + # PEP 479: + with self.assert_generator_raised_stop_iteration() as caught: + await g.aclose() + self.assertIs(caught.exception.__context__, raised) + self.assertIsInstance(caught.exception.__context__.__context__, GeneratorExit) + self.assertIsNone(caught.exception.__context__.__context__.__context__) + await self.assert_stop_iteration(g) + + with self.subTest("athrow GeneratorExit"): + g = outer() + self.assertIs(await anext(g), yielded_first) + raised = StopAsyncIteration() + thrown = GeneratorExit() + # PEP 479: + with self.assert_generator_raised_stop_iteration() as caught: + await g.athrow(thrown) + self.assertIs(caught.exception.__context__, raised) + # This isn't the same GeneratorExit as thrown! It's the one created + # by calling inner.aclose(): + self.assertIsInstance(caught.exception.__context__.__context__, GeneratorExit) + self.assertIsNone(caught.exception.__context__.__context__.__context__) + await self.assert_stop_iteration(g) + + with self.subTest("athrow StopAsyncIteration"): + g = outer() + self.assertIs(await anext(g), yielded_first) + raised = StopAsyncIteration() + thrown = StopAsyncIteration() + # PEP 479: + with self.assert_generator_raised_stop_iteration() as caught: + await g.athrow(thrown) + self.assertIs(caught.exception.__context__, raised) + self.assertIs(caught.exception.__context__.__context__, thrown) + self.assertIsNone(caught.exception.__context__.__context__.__context__) + await self.assert_stop_iteration(g) + + with self.subTest("athrow BaseException"): + g = outer() + self.assertIs(await anext(g), yielded_first) + raised = StopAsyncIteration() + thrown = BaseException() + # PEP 479: + with self.assert_generator_raised_stop_iteration() as caught: + await g.athrow(thrown) + self.assertIs(caught.exception.__context__, raised) + self.assertIs(caught.exception.__context__.__context__, thrown) + self.assertIsNone(caught.exception.__context__.__context__.__context__) + await self.assert_stop_iteration(g) + + with self.subTest("athrow Exception"): + g = outer() + self.assertIs(await anext(g), yielded_first) + raised = StopAsyncIteration() + thrown = Exception() + # PEP 479: + with self.assert_generator_raised_stop_iteration() as caught: + await g.athrow(thrown) + self.assertIs(caught.exception.__context__, raised) + self.assertIs(caught.exception.__context__.__context__, thrown) + self.assertIsNone(caught.exception.__context__.__context__.__context__) + await self.assert_stop_iteration(g) + + @_async_test + async def test_close_and_throw_raise_base_exception_ayf(self): + + yielded_first = sentinel("yielded_first") + yielded_second = sentinel("yielded_second") + returned = sentinel("returned") + + async def inner(): + try: + yield yielded_first + yield yielded_second + return returned + finally: + raise raised + + async def outer(): + return (yield from inner()) + + with self.subTest("aclose"): + g = outer() + self.assertIs(await anext(g), yielded_first) + raised = BaseException() + with self.assertRaises(BaseException) as caught: + await g.aclose() + self.assertIs(caught.exception, raised) + self.assertIsInstance(caught.exception.__context__, GeneratorExit) + self.assertIsNone(caught.exception.__context__.__context__) + await self.assert_stop_iteration(g) + + with self.subTest("athrow GeneratorExit"): + g = outer() + self.assertIs(await anext(g), yielded_first) + raised = BaseException() + thrown = GeneratorExit() + with self.assertRaises(BaseException) as caught: + await g.athrow(thrown) + self.assertIs(caught.exception, raised) + # This isn't the same GeneratorExit as thrown! It's the one created + # by calling inner.aclose(): + self.assertIsInstance(caught.exception.__context__, GeneratorExit) + self.assertIsNone(caught.exception.__context__.__context__) + await self.assert_stop_iteration(g) + + with self.subTest("athrow StopAsyncIteration"): + g = outer() + self.assertIs(await anext(g), yielded_first) + raised = BaseException() + thrown = StopAsyncIteration() + with self.assertRaises(BaseException) as caught: + await g.athrow(thrown) + self.assertIs(caught.exception, raised) + self.assertIs(caught.exception.__context__, thrown) + self.assertIsNone(caught.exception.__context__.__context__) + await self.assert_stop_iteration(g) + + with self.subTest("athrow BaseException"): + g = outer() + self.assertIs(await anext(g), yielded_first) + raised = BaseException() + thrown = BaseException() + with self.assertRaises(BaseException) as caught: + await g.athrow(thrown) + self.assertIs(caught.exception, raised) + self.assertIs(caught.exception.__context__, thrown) + self.assertIsNone(caught.exception.__context__.__context__) + await self.assert_stop_iteration(g) + + with self.subTest("athrow Exception"): + g = outer() + self.assertIs(await anext(g), yielded_first) + raised = BaseException() + thrown = Exception() + with self.assertRaises(BaseException) as caught: + await g.athrow(thrown) + self.assertIs(caught.exception, raised) + self.assertIs(caught.exception.__context__, thrown) + self.assertIsNone(caught.exception.__context__.__context__) + await self.assert_stop_iteration(g) + + @_async_test + async def test_close_and_throw_raise_exception_ayf(self): + + yielded_first = sentinel("yielded_first") + yielded_second = sentinel("yielded_second") + returned = sentinel("returned") + + async def inner(): + try: + yield yielded_first + yield yielded_second + return returned + finally: + raise raised + + async def outer(): + return (yield from inner()) + + with self.subTest("aclose"): + g = outer() + self.assertIs(await anext(g), yielded_first) + raised = Exception() + with self.assertRaises(Exception) as caught: + await g.aclose() + self.assertIs(caught.exception, raised) + self.assertIsInstance(caught.exception.__context__, GeneratorExit) + self.assertIsNone(caught.exception.__context__.__context__) + await self.assert_stop_iteration(g) + + with self.subTest("athrow GeneratorExit"): + g = outer() + self.assertIs(await anext(g), yielded_first) + raised = Exception() + thrown = GeneratorExit() + with self.assertRaises(Exception) as caught: + await g.athrow(thrown) + self.assertIs(caught.exception, raised) + # This isn't the same GeneratorExit as thrown! It's the one created + # by calling inner.aclose(): + self.assertIsInstance(caught.exception.__context__, GeneratorExit) + self.assertIsNone(caught.exception.__context__.__context__) + await self.assert_stop_iteration(g) + + with self.subTest("athrow StopAsyncIteration"): + g = outer() + self.assertIs(await anext(g), yielded_first) + raised = Exception() + thrown = StopAsyncIteration() + with self.assertRaises(Exception) as caught: + await g.athrow(thrown) + self.assertIs(caught.exception, raised) + self.assertIs(caught.exception.__context__, thrown) + self.assertIsNone(caught.exception.__context__.__context__) + await self.assert_stop_iteration(g) + + with self.subTest("athrow BaseException"): + g = outer() + self.assertIs(await anext(g), yielded_first) + raised = Exception() + thrown = BaseException() + with self.assertRaises(Exception) as caught: + await g.athrow(thrown) + self.assertIs(caught.exception, raised) + self.assertIs(caught.exception.__context__, thrown) + self.assertIsNone(caught.exception.__context__.__context__) + await self.assert_stop_iteration(g) + + with self.subTest("athrow Exception"): + g = outer() + self.assertIs(await anext(g), yielded_first) + raised = Exception() + thrown = Exception() + with self.assertRaises(Exception) as caught: + await g.athrow(thrown) + self.assertIs(caught.exception, raised) + self.assertIs(caught.exception.__context__, thrown) + self.assertIsNone(caught.exception.__context__.__context__) + await self.assert_stop_iteration(g) + + @_async_test + async def test_close_and_throw_yield_ayf(self): + + yielded_first = sentinel("yielded_first") + yielded_second = sentinel("yielded_second") + returned = sentinel("returned") + + async def inner(): + try: + yield yielded_first + finally: + yield yielded_second + return returned + + async def outer(): + return (yield from inner()) + + with self.subTest("aclose"): + g = outer() + self.assertIs(await anext(g), yielded_first) + # No chaining happens. This is analogous to PEP 342: + # https://peps.python.org/pep-0342/#new-generator-method-close + with self.assert_generator_ignored_generator_exit() as caught: + await g.aclose() + self.assertIsNone(caught.exception.__context__) + await self.assert_stop_iteration(g) + + with self.subTest("athrow GeneratorExit"): + g = outer() + self.assertIs(await anext(g), yielded_first) + thrown = GeneratorExit() + # No chaining happens. This is analogous to PEP 342: + # https://peps.python.org/pep-0342/#new-generator-method-close + with self.assert_generator_ignored_generator_exit() as caught: + await g.athrow(thrown) + self.assertIsNone(caught.exception.__context__) + await self.assert_stop_iteration(g) + + with self.subTest("athrow StopAsyncIteration"): + g = outer() + self.assertIs(await anext(g), yielded_first) + thrown = StopAsyncIteration() + self.assertEqual(await g.athrow(thrown), yielded_second) + # PEP 479: + with self.assert_generator_raised_stop_iteration() as caught: + await anext(g) + self.assertIs(caught.exception.__context__, thrown) + self.assertIsNone(caught.exception.__context__.__context__) + await self.assert_stop_iteration(g) + + with self.subTest("athrow BaseException"): + g = outer() + self.assertIs(await anext(g), yielded_first) + thrown = BaseException() + self.assertEqual(await g.athrow(thrown), yielded_second) + with self.assertRaises(BaseException) as caught: + await anext(g) + self.assertIs(caught.exception, thrown) + self.assertIsNone(caught.exception.__context__) + await self.assert_stop_iteration(g) + + with self.subTest("athrow Exception"): + g = outer() + self.assertIs(await anext(g), yielded_first) + thrown = Exception() + self.assertEqual(await g.athrow(thrown), yielded_second) + with self.assertRaises(Exception) as caught: + await anext(g) + self.assertIs(caught.exception, thrown) + self.assertIsNone(caught.exception.__context__) + await self.assert_stop_iteration(g) + + @_async_test + async def test_close_and_throw_return_ayf(self): + yielded_first = sentinel("yielded_first") + yielded_second = sentinel("yielded_second") + returned = sentinel("returned") + + async def inner(): + try: + yield yielded_first + yield yielded_second + except: + pass + return returned + + async def outer(): + return (yield from inner()) + + with self.subTest("aclose"): + g = outer() + self.assertIs(await anext(g), yielded_first) + # StopAsyncIteration is suppressed. This is analogous to PEP 342: + # https://peps.python.org/pep-0342/#new-generator-method-close + await g.aclose() + await self.assert_stop_iteration(g) + + with self.subTest("athrow GeneratorExit"): + g = outer() + self.assertIs(await anext(g), yielded_first) + thrown = GeneratorExit() + # StopAsyncIteration is suppressed. This is analogous to PEP 342: + # https://peps.python.org/pep-0342/#new-generator-method-close + with self.assertRaises(GeneratorExit) as caught: + await g.athrow(thrown) + self.assertIs(caught.exception, thrown) + self.assertIsNone(caught.exception.__context__) + await self.assert_stop_iteration(g) + + with self.subTest("athrow StopAsyncIteration"): + g = outer() + self.assertIs(await anext(g), yielded_first) + thrown = StopAsyncIteration() + with self.assertRaises(StopAsyncIteration) as caught: + await g.athrow(thrown) + self.assertIs(caught.exception.value, returned) + self.assertIsNone(caught.exception.__context__) + await self.assert_stop_iteration(g) + + with self.subTest("athrow BaseException"): + g = outer() + self.assertIs(await anext(g), yielded_first) + thrown = BaseException() + with self.assertRaises(StopAsyncIteration) as caught: + await g.athrow(thrown) + self.assertIs(caught.exception.value, returned) + self.assertIsNone(caught.exception.__context__) + await self.assert_stop_iteration(g) + + with self.subTest("athrow Exception"): + g = outer() + self.assertIs(await anext(g), yielded_first) + thrown = Exception() + with self.assertRaises(StopAsyncIteration) as caught: + await g.athrow(thrown) + self.assertIs(caught.exception.value, returned) + self.assertIsNone(caught.exception.__context__) + await self.assert_stop_iteration(g) + + @_async_test + async def test_throws_in_iter_ayf(self): + class Silly: + async def __aiter__(self): + yield from AsAsyncIterator(()) + raise RuntimeError("nobody expects the spanish inquisition") + + async def my_generator(): + yield from Silly() + + with self.assertRaisesRegex(RuntimeError, "nobody expects the spanish inquisition"): + await anext(my_generator()) + + +class TestParityWithPEP380(unittest.TestCase): + """Enforce PEP 828 tests cover every PEP 380 test.""" + + def assert_parity(self, base_class, variant_class, *, suffix): + """Assert variant_class is in 1:1 parity with base_class via ``suffix``. + + Every method ``test_xxx`` on ``base_class`` must have a counterpart + ``test_xxx`` on ``variant_class`` and vice versa. Variant-only + tests belong in a separate TestCase class. + """ + def test_methods(cls): + return {n for n in dir(cls) if n.startswith("test_")} + + def fqn(cls): + return f"{cls.__module__}.{cls.__qualname__}" + + expected = {n + suffix for n in test_methods(base_class)} + actual = test_methods(variant_class) + missing = sorted(expected - actual) + extra = sorted(actual - expected) + if missing or extra: + lines = [ + f"{fqn(variant_class)} is not a 1:1 mirror of " + f"{fqn(base_class)} (suffix {suffix!r}):" + ] + for name in missing: + lines.append(f" missing in {fqn(variant_class)}: {name}") + for name in extra: + lines.append(f" no counterpart in {fqn(base_class)}: {name}") + self.fail("\n".join(lines)) + + def test_TestPEP828Operation(self): + self.assert_parity( + test_yield_from.TestPEP380Operation, + TestPEP828Operation, + suffix="_ayf", + ) + + def test_TestInterestingEdgeCases(self): + self.assert_parity( + test_yield_from.TestInterestingEdgeCases, + TestInterestingEdgeCases, + suffix="_ayf", + ) + + +class TestPEP828Extras(unittest.TestCase): + """Tests with no PEP 380 counterpart. + + Anything added here describes behaviour specific to ``yield from``. + Tests that have a logical equivalent in plain ``yield from`` belong in + ``TestPEP828Operation`` or ``TestInterestingEdgeCases`` and are + parity-checked against ``test_yield_from``. + """ + + @_async_test + async def test_delegate_exception(self): + yielded_first = sentinel("yielded_first") + yielded_second = sentinel("yielded_second") + returned = sentinel("returned") + + async def inner(): + try: + yield yielded_first + yield yielded_second + return returned + finally: + raise raised + + async def outer(): + return (yield from inner()) + + g = outer() + assert (await anext(g)) is yielded_first + raised = RuntimeError() + with self.assertRaises(RuntimeError) as error: + await g.athrow(SystemError) + self.assertIs(raised, error.exception) + + +if __name__ == '__main__': + unittest.main() diff --git a/Python/codegen.c b/Python/codegen.c index 760ff6bf5e10cb..d684cad0b81902 100644 --- a/Python/codegen.c +++ b/Python/codegen.c @@ -2393,9 +2393,6 @@ codegen_return(compiler *c, stmt_ty s) if (!_PyST_IsFunctionLike(ste)) { return _PyCompile_Error(c, loc, "'return' outside function"); } - if (s->v.Return.value != NULL && ste->ste_coroutine && ste->ste_generator) { - return _PyCompile_Error(c, loc, "'return' with value in async generator"); - } if (preserve_tos) { VISIT(c, expr, s->v.Return.value); From 6a891918474c394b519ee21ea12b7e8a1b738f56 Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Mon, 3 Aug 2026 12:20:15 -0400 Subject: [PATCH 3/6] Add blurb. --- .../2026-08-03-12-20-13.gh-issue-155126.sem_Gk.rst | 1 + 1 file changed, 1 insertion(+) create mode 100644 Misc/NEWS.d/next/Core_and_Builtins/2026-08-03-12-20-13.gh-issue-155126.sem_Gk.rst diff --git a/Misc/NEWS.d/next/Core_and_Builtins/2026-08-03-12-20-13.gh-issue-155126.sem_Gk.rst b/Misc/NEWS.d/next/Core_and_Builtins/2026-08-03-12-20-13.gh-issue-155126.sem_Gk.rst new file mode 100644 index 00000000000000..142d09a5c420fb --- /dev/null +++ b/Misc/NEWS.d/next/Core_and_Builtins/2026-08-03-12-20-13.gh-issue-155126.sem_Gk.rst @@ -0,0 +1 @@ +Implement :pep:`828`. From 8a7b7b0cb335b78b03073ce6374be44e137e1662 Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Mon, 3 Aug 2026 12:30:45 -0400 Subject: [PATCH 4/6] Fix versionadded marker. --- Doc/library/exceptions.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/library/exceptions.rst b/Doc/library/exceptions.rst index f327239216b4b0..cf1b7919613564 100644 --- a/Doc/library/exceptions.rst +++ b/Doc/library/exceptions.rst @@ -532,7 +532,7 @@ The following exceptions are the exceptions that are usually raised. defaults to :const:`None`. This is used for the result of ``async yield from`` expressions (see :ref:`async-yield-from`). - .. versionadded: next + .. versionadded:: next .. versionadded:: 3.5 From 1d62419dcc369d63e35f6a56b6114ce82e9bcf67 Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Mon, 3 Aug 2026 12:59:25 -0400 Subject: [PATCH 5/6] Regen frozenmain test. --- Programs/test_frozenmain.h | 26 +++++++++++++------------- 1 file changed, 13 insertions(+), 13 deletions(-) diff --git a/Programs/test_frozenmain.h b/Programs/test_frozenmain.h index c82b791439a3fa..a23945ab66f042 100644 --- a/Programs/test_frozenmain.h +++ b/Programs/test_frozenmain.h @@ -1,19 +1,19 @@ // Auto-generated by Programs/freeze_test_frozenmain.py unsigned char M_test_frozenmain[] = { 227,0,0,0,0,0,0,0,0,0,0,0,0,9,0,0, - 0,0,0,0,0,243,188,0,0,0,128,0,0,0,90,0, - 77,7,69,0,112,0,90,0,77,7,69,4,112,1,89,2, - 31,0,78,1,50,1,0,0,0,0,0,0,29,0,89,2, - 31,0,78,2,89,0,76,6,0,0,0,0,0,0,0,0, - 0,0,0,0,0,0,0,0,0,0,50,2,0,0,0,0, - 0,0,29,0,89,1,76,8,0,0,0,0,0,0,0,0, - 0,0,0,0,0,0,0,0,0,0,31,0,50,0,0,0, - 0,0,0,0,78,3,42,26,0,0,0,0,0,0,0,0, - 0,0,112,5,78,6,67,0,0,0,65,24,0,0,112,6, - 89,2,31,0,78,4,89,6,12,0,78,5,89,5,89,6, - 42,26,0,0,0,0,0,0,0,0,0,0,12,0,48,4, - 50,1,0,0,0,0,0,0,29,0,71,26,0,0,9,0, - 28,0,77,7,33,0,41,7,233,0,0,0,0,122,18,70, + 0,0,0,0,0,243,188,0,0,0,128,0,0,0,91,0, + 78,7,70,0,113,0,91,0,78,7,70,4,113,1,90,2, + 32,0,79,1,51,1,0,0,0,0,0,0,30,0,90,2, + 32,0,79,2,90,0,77,6,0,0,0,0,0,0,0,0, + 0,0,0,0,0,0,0,0,0,0,51,2,0,0,0,0, + 0,0,30,0,90,1,77,8,0,0,0,0,0,0,0,0, + 0,0,0,0,0,0,0,0,0,0,32,0,51,0,0,0, + 0,0,0,0,79,3,43,26,0,0,0,0,0,0,0,0, + 0,0,113,5,79,6,68,0,0,0,66,24,0,0,113,6, + 90,2,32,0,79,4,90,6,13,0,79,5,90,5,90,6, + 43,26,0,0,0,0,0,0,0,0,0,0,13,0,49,4, + 51,1,0,0,0,0,0,0,30,0,72,26,0,0,10,0, + 29,0,78,7,34,0,41,7,233,0,0,0,0,122,18,70, 114,111,122,101,110,32,72,101,108,108,111,32,87,111,114,108, 100,122,8,115,121,115,46,97,114,103,118,218,6,99,111,110, 102,105,103,122,7,99,111,110,102,105,103,32,122,2,58,32, From c1ac46104535323cb061b328f95d446a54cdbfa7 Mon Sep 17 00:00:00 2001 From: Peter Bierma Date: Mon, 3 Aug 2026 20:11:52 -0400 Subject: [PATCH 6/6] Remove remnants of old proposal. --- Doc/library/exceptions.rst | 2 +- Doc/reference/expressions.rst | 12 ++++++------ 2 files changed, 7 insertions(+), 7 deletions(-) diff --git a/Doc/library/exceptions.rst b/Doc/library/exceptions.rst index cf1b7919613564..d67a9140413ffd 100644 --- a/Doc/library/exceptions.rst +++ b/Doc/library/exceptions.rst @@ -530,7 +530,7 @@ The following exceptions are the exceptions that are usually raised. This is given as an argument when constructing the exception, and defaults to :const:`None`. This is used for the result of - ``async yield from`` expressions (see :ref:`async-yield-from`). + ``yield from`` expressions (see :ref:`async-yield-from`). .. versionadded:: next diff --git a/Doc/reference/expressions.rst b/Doc/reference/expressions.rst index 29e0e6d1340137..26a6234e64ff13 100644 --- a/Doc/reference/expressions.rst +++ b/Doc/reference/expressions.rst @@ -1388,8 +1388,8 @@ In particular: :widths: auto :header-rows: 1 - * * ``yield from`` construct - * ``async yield from`` construct + * * Synchronous ``yield from`` + * Asynchronous ``yield from`` * * :meth:`~object.__iter__` * :meth:`~object.__aiter__` * * :meth:`~generator.__next__` @@ -1404,7 +1404,7 @@ To describe the above: * The object being delegated to must be asynchronously iterable (that is, it must implement ``__aiter__`` instead of ``__iter__``). * When ``anext`` is called on the parent generator (the one that contains - ``async yield from``), ``__anext__`` will be invoked on the subgenerator. + ``yield from``), ``__anext__`` will be invoked on the subgenerator. In contrast, a synchronous ``yield from`` would invoke ``__next__`` instead. (Note that calling ``asend`` with a ``None`` value is equivalent to calling ``anext()``, and thus applies here.) @@ -1412,7 +1412,7 @@ To describe the above: subgenerator (the object returned by ``__aiter__`` in this case). This means that a call to ``parent_generator.asend(x)`` is semantically equivalent to ``sub_generator.asend(x)``, where ``parent_generator`` is currently executing - an ``async yield from`` on ``sub_generator``. + an asynchronous ``yield from`` on ``sub_generator``. * The result of the expression is retrieved through :attr:`StopAsyncIteration.value` instead of :attr:`StopIteration.value`. @@ -1449,8 +1449,8 @@ An example of usage for ``yield from`` in async generator: File "", line 1, in await ag.athrow(ValueError("Nobody expects the Spanish Inquisition")) File "", line 8, in counter - final_number = async yield from sleepy_count(4) - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + final_number = yield from sleepy_count(4) + ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "", line 3, in sleepy_count result = yield num ^^^^^^^^^