gh-143055: Delegate to subiterators when unpacking in generator expressions

[graingert]

Unpacking a sub-iterable with * in a generator expression (PEP 798) now delegates to the sub-iterable using yield from semantics, rather than re-yielding each value with a manual loop.

Sync generator expressions

(*sub for sub in subs) now forwards send() and throw() to the sub-iterator currently being unpacked, and discards the sub-iterator's return value:

>>> def sub():
...     while True:
...         print("sub got", (yield "value"))
>>> g = (*sub() for _ in range(1))
>>> next(g)
'value'
>>> g.send(42)
sub got 42
'value'

Asynchronous generator expressions

* unpacking is synchronous (per PEP 448), so the sub-iterable is always a sync iterable. But it is delegated to from inside an async generator, where each yielded value must be wrapped as an async-generator value — otherwise the async-generator machinery treats it as an awaited value. The throw path is the subtle case: _gen_throw forwards throws to the sub in the runtime and returns the result bypassing any bytecode-level wrap (and that same path is shared with await/async for passthrough, which must stay unwrapped).

To handle this, a small internal iterator _PyAsyncGenUnpack wraps the sync iterable so that every value it produces — via __next__, am_send, and throw — is wrapped, while the return value and exceptions pass through unwrapped. It is constructed via a new INTRINSIC_ASYNC_GEN_UNPACK intrinsic emitted only in the async-generator case. As a result asend(), athrow() and aclose() are forwarded to the sub-iterator's send(), throw() and close().

No async yield from / PEP 828 syntax is introduced; this is purely about delegating PEP 798 unpacking.

Notes

• The PYC_MAGIC_NUMBER and the managed static type count are bumped (new helper type + changed codegen).
• This makes 3.16 generator-expression unpacking semantically differ from 3.15 (which shipped the re-yield behaviour). It may be worth considering for 3.15 as well.

Tests

New unittest.TestCase classes in Lib/test/test_unpack_ex.py cover sync send/throw/close forwarding and async asend/athrow/aclose forwarding. The async tests drive the generators by hand (no IsolatedAsyncioTestCase/asyncio), matching the convention in test_asyncgen.py.

🤖 Generated with Claude Code

• Issue: Implement PEP 798 – Unpacking in Comprehensions #143055

[read-the-docs-community]

Documentation build overview

📚 cpython-previews | 🛠️ Build #33364398 | 📁 Comparing dee268c against main (2670cb0)

  🔍 Preview build  

9 files changed · ± 9 modified

± Modified

• howto/mro.html
• library/curses.html
• library/dialog.html
• library/hashlib.html
• library/numbers.html
• library/tkinter.html
• reference/expressions.html
• whatsnew/3.16.html
• whatsnew/changelog.html

[ZeroIntensity]

@1st1 has expressed the view that an asynchronous generator should never delegate to a synchronous one because of some nasty edge cases. See this section of PEP 828.

[graingert]

@1st1 has expressed the view that an asynchronous generator should never delegate to a synchronous one because of some nasty edge cases. See this section of PEP 828.

The hazard in that section is async with asyncio.timeout(...): yield from subgen() — yielding across a structured-concurrency scope. You can't construct that here: a generator expression has no statement body, so there's nowhere to open an async with (timeout, TaskGroup, a nursery) inside (*sub for sub in subs). No scope, so nothing for the runtime to inject into.

And it's wrong independent of this PR anyway — yielding repeatedly across a structured-concurrency scope is never valid; the single yield in @asynccontextmanager is the only sanctioned case. So the example just doesn't bear on *-unpack delegation. The sub only ever sees throw()/close() the consumer forwards via athrow()/aclose(), exactly as in the sync genexp.

[ZeroIntensity]

I don't appreciate you just taking my comment and pasting Claude's response -- please use your own words!

The async with asyncio.timeout thing is just there as an example; fundamentally, this PR allows mixing async and sync generator frames, which is not good. But anyway, asyncio.timeout is still a problem with this PR:

async def main():
    async with asyncio.timeout(1):
        # The synchronous subgenerator is not constructed to handle asynchronous exceptions
        # (such as asyncio.CancelledError). If the timeout expires, the synchronous subgenerator
        # may swallow them.
        async for item in (*subgen() async for _ in whatever()):
            await asyncio.sleep(0.5)

This is only one of many potential problems. I had a long discussion with Yury about this topic at PyCon last month, and he eventually convinced me that delegating from an async to a sync generator is a bad idea.

It's unlikely that people do this in practice, but it doesn't matter. This PR essentially adds a way to sneakily convert a synchronous generator into an asynchronous one, which opens the door to a bunch of weird bugs and increased maintenance burden.

In fact, if PEP 828 is accepted, you could take this a step further to get the exact idea that Yury rejected:

async def main():
    ag = (*subgenerator() async for _ in single_element())
    async yield from ag  # Would delegate to a synchronous generator!

[graingert]

Sorry for using claude I'll avoid using it in future comments on the tracker.

If the timeout expires, the synchronous subgenerator

# may swallow them. The synchronous generator will never receive a CancelledError because that is only thrown out from Futures that are cancelled Thomas Grainger

…

On Mon, 29 Jun 2026, 22:31 Peter Bierma, ***@***.***> wrote: *ZeroIntensity* left a comment (python/cpython#152550) <#152550 (comment)> I don't appreciate you just taking my comment and pasting Claude's response -- please use your own words! The async with asyncio.timeout thing is just there as an example; fundamentally, this PR allows mixing async and sync generator frames, which is not good. But anyway, asyncio.timeout is still a problem with this PR: async def main(): async with asyncio.timeout(1): # The synchronous subgenerator is not constructed to handle asynchronous exceptions # (such as asyncio.CancelledError). If the timeout expires, the synchronous subgenerator # may swallow them. async for item in (*subgen() async for _ in whatever()): await asyncio.sleep(0.5) This is only one of many potential problems. I had a long discussion with Yury about this topic at PyCon last month, and he eventually convinced me that delegating from an async to a sync generator is a bad idea. It's unlikely that people do this in practice, but it doesn't matter. This PR essentially adds a way to sneakily convert a synchronous generator into an asynchronous one, which opens the door to a bunch of weird bugs and increased maintenance burden. In fact, if PEP 828 is accepted, you could take this a step further to get the exact idea that Yury rejected: async def main(): ag = (*subgenerator() async for _ in single_element()) async yield from ag # Would delegate to a synchronous generator! — Reply to this email directly, view it on GitHub <#152550?email_source=notifications&email_token=AADFATFCFBHOTZQVT64URN35CLN3NA5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTIOBTG4ZDKMBSHA22M4TFMFZW63VGMF2XI2DPOKSWK5TFNZ2KYZTPN52GK4S7MNWGSY3L#issuecomment-4837250285>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AADFATFTI4QOVIEIL2HTNE35CLN3NAVCNFSNUABEKJSXA33TNF2G64TZHM4DCNJZHA4TMMJ3JFZXG5LFHM2DONRVHEYDENRXGOQXMAQ> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[graingert]

With the correct code using aclosing it shows that CancelledError won't be thrown into the wrong place. Trace where the cancellation actually lands. A sync generator never awaits we can guarantee it doesn't yield a Future, i.e there is no suspension point inside subgen().

async def main():
    async with asyncio.timeout(1):
        # The synchronous subgenerator is not constructed to handle asynchronous exceptions
        # (such as asyncio.CancelledError). If the timeout expires, the synchronous subgenerator
        # will never recieve a CancelledError. Instead it will get a GeneratorExit when the agen is a aclosed()
        agen = (*subgen() async for _ in whatever())
        async with contextlib.aclosing(agen):
            async for item in agen:
                await asyncio.sleep(0.5)

The CancelledError can only be thrown out of whatever() or asyncio.sleep() as they are (or produce) the only coroutines that delegate to a Future

[ZeroIntensity]

I think you're missing my overall point -- we have explicitly rejected the ability to delegate to synchronous generators from asynchronous ones. I'm not going to continue rehashing that discussion here, but since we rejected the idea for normal async generators, that should apply to generator expressions too.