Skip to content

itertools.zip_longest use-after-free via re-entrant iterator exhaust #154672

Description

@tonghuaroot

Bug description

itertools.zip_longest has a use-after-free when one iterator's callback re-enters the zip_longest and exhausts a sibling iterator.

zip_longest_next_lock_held() borrows an iterator from lz->ittuple with PyTuple_GET_ITEM (no Py_INCREF), then calls PyIter_Next(it). If that call re-enters and drains the same iterator, the inner call does PyTuple_SET_ITEM(ittuple, i, NULL) + Py_DECREF(it). When the tuple holds the only reference, it is freed while the outer call still uses it.

The reproducer uses filter(None, ...) so the predicate calls bool(item), and del leaves the tuple as the sole owner of the filter:

import itertools, gc

zl_ref = [None]

class CustomIter:
    def __init__(self, data):
        self.data = data
        self.idx = 0
    def __iter__(self):
        return self
    def __next__(self):
        if self.idx >= len(self.data):
            raise StopIteration
        val = self.data[self.idx]
        self.idx += 1
        return val

class Evil:
    armed = True
    def __bool__(self):
        if Evil.armed and zl_ref[0] is not None:
            Evil.armed = False
            try:
                while True:          # re-enter and drain the filter iterator
                    next(zl_ref[0])
            except StopIteration:
                pass
        return False

f = filter(None, CustomIter([Evil() for _ in range(3)]))
zl = itertools.zip_longest(f, range(5), fillvalue=-1)
zl_ref[0] = zl
del f                                # ittuple now owns the only ref to the filter
gc.collect()
print(next(zl))

Run with PYTHONMALLOC=debug python repro.py -> SIGSEGV. With the fix it runs fine.

CPython versions tested on

main

Operating systems tested on

macOS

Metadata

Metadata

Assignees

No one assigned

    Labels

    extension-modulesC modules in the Modules dirtype-crashA hard crash of the interpreter, possibly with a core dump

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions