Bug description
itertools.zip_longest has a use-after-free when one iterator's callback re-enters the zip_longest and exhausts a sibling iterator.
zip_longest_next_lock_held() borrows an iterator from lz->ittuple with PyTuple_GET_ITEM (no Py_INCREF), then calls PyIter_Next(it). If that call re-enters and drains the same iterator, the inner call does PyTuple_SET_ITEM(ittuple, i, NULL) + Py_DECREF(it). When the tuple holds the only reference, it is freed while the outer call still uses it.
The reproducer uses filter(None, ...) so the predicate calls bool(item), and del leaves the tuple as the sole owner of the filter:
import itertools, gc
zl_ref = [None]
class CustomIter:
def __init__(self, data):
self.data = data
self.idx = 0
def __iter__(self):
return self
def __next__(self):
if self.idx >= len(self.data):
raise StopIteration
val = self.data[self.idx]
self.idx += 1
return val
class Evil:
armed = True
def __bool__(self):
if Evil.armed and zl_ref[0] is not None:
Evil.armed = False
try:
while True: # re-enter and drain the filter iterator
next(zl_ref[0])
except StopIteration:
pass
return False
f = filter(None, CustomIter([Evil() for _ in range(3)]))
zl = itertools.zip_longest(f, range(5), fillvalue=-1)
zl_ref[0] = zl
del f # ittuple now owns the only ref to the filter
gc.collect()
print(next(zl))
Run with PYTHONMALLOC=debug python repro.py -> SIGSEGV. With the fix it runs fine.
CPython versions tested on
main
Operating systems tested on
macOS
Bug description
itertools.zip_longesthas a use-after-free when one iterator's callback re-enters the zip_longest and exhausts a sibling iterator.zip_longest_next_lock_held()borrows an iterator fromlz->ittuplewithPyTuple_GET_ITEM(noPy_INCREF), then callsPyIter_Next(it). If that call re-enters and drains the same iterator, the inner call doesPyTuple_SET_ITEM(ittuple, i, NULL)+Py_DECREF(it). When the tuple holds the only reference,itis freed while the outer call still uses it.The reproducer uses
filter(None, ...)so the predicate callsbool(item), anddelleaves the tuple as the sole owner of the filter:Run with
PYTHONMALLOC=debug python repro.py-> SIGSEGV. With the fix it runs fine.CPython versions tested on
main
Operating systems tested on
macOS