ThreadCache workers leaking spawner's Context on py314+ free-threaded builds#3473
Conversation
for more information, see https://pre-commit.ci
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #3473 +/- ##
===============================================
Coverage 100.00000% 100.00000%
===============================================
Files 128 128
Lines 19423 19439 +16
Branches 1317 1318 +1
===============================================
+ Hits 19423 19439 +16
🚀 New features to boost your workflow:
|
for more information, see https://pre-commit.ci
…dcache' into free-threaded-context-leak-threadcache
for more information, see https://pre-commit.ci
A5rocks
left a comment
There was a problem hiding this comment.
Makes sense to me. Could you add a test? I guess just something to make sure that threads don't inherit context would work well enough.
…dcache' into free-threaded-context-leak-threadcache
for more information, see https://pre-commit.ci
…dcache' into free-threaded-context-leak-threadcache
A5rocks
left a comment
There was a problem hiding this comment.
Test could be simplified I think, but looks good to me.
| "https://github.com/pypy/pypy/issues/5075)" | ||
| ), | ||
| ) | ||
| async def test_worker_thread_context_not_leaked() -> None: |
There was a problem hiding this comment.
IMO you should just check whether context gets carried over, no need for GC or whatever. Technically there could be a bug where we clear it in the frame before handing it to the user but store it elsewhere -- and you could test for that using weakref (to ensure it works on PyPy) -- but IMO not necessary.
There was a problem hiding this comment.
Yeah, we could use weakref too. I leaned more towards how anyio did it.
for more information, see https://pre-commit.ci
|
Forgive these so many commits. Not sure if you squash commits when merging, but feel free to tidy them up if needed. |
…dcache' into free-threaded-context-leak-threadcache
|
@A5rocks what do you think about this approach? used weakref and just asserted that the object is dead. |
|
PyPy failures are now irrelevant because we don't need |
|
By the way, this was found while exercising anyio's test suite on the Python 3.14t free-threaded build. anyio has a regression test that checks for leaked references from its worker-thread pool, and running it against trio's backend surfaced this. |
|
Thanks for this! |
|
Hey @EmmanuelNiyonshuti, it looks like that was the first time we merged one of your PRs! Thanks so much! 🎉 🎂 If you want to keep contributing, we'd love to have you. So, I just sent you an invitation to join the python-trio organization on Github! If you accept, then here's what will happen:
If you want to read more, here's the relevant section in our contributing guide. Alternatively, you're free to decline or ignore the invitation. You'll still be able to contribute as much or as little as you like, and I won't hassle you about joining again. But if you ever change your mind, just let us know and we'll send another invitation. We'd love to have you, but more importantly we want you to do whatever's best for you. If you have any questions, well... I am just a humble Python script, so I probably can't help. But please do post a comment here, or in our chat, or on our forum, whatever's easiest, and someone will help you out! |
fixes #3472