test(pyamber): add unit tests for stoppable_queue_blocking_thread#6669
test(pyamber): add unit tests for stoppable_queue_blocking_thread#6669Ma77Ball wants to merge 10 commits into
Conversation
Automated Reviewer SuggestionsBased on the
|
|
| config | throughput | MB/s | latency | max Δ latest / 7d | |
|---|---|---|---|---|---|
| 🔴 | bs=10 sw=10 sl=64 | 541 | 0.33 | 18,854/22,574/22,574 us | 🔴 +9.5% / 🔴 +49.4% |
| 🔴 | bs=100 sw=10 sl=64 | 1,096 | 0.669 | 85,066/132,549/132,549 us | 🔴 +15.0% / 🔴 +24.6% |
| ⚪ | bs=1000 sw=10 sl=64 | 1,362 | 0.831 | 733,777/775,957/775,957 us | ⚪ within ±5% / 🟢 +31.7% |
Baseline details
Latest main a351f44 from same runner
| config | metric | PR | latest main | 7d avg | Δ latest | Δ 7d |
|---|---|---|---|---|---|---|
| bs=10 sw=10 sl=64 | throughput | 541 tuples/sec | 583 tuples/sec | 767.66 tuples/sec | -7.2% | -29.5% |
| bs=10 sw=10 sl=64 | MB/s | 0.33 MB/s | 0.356 MB/s | 0.469 MB/s | -7.3% | -29.6% |
| bs=10 sw=10 sl=64 | p50 | 18,854 us | 17,220 us | 12,623 us | +9.5% | +49.4% |
| bs=10 sw=10 sl=64 | p95 | 22,574 us | 21,335 us | 16,011 us | +5.8% | +41.0% |
| bs=10 sw=10 sl=64 | p99 | 22,574 us | 21,335 us | 18,856 us | +5.8% | +19.7% |
| bs=100 sw=10 sl=64 | throughput | 1,096 tuples/sec | 1,176 tuples/sec | 998.44 tuples/sec | -6.8% | +9.8% |
| bs=100 sw=10 sl=64 | MB/s | 0.669 MB/s | 0.718 MB/s | 0.609 MB/s | -6.8% | +9.8% |
| bs=100 sw=10 sl=64 | p50 | 85,066 us | 82,491 us | 100,463 us | +3.1% | -15.3% |
| bs=100 sw=10 sl=64 | p95 | 132,549 us | 115,247 us | 106,400 us | +15.0% | +24.6% |
| bs=100 sw=10 sl=64 | p99 | 132,549 us | 115,247 us | 117,712 us | +15.0% | +12.6% |
| bs=1000 sw=10 sl=64 | throughput | 1,362 tuples/sec | 1,367 tuples/sec | 1,034 tuples/sec | -0.4% | +31.7% |
| bs=1000 sw=10 sl=64 | MB/s | 0.831 MB/s | 0.834 MB/s | 0.631 MB/s | -0.4% | +31.7% |
| bs=1000 sw=10 sl=64 | p50 | 733,777 us | 726,749 us | 973,294 us | +1.0% | -24.6% |
| bs=1000 sw=10 sl=64 | p95 | 775,957 us | 787,163 us | 1,019,213 us | -1.4% | -23.9% |
| bs=1000 sw=10 sl=64 | p99 | 775,957 us | 787,163 us | 1,049,896 us | -1.4% | -26.1% |
Raw CSV
config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,369.99,200,128000,541,0.330,18853.96,22574.38,22574.38
1,100,10,64,20,1824.19,2000,1280000,1096,0.669,85065.58,132549.23,132549.23
2,1000,10,64,20,14685.23,20000,12800000,1362,0.831,733777.10,775956.66,775956.66There was a problem hiding this comment.
Pull request overview
Adds Python unit test coverage for Amber’s StoppableQueueBlockingRunnable (the stop-sentinel + run-loop lifecycle behavior), improving confidence in the stoppable-queue interrupt mechanism without modifying production code.
Changes:
- Introduces a new pytest module covering
stop()enqueuingRUNNABLE_STOPandinterruptible_get()raisingInterruptRunnableonly for the stop control. - Adds run-loop lifecycle tests verifying
pre_start()/receive()sequencing and thatpost_stop()runs even whenreceive()raises. - Uses small helper fakes (
FakeQueue,RecordingRunnable,RaisingRunnable) to test behavior deterministically without real blocking.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Signed-off-by: Xinyuan Lin <xinyual3@uci.edu>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #6669 +/- ##
=============================================
+ Coverage 77.30% 92.15% +14.85%
=============================================
Files 1161 121 -1040
Lines 45922 4385 -41537
Branches 5101 0 -5101
=============================================
- Hits 35499 4041 -31458
+ Misses 8840 344 -8496
+ Partials 1583 0 -1583
☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
….com:Ma77Ball/texera into test/stoppable-queue-blocking-thread-py-spec
|
Merged main to trigger coverage report. |
|
Closing as it doesn't increase coverage: #6669 (comment) |
|
Will reevaluate after fixing the Codecov Report. |
What changes were proposed in this PR?
Any related issues, documentation, discussions?
Closes: #6668
How was this PR tested?
cd amber && python -m pytest src/test/python/core/util/stoppable/test_stoppable_queue_blocking_thread.py, expect all 7 tests passing.Was this PR authored or co-authored using generative AI tooling?
Co-authored with Claude Opus 4.8 in compliance with ASF