Skip to content

test(pyamber): add unit tests for stoppable_queue_blocking_thread#6669

Open
Ma77Ball wants to merge 10 commits into
apache:mainfrom
Ma77Ball:test/stoppable-queue-blocking-thread-py-spec
Open

test(pyamber): add unit tests for stoppable_queue_blocking_thread#6669
Ma77Ball wants to merge 10 commits into
apache:mainfrom
Ma77Ball:test/stoppable-queue-blocking-thread-py-spec

Conversation

@Ma77Ball

Copy link
Copy Markdown
Contributor

What changes were proposed in this PR?

  • Add unit tests for StoppableQueueBlockingRunnable.
  • Cover stop() enqueuing the RUNNABLE_STOP sentinel and interruptible_get() raising InterruptRunnable only on that exact control.
  • Cover run() processing entries until interrupted and always calling post_stop(), including on unexpected errors.

Any related issues, documentation, discussions?

Closes: #6668

How was this PR tested?

  • Run: cd amber && python -m pytest src/test/python/core/util/stoppable/test_stoppable_queue_blocking_thread.py, expect all 7 tests passing.
  • Test-only change; no production code is modified.

Was this PR authored or co-authored using generative AI tooling?

Co-authored with Claude Opus 4.8 in compliance with ASF

@github-actions

Copy link
Copy Markdown
Contributor

Automated Reviewer Suggestions

Based on the git blame history of the changed files, we recommend the following reviewers:

  • No candidates found from git blame history.

@github-actions

github-actions Bot commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

⚠️ Benchmark changes need a look

🟢 0 better · 🔴 9 worse · ⚪ 6 noise (<±5%) · 0 without baseline

Compared against main a351f44 benchmarked on this same runner, so the delta is largely free of cross-runner hardware noise. The "7d avg" column still reflects the gh-pages dashboard. Treat <±5% as noise unless repeated.

Dashboard · Run

config throughput MB/s latency max Δ latest / 7d
🔴 bs=10 sw=10 sl=64 541 0.33 18,854/22,574/22,574 us 🔴 +9.5% / 🔴 +49.4%
🔴 bs=100 sw=10 sl=64 1,096 0.669 85,066/132,549/132,549 us 🔴 +15.0% / 🔴 +24.6%
bs=1000 sw=10 sl=64 1,362 0.831 733,777/775,957/775,957 us ⚪ within ±5% / 🟢 +31.7%
Baseline details

Latest main a351f44 from same runner

config metric PR latest main 7d avg Δ latest Δ 7d
bs=10 sw=10 sl=64 throughput 541 tuples/sec 583 tuples/sec 767.66 tuples/sec -7.2% -29.5%
bs=10 sw=10 sl=64 MB/s 0.33 MB/s 0.356 MB/s 0.469 MB/s -7.3% -29.6%
bs=10 sw=10 sl=64 p50 18,854 us 17,220 us 12,623 us +9.5% +49.4%
bs=10 sw=10 sl=64 p95 22,574 us 21,335 us 16,011 us +5.8% +41.0%
bs=10 sw=10 sl=64 p99 22,574 us 21,335 us 18,856 us +5.8% +19.7%
bs=100 sw=10 sl=64 throughput 1,096 tuples/sec 1,176 tuples/sec 998.44 tuples/sec -6.8% +9.8%
bs=100 sw=10 sl=64 MB/s 0.669 MB/s 0.718 MB/s 0.609 MB/s -6.8% +9.8%
bs=100 sw=10 sl=64 p50 85,066 us 82,491 us 100,463 us +3.1% -15.3%
bs=100 sw=10 sl=64 p95 132,549 us 115,247 us 106,400 us +15.0% +24.6%
bs=100 sw=10 sl=64 p99 132,549 us 115,247 us 117,712 us +15.0% +12.6%
bs=1000 sw=10 sl=64 throughput 1,362 tuples/sec 1,367 tuples/sec 1,034 tuples/sec -0.4% +31.7%
bs=1000 sw=10 sl=64 MB/s 0.831 MB/s 0.834 MB/s 0.631 MB/s -0.4% +31.7%
bs=1000 sw=10 sl=64 p50 733,777 us 726,749 us 973,294 us +1.0% -24.6%
bs=1000 sw=10 sl=64 p95 775,957 us 787,163 us 1,019,213 us -1.4% -23.9%
bs=1000 sw=10 sl=64 p99 775,957 us 787,163 us 1,049,896 us -1.4% -26.1%
Raw CSV
config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,369.99,200,128000,541,0.330,18853.96,22574.38,22574.38
1,100,10,64,20,1824.19,2000,1280000,1096,0.669,85065.58,132549.23,132549.23
2,1000,10,64,20,14685.23,20000,12800000,1362,0.831,733777.10,775956.66,775956.66

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds Python unit test coverage for Amber’s StoppableQueueBlockingRunnable (the stop-sentinel + run-loop lifecycle behavior), improving confidence in the stoppable-queue interrupt mechanism without modifying production code.

Changes:

  • Introduces a new pytest module covering stop() enqueuing RUNNABLE_STOP and interruptible_get() raising InterruptRunnable only for the stop control.
  • Adds run-loop lifecycle tests verifying pre_start()/receive() sequencing and that post_stop() runs even when receive() raises.
  • Uses small helper fakes (FakeQueue, RecordingRunnable, RaisingRunnable) to test behavior deterministically without real blocking.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread amber/src/test/python/core/util/stoppable/test_stoppable_queue_blocking_thread.py Outdated
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Xinyuan Lin <xinyual3@uci.edu>
@aglinxinyuan
aglinxinyuan enabled auto-merge July 22, 2026 01:56
@codecov-commenter

codecov-commenter commented Jul 22, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 92.15%. Comparing base (a351f44) to head (7b75d1e).

Additional details and impacted files
@@              Coverage Diff              @@
##               main    #6669       +/-   ##
=============================================
+ Coverage     77.30%   92.15%   +14.85%     
=============================================
  Files          1161      121     -1040     
  Lines         45922     4385    -41537     
  Branches       5101        0     -5101     
=============================================
- Hits          35499     4041    -31458     
+ Misses         8840      344     -8496     
+ Partials       1583        0     -1583     
Flag Coverage Δ
access-control-service ?
agent-service ?
amber ?
computing-unit-managing-service ?
config-service ?
file-service ?
frontend ?
notebook-migration-service ?
pyamber 92.15% <ø> (ø)
workflow-compiling-service ?

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@aglinxinyuan
aglinxinyuan disabled auto-merge July 22, 2026 08:40
@aglinxinyuan
aglinxinyuan enabled auto-merge July 22, 2026 08:40
@aglinxinyuan

Copy link
Copy Markdown
Contributor

Merged main to trigger coverage report.

@aglinxinyuan
aglinxinyuan disabled auto-merge July 23, 2026 05:16
@aglinxinyuan

Copy link
Copy Markdown
Contributor

Closing as it doesn't increase coverage: #6669 (comment)

@aglinxinyuan

Copy link
Copy Markdown
Contributor

Will reevaluate after fixing the Codecov Report.

@aglinxinyuan aglinxinyuan reopened this Jul 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add unit test coverage for stoppable_queue_blocking_thread

4 participants