Skip to content

DO NOT MERGE Combine PiecewiseSequence take stack#8837

Draft
danking wants to merge 39 commits into
developfrom
codex/piecewise-sequence-combined-pr
Draft

DO NOT MERGE Combine PiecewiseSequence take stack#8837
danking wants to merge 39 commits into
developfrom
codex/piecewise-sequence-combined-pr

Conversation

@danking

@danking danking commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

Summary

Combines the non-benchmark PiecewiseSequence take stack into one PR:

  • core PiecewiseSequence array
  • FixedSizeList take using PiecewiseSequence
  • run-take consumers for contiguous slices
  • VarBin and VarBinView PiecewiseSequence take specializations
  • cursor-copy slice materialization
  • List take element gathering via PiecewiseSequence

Excludes the benchmark-only PR #8830.

Validation

  • Ran all eight non-CUDA CodSpeed microbenchmark shards locally on this PR branch and on origin/develop using cargo codspeed build/run -m walltime.
  • Parsed 1,759 common benchmark medians from both branches.
  • At a 10% median walltime cutoff: 253 common benchmarks changed (161 faster, 92 slower), plus 128 PR-only benchmarks.
  • Local walltime is noisier than CodSpeed simulation; the strongest intended improvements are in take_fsl and listview_rebuild.

danking added 30 commits July 16, 2026 19:36
Signed-off-by: Daniel King <dan@spiraldb.com>
Signed-off-by: Daniel King <dan@spiraldb.com>
Signed-off-by: Daniel King <dan@spiraldb.com>
Signed-off-by: Daniel King <dan@spiraldb.com>
Signed-off-by: Daniel King <dan@spiraldb.com>
Signed-off-by: Daniel King <dan@spiraldb.com>
Signed-off-by: Daniel King <dan@spiraldb.com>
Signed-off-by: Daniel King <dan@spiraldb.com>
Signed-off-by: Daniel King <dan@spiraldb.com>
Signed-off-by: Daniel King <dan@spiraldb.com>
Signed-off-by: Daniel King <dan@spiraldb.com>
Signed-off-by: Daniel King <dan@spiraldb.com>
Signed-off-by: Daniel King <dan@spiraldb.com>
Signed-off-by: Daniel King <dan@spiraldb.com>
Signed-off-by: Daniel King <dan@spiraldb.com>
Signed-off-by: Daniel King <dan@spiraldb.com>
Signed-off-by: Daniel King <dan@spiraldb.com>
Signed-off-by: Daniel King <dan@spiraldb.com>
Signed-off-by: Daniel King <dan@spiraldb.com>
Signed-off-by: Daniel King <dan@spiraldb.com>
Signed-off-by: Daniel King <dan@spiraldb.com>
Signed-off-by: Daniel King <dan@spiraldb.com>
Signed-off-by: Daniel King <dan@spiraldb.com>
Signed-off-by: Daniel King <dan@spiraldb.com>
Signed-off-by: Daniel King <dan@spiraldb.com>
Signed-off-by: Daniel King <dan@spiraldb.com>
Signed-off-by: Daniel King <dan@spiraldb.com>
Signed-off-by: Daniel King <dan@spiraldb.com>
Signed-off-by: Daniel King <dan@spiraldb.com>
Signed-off-by: Daniel King <dan@spiraldb.com>
danking added 9 commits July 17, 2026 16:37
Signed-off-by: Daniel King <dan@spiraldb.com>
Signed-off-by: Daniel King <dan@spiraldb.com>
Signed-off-by: Daniel King <dan@spiraldb.com>
Signed-off-by: Daniel King <dan@spiraldb.com>
Signed-off-by: Daniel King <dan@spiraldb.com>
Signed-off-by: Daniel King <dan@spiraldb.com>
Signed-off-by: Daniel King <dan@spiraldb.com>
…copy' into codex/piecewise-sequence-combined-pr

Signed-off-by: Daniel King <dan@spiraldb.com>
…ts' into codex/piecewise-sequence-combined-pr

Signed-off-by: Daniel King <dan@spiraldb.com>
@codspeed-hq

codspeed-hq Bot commented Jul 17, 2026

Copy link
Copy Markdown

Merging this PR will regress 1 benchmark

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 32 improved benchmarks
❌ 1 regressed benchmark
✅ 1637 untouched benchmarks
🆕 128 new benchmarks
⏩ 53 skipped benchmarks1

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
Simulation varbinview_large 389.9 µs 446.4 µs -12.66%
Simulation struct_large 432.6 µs 38 µs ×11
Simulation fsl_large 616.5 µs 87.8 µs ×7
Simulation take_fsl_random[1024, 1000] 15.2 ms 6 ms ×2.5
Simulation take_fsl_nullable_random[1024, 1000] 14.8 ms 6 ms ×2.5
Simulation take_fsl_random[256, 1000] 3.5 ms 1.6 ms ×2.2
Simulation take_fsl_nullable_random[256, 1000] 3.4 ms 1.6 ms ×2.1
Simulation take_fsl_random[1024, 100] 1,568.1 µs 798 µs +96.51%
Simulation take_fsl_nullable_random[1024, 100] 1,535.9 µs 808.3 µs +90.02%
Simulation take_fsl_random[256, 100] 418.5 µs 237.7 µs +76.06%
Simulation take_fsl_random[64, 1000] 815.7 µs 468.1 µs +74.28%
Simulation take_fsl_nullable_random[64, 1000] 812 µs 482.8 µs +68.19%
Simulation take_fsl_nullable_random[256, 100] 415.8 µs 247.9 µs +67.73%
Simulation list_i32_large 172.3 µs 106.2 µs +62.22%
Simulation take_fsl_random[16, 1000] 237.9 µs 182.4 µs +30.43%
Simulation take_filter_list_slice_mask_random_indices[256, 50] 118.4 µs 93.2 µs +27.06%
Simulation take_filter_list_small_uncached_random_mask_random_indices[256, 10] 116.1 µs 91.6 µs +26.82%
Simulation take_filter_list_nullable_random_mask_random_indices[256, 50] 128.4 µs 101.3 µs +26.78%
Simulation take_filter_list_small_random_mask_random_indices[256, 10] 115.4 µs 91.1 µs +26.69%
Simulation take_filter_list_slice_mask_random_indices[768, 50] 118.6 µs 93.9 µs +26.3%
... ... ... ... ... ...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing codex/piecewise-sequence-combined-pr (012f18f) with develop (e6eeaae)

Open in CodSpeed

Footnotes

  1. 53 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@danking danking changed the title Combine PiecewiseSequence take stack DO NOT MERGE Combine PiecewiseSequence take stack Jul 17, 2026
@github-actions

github-actions Bot commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

Polar Signals Profiling Results

Latest Run

Status Commit Job Attempt Link
🟢 Done 012f18f 1 Explore Profiling Data

Powered by Polar Signals Cloud

@github-actions

Copy link
Copy Markdown
Contributor

Benchmarks: Vortex queries 📖

Verdict: No clear signal (medium confidence)
Attributed Vortex impact: +4.4%
Engines: DataFusion No clear signal (+3.0%, low confidence) · DuckDB No clear signal (+5.7%, low confidence)
Vortex (geomean): 0.911x ➖
Parquet (geomean): 0.892x ✅
Shifts: Parquet (control) -10.8% · Median polish -9.5%

How to read Verdict and Engines
  • Verdict: Overall PR-level signal after subtracting baseline drift estimated from Parquet control rows. It can be Likely improvement, Likely regression, or No clear signal.
  • Engines: Per-engine attribution. DataFusion is compared against DataFusion/Parquet controls; DuckDB is compared against DuckDB/Parquet controls. This answers whether each engine improved or regressed independently.
  • Confidence: Based on directional consistency, share of rows above the noise floor, and control-run noise.

datafusion / vortex-file-compressed (0.855x ✅, 2↑ 0↓)
name PR 012f18f (ns) base e6eeaae (ns) ratio (PR/base)
vortex_q00/datafusion:vortex-file-compressed 🚀 9794835 11446321 0.86
vortex_q01/datafusion:vortex-file-compressed 🚀 6368989 7449028 0.86
datafusion / parquet (0.830x ✅, 2↑ 0↓)
name PR 012f18f (ns) base e6eeaae (ns) ratio (PR/base)
vortex_q00/datafusion:parquet 🚀 19710916 23584739 0.84
vortex_q01/datafusion:parquet 🚀 4634462 5616200 0.83
duckdb / vortex-file-compressed (1.013x ➖, 0↑ 0↓)
name PR 012f18f (ns) base e6eeaae (ns) ratio (PR/base)
vortex_q00/duckdb:vortex-file-compressed 10395704 10387695 1.00
vortex_q01/duckdb:vortex-file-compressed 6039147 5885688 1.03
duckdb / parquet (0.958x ➖, 0↑ 0↓)
name PR 012f18f (ns) base e6eeaae (ns) ratio (PR/base)
vortex_q00/duckdb:parquet 23273908 24576114 0.95
vortex_q01/duckdb:parquet 9432237 9723548 0.97

No file size changes detected.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant