Skip to content

Use PiecewiseSequence for List take elements#8833

Draft
danking wants to merge 4 commits into
codex/piecewise-sequential-run-take-consumersfrom
codex/list-take-piecewise-elements
Draft

Use PiecewiseSequence for List take elements#8833
danking wants to merge 4 commits into
codex/piecewise-sequential-run-take-consumersfrom
codex/list-take-piecewise-elements

Conversation

@danking

@danking danking commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

This is just a simplification. List::take already had a specialized implementation. The new PiecewiseSequence specializations in various primitive arrays supersede this change.

EDIT: Heh, I guess the PiecewiseSequence stuff is actually even faster than old List::take. Nice.


codex:

Summary

  • replace List take's dense per-element index expansion with PiecewiseSequence element runs
  • preserve FSL-style null handling: null index/source rows emit zero-length placeholder runs while output validity still comes from Validity::take
  • let the elements child choose its own optimized take implementation through the normal ArrayRef::take path

Validation

  • cargo +nightly fmt --all
  • cargo test -p vortex-array arrays::list::compute::take
  • cargo check -p vortex-array
  • cargo clippy -p vortex-array --all-targets --all-features -- -D warnings
  • git diff --check

@codspeed-hq

codspeed-hq Bot commented Jul 17, 2026

Copy link
Copy Markdown

Merging this PR will improve performance by 27.05%

⚡ 12 improved benchmarks
✅ 1786 untouched benchmarks
⏩ 52 skipped benchmarks1

Performance Changes

Mode Benchmark BASE HEAD Efficiency
Simulation copy_nullable[65536] 1.4 ms 1 ms +32.01%
Simulation take_filter_list_slice_mask_random_indices[256, 50] 119.2 µs 92.6 µs +28.69%
Simulation take_filter_list_slice_mask_random_indices[768, 50] 119.6 µs 93.4 µs +28.05%
Simulation take_filter_list_small_random_mask_random_indices[256, 10] 116.2 µs 91 µs +27.69%
Simulation take_filter_list_small_uncached_random_mask_random_indices[256, 10] 116.9 µs 91.6 µs +27.6%
Simulation take_filter_list_nullable_random_mask_random_indices[256, 50] 128.5 µs 100.7 µs +27.56%
Simulation take_filter_list_random_mask_random_indices[256, 50] 122.6 µs 96.5 µs +27.07%
Simulation take_filter_list_nullable_random_mask_random_indices[768, 50] 132.6 µs 104.6 µs +26.73%
Simulation take_filter_list_small_random_mask_random_indices[768, 10] 120.3 µs 95 µs +26.6%
Simulation take_filter_list_small_uncached_random_mask_random_indices[768, 10] 120.1 µs 94.9 µs +26.55%
Simulation take_filter_list_random_mask_random_indices[768, 50] 127 µs 100.4 µs +26.51%
Simulation copy_non_nullable[65536] 1,089.5 µs 908.8 µs +19.89%

Tip

Curious why this is faster? Comment @codspeedbot explain why this is faster on this PR, or directly use the CodSpeed MCP with your agent.


Comparing codex/list-take-piecewise-elements (243ed19) with codex/piecewise-sequential-run-take-consumers (e4a0543)

Open in CodSpeed

Footnotes

  1. 52 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@danking
danking force-pushed the codex/piecewise-sequential-run-take-consumers branch from c4a0358 to c76531d Compare July 17, 2026 20:05
@danking
danking force-pushed the codex/list-take-piecewise-elements branch 2 times, most recently from 77d37d8 to b7ad4df Compare July 17, 2026 20:15
@danking danking added the changelog/performance A performance improvement label Jul 17, 2026
@danking
danking force-pushed the codex/piecewise-sequential-run-take-consumers branch from 80f7b89 to 30870aa Compare July 17, 2026 20:19
@danking
danking force-pushed the codex/list-take-piecewise-elements branch from b7ad4df to f0eec2d Compare July 17, 2026 20:19
@danking
danking force-pushed the codex/piecewise-sequential-run-take-consumers branch from 30870aa to e777ab4 Compare July 17, 2026 20:25
@danking
danking force-pushed the codex/list-take-piecewise-elements branch from f0eec2d to d40cee8 Compare July 17, 2026 20:25
@danking
danking force-pushed the codex/piecewise-sequential-run-take-consumers branch from e777ab4 to e4a0543 Compare July 17, 2026 20:37
danking added 3 commits July 17, 2026 16:39
Signed-off-by: Daniel King <dan@spiraldb.com>
Signed-off-by: Daniel King <dan@spiraldb.com>
Signed-off-by: Daniel King <dan@spiraldb.com>
@danking
danking force-pushed the codex/list-take-piecewise-elements branch from d40cee8 to e781b61 Compare July 17, 2026 20:40
Signed-off-by: Daniel King <dan@spiraldb.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/performance A performance improvement

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant