feat(vortex-geo): add MultiPolygon extension type + WKB export for native geometry#8608
Merged
Conversation
Merging this PR will not alter performance
|
| Mode | Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|---|
| ❌ | Simulation | chunked_bool_canonical_into[(1000, 10)] |
22.6 µs | 26.7 µs | -15.64% |
| ⚡ | WallTime | cuda/bitpacked_u8/unpack/3bw[100M] |
350.8 µs | 300.6 µs | +16.67% |
| ⚡ | Simulation | bitwise_not_vortex_buffer_mut[128] |
273.6 ns | 244.4 ns | +11.93% |
Tip
Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.
Comparing nemo/geo-multipolygin (195833e) with develop (a21bf76)
Footnotes
-
4 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩
Signed-off-by: Nemo Yu <zyu379@wisc.edu>
HarukiMoriarty
force-pushed
the
nemo/spatial-wire-vx-bench
branch
from
fdb0872 to
b362551
Compare
DuckDB's GEOMETRY only accepts little-endian (NDR) WKB, but the externally sourced zone table (Overture Maps via spatialbench-cli) is big-endian, so the vortex lane failed spatial queries with "Only little-endian WKB is supported". Re-encode geometry columns to little-endian during the parquet->vortex conversion so the vortex file stores canonical little-endian WKB; columns that are already little-endian pass through without a copy. Also drop the best-effort warn around zone generation so data-generation failures propagate. Signed-off-by: Nemo Yu <zyu379@wisc.edu>
The zone table's data generation hand-rolled idempotency: an existence check plus a scratch dir cleared with remove_dir_all to survive interrupted runs. Reuse the shared idempotent_async helper instead, which skips when zone_0.parquet exists and writes via a unique temp path + atomic rename, so interrupted runs leave nothing to clean up. Zone is a single part (the CLI runs without --parts), so it is handled as one file. Signed-off-by: Nemo Yu <zyu379@wisc.edu>
Signed-off-by: Nemo Yu <zyu379@wisc.edu>
Signed-off-by: Nemo Yu <zyu379@wisc.edu>
HarukiMoriarty
force-pushed
the
nemo/geo-multipolygin
branch
from
a3c046b to
790a0d7
Compare
`ExtVTable::id` interned the id with `ExtId::new_static` on every call, tripping the `disallowed_methods` clippy lint added on the base branch (#8617) since it grabs the interner lock per call. Match the `Point` and `Polygon` types by caching the id in a `static CachedId`. Signed-off-by: Nemo Yu <zyu379@wisc.edu>
The `to_wkb` methods on `PointData`, `PolygonData`, and `MultiPolygonData` were byte-identical except for the geometry array type. Hoist the WKB cast into a single `geoarrow_to_wkb(&dyn GeoArrowArray)` in the extension module so each `to_wkb` is a one-liner, drop the redundant `storage_array().clone()` (the `*_array` helpers take `&ArrayRef`), and remove the now-unused `GeoArrowType`/`WkbType`/`cast`/`GeoArrowArray` imports from the three types. Signed-off-by: Nemo Yu <zyu379@wisc.edu>
danking
approved these changes
Jul 1, 2026
danking
left a comment
danking
left a comment
Contributor
There was a problem hiding this comment.
Seems broadly correct to me.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a native
MultiPolygongeometry extension type tovortex-geo, and a WKB-export helper (to_wkb) for the native geometry types (Point,Polygon,MultiPolygon).MultiPolygon(vortex.geo.multipolygon) mirrors the existingPolygontype, mainly for SpatialBenchz_boundarycolumn, which is mixture ofPolygonandMultiPolygon.to_wkbserializes a native geometry array to WKB (viageoarrow-cast), the form DuckDB'sGEOMETRYvectors carry, for communicate with DuckDB.What APIs are changed? Are there any user-facing changes?
Purely additive, new public API in
vortex-geo:extension::MultiPolygon, the new extension type (with itsExtVTable+ Arrow import/export).extension::{PointData, PolygonData}+to_wkbon the existingPoint/Polygontypes.