[core] Add primary-key vector bucket search kernel

[JunRuiLee]

Purpose

Linked issue: part of #514 — PR 2 of 5 (does not close the issue)

This is the second, read-only slice of primary-key vector (bucket-local ANN) search support. It mirrors Apache Paimon (Java) apache/paimon#8569 as of commit a50a36ff8. It builds on the _SOURCE_META parsing merged in #515.

It ports the bucket-local search kernel: given one snapshot bucket's ANN segments, active data files, and deletion vectors, merge ANN hits with an exact fallback over the files not covered by any ANN segment into a deterministic bounded Top-K. This is a pure, synthetically-tested kernel — the read-path wiring and real ANN index-byte validation come in PR 3 / PR 4.

Read-only. Out of scope for this PR (tracked in #514): the physical-position single-file reader (PR 3a), cross-bucket Top-K merge + physical-row materialization (PR 3b), snapshot-consistent planning and end-to-end validation against Java-written fixtures (PR 4), index building, and engine SQL entry points.

Brief change log

New crate-private module crates/paimon/src/vindex/pkvector/:

• metric.rs — VectorSearchMetric (l2 / cosine / inner_product): normalize/is_supported (lowercase + -→_, no trim, matching Java), parse-once enum, compute_score (higher-better), compute_distance (lower-better, squared-L2 with no sqrt, cosine denominator widened to f64 like Java, zero-norm→0), and score_to_distance (l2 1/score-1→inf at 0, no clamp).
• result.rs — PkVectorSearchResult { data_file_name, row_position, distance }.
• reader.rs — PkVectorReader sequential-scan trait (null rows return false but still advance the physical position); an in-#[cfg(test)] ArrayReader fake.
• exact.rs — bounded-memory exact Top-K over one file (BEST_FIRST = distance ASC, then row_position ASC; total_cmp for NaN-safety), with dimension/limit/finite/row-count validation and a deletion-vector exclusion predicate.
• ann.rs — PkVectorAnnSearcher trait (bucket search's ANN dependency, faked in tests) + a structural VindexAnnSearcher composed around an injected scorer seam; plus the pure helpers build_live_row_ids (live-row mask in segment-ordinal space, offsets guarded by checked_add) and map_ann_results (ordinal→(file, position) via [core] Parse primary-key vector index source metadata (_SOURCE_META) #515's PkVectorSourceMeta::resolve, rejects hits on snapshot-deleted rows, score_to_distance, BEST_FIRST sort).
• bucket.rs — bucket_search: validates ANN source metadata against active files, skips ANN-covered files in the exact fallback (no rescanning), forwards deletion vectors and search options, and deterministically merges ANN + exact candidates (3-key BEST_FIRST across files).

Plus a small cross-module accessor: DeletionVector::is_deleted(position) -> bool (guards > u32::MAX → not deleted).

The module is crate-private and guarded by a temporary module-level #![allow(dead_code)]: PR 2 ports the kernel ahead of the read-path wiring, so it has no production caller yet. The allow is removed once PR 3 / PR 4 lands callers.

Tests

All synthetic unit tests inside the module (30 tests; full cargo test -p paimon green; cargo fmt --all --check and cargo clippy -p paimon --all-targets -- -D warnings both clean):

• metric: three-metric distance/score anchors (matching Java PkVectorExactSearcherTest), normalize no-trim, cosine zero-norm→0, cosine f64-sqrt precision (pinned with an irrational-norm case), score_to_distance l2→inf.
• exact: three-metric distances, dimension/limit/finite/row-count<0 rejection, null + excluded physical-position preservation (Java parity), tie-break by smaller row_position, limit truncation.
• ann: build_live_row_ids None-when-no-relevant-DV + multi-file-offset masking; map_ann_results ordinal→position mapping + score, out-of-range ordinal rejection, ANN-hit-on-deleted rejection; adapter composition through the scorer seam.
• bucket: ANN + exact merge with covered files never re-read (asserted via a recording factory), pure exact fallback with deletion-vector exclusion, non-positive limit / duplicate file name / ANN-source row-count mismatch / segments-without-searcher rejection, cross-file BEST_FIRST tie-break.
• deletion vector: is_deleted membership + > u32::MAX guard.

Real ANN vector-search correctness (driving the vindex reader with genuine index bytes) is intentionally not exercised here — the vindex crate is read-only, so index bytes cannot be synthesized in-process. It is deferred to PR 4's Java-written fixtures.

API and Format

• No storage-format change: this consumes the _SOURCE_META format defined by Java [core] Add primary-key vector index foundation paimon#8549 (parsed in [core] Parse primary-key vector index source metadata (_SOURCE_META) #515) and reproduces the search semantics of [core] Add primary-key vector bucket search paimon#8569.
• All new pkvector items are crate-private (pub(crate)); the one cross-module accessor DeletionVector::is_deleted is pub, matching that type's existing public accessors (cardinality, is_empty, …). No existing API changed.

Documentation

No user-facing documentation changes.

[JingsongLi]

1. When some source files become inactive, the behavior is inconsistent with the current Java implementation, which constitutes a correctness/interop blocker.

bucket.rs:129-135 requires that all sources in an ANN segment remain active files; if even one file becomes inactive, an error is immediately reported. The current Java master branch skips inactive source files, searches only the still-active portions, and masks the ordinal ranges corresponding to inactive source files in the ANN live-row bitmap.

For example, if a segment covers [A, B] and only A is currently active, Rust would cause the entire query to fail, while Java would still correctly search A. We need to pass the set of active files to the ANN search, mask the ranges of inactive sources, and add corresponding regression tests. The existing test_rejects_ann_source_missing_or_mismatched_active_file actually solidifies the old behavior.

2. The complexity of merging Bucket Top-K is too high.

bucket.rs:57-72: Each time a candidate is added, the current Top-K is linearly scanned using Vec::max_by. The exact fallback returns at most K entries per uncovered file, resulting in a time complexity of approximately O(files × K²) in scenarios with multiple files; CPU amplification is significant when K is large. We recommend replacing this with a BinaryHeap that maintains a stable sort by distance, file, and row position, reducing the complexity to O(candidate_count log K).

[JunRuiLee]

Thanks for the review, both points are addressed in the latest commit.

1. Inactive source files. You're right — this was an interop gap. The PR originally mirrored the read kernel at a50a36ff8, which still rejected any ANN source that wasn't an active file; current Java master (PrimaryKeyVectorBucketSearch / PkVectorAnnSegmentSearcher) instead skips inactive sources and searches only the still-active portions. I've adopted the master behavior:

• build_live_row_ids now takes the active-source set and masks out the ordinal ranges of inactive sources (they contribute no live ids; deletion vectors are applied only to active sources). It returns None only when all sources are active and no deletion vector is relevant.
• bucket_search skips ANN sources missing from the active set; an active source with a mismatched row count is still a hard error.
• map_ann_results additionally rejects a hit that resolves to an inactive source, mirroring the Java checkArgument (defensive — the live-row mask should already exclude it).
• The active-source set is threaded through PkVectorAnnSearcher::search and VindexAnnSearcher.

I replaced test_rejects_ann_source_missing_or_mismatched_active_file (which solidified the old reject) with a row-count-mismatch reject test plus skip-inactive coverage, and added tests for inactive ordinal-range masking and inactive-hit rejection.

2. Bucket Top-K complexity. Replaced the Vec + max_by linear scan with a BinaryHeap<WorstFirst> keeping the BEST_FIRST-worst on top, so the merge is O(candidate_count log K), matching the Java PriorityQueue and the existing exact.rs heap. Added a bounded-heap tie-break test (equal distances across files, eviction ordered by (data_file_name, row_position)).

I've kept the larger current-master features (GlobalIndexSearchMode / FAST mode, the indexed/exact Result split) out of this PR to keep it scoped to these two comments; they'll come in a later PR.

[JingsongLi]

+1