Skip to content

fix(deps): pin lancedb to >=0.34.0,<0.35.0#355

Open
gloryfromca wants to merge 1 commit into
mainfrom
fix/pin-lancedb-upper-bound
Open

fix(deps): pin lancedb to >=0.34.0,<0.35.0#355
gloryfromca wants to merge 1 commit into
mainfrom
fix/pin-lancedb-upper-bound

Conversation

@gloryfromca

Copy link
Copy Markdown
Collaborator

Summary

Tightens the lancedb dependency from the open-ended >=0.13.0 to
>=0.34.0,<0.35.0.

The unbounded floor let any environment resolve to an untested lancedb
release. Versions 0.32.0 → 0.34.0 embed lance-rust v6/v7/v8, which
carry a compaction offset-overflow regression
(lance-format/lance#7653):
optimize() crashes while merging an unindexed tail of a with_position
FTS index, which stalls version cleanup and grows the index directory
without bound (observed 344 GB from ~24k rows on a user deployment).

  • Floor 0.34.0 — the currently-resolved version. We run it safely
    via the with_position=False FTS workaround shipped in fix(lancedb): disable FTS with_position to avoid optimize crash #336. Pinning
    the floor here prevents silently dropping back to a version without
    that mitigation.
  • Ceiling <0.35.0 — 0.35 embeds lance-rust v9 (the branch that
    actually fixes #7653 upstream via lance#7546), but v9 is a large
    encoding jump and has no stable release yet (0.35.0b2 is GitHub-release
    only, not on PyPI). We gate the upgrade behind a soak-test harness
    rather than floating onto it blindly.

Resolved version is unchanged (still 0.34.0); this only tightens the
declared constraint and its uv.lock specifier.

Area

  • Architecture method
  • Benchmark
  • Use case
  • Documentation
  • Developer experience
  • CI, build, or release

Verification

# Constraint resolves + installs cleanly, resolved version unchanged
$ uv lock            # lancedb specifier -> >=0.34.0,<0.35.0 ; resolved 0.34.0
$ uv sync            # OK
$ python -c "import everos, lancedb; print(lancedb.__version__)"  # everos OK; 0.34.0
$ make lint          # import-linter contracts KEPT; datetime clean; openapi matches

# Cross-version read compatibility (upgrade direction is safe: new reader / old data)
# wrote 3000 rows (1024-dim vector + with_position=True FTS) under lancedb 0.32.0 (lance v6),
# then opened + counted + vector-searched + FTS-searched under 0.34.0 (lance v8):
WRITE lancedb 0.32.0  rows 3000
READ  lancedb 0.34.0  count 3000
vector-search ok, got 3
fts-search   ok, got 3
# => existing 0.32/0.33 deployments upgrade to 0.34.0 without a rebuild.

Checklist

  • I kept the change scoped to the relevant area.
  • I am opening this from a separate branch, not pushing directly to main.
  • I updated docs, examples, or setup notes when behavior changed. (No runtime behavior change; the rationale is documented inline in pyproject.toml.)
  • I added or updated tests when the change affects behavior. (Metadata-only change, no runtime behavior; cross-version compatibility verified manually — see Verification.)
  • I did not commit secrets, .env files, dependency folders, or generated output.
  • Active relative links in Markdown files resolve.

Notes for Reviewers

  • The <0.35.0 ceiling is deliberate and should not be relaxed until the
    soak harness validates 0.35.x. The inline comment in pyproject.toml
    records the rationale so a future dependency bump doesn't silently undo it.
  • Do not widen the floor back below 0.34: older lance (v4, ≤0.30.2)
    cannot read v8-format data written by 0.34, so a downgrade would break
    existing indexes (rebuild-from-md required). The upgrade direction is the
    safe one, as verified above.

By submitting this pull request, I agree that my contribution is licensed under
the Apache License 2.0.

The previous open-ended `>=0.13.0` let any environment float to an
untested release, including 0.32-0.34 which carry a compaction
offset-overflow regression (lance-format/lance#7653) that stalls
version cleanup and grows the index dir without bound.

- Floor 0.34.0: the current resolved version; runs safely thanks to
  the with_position=False FTS workaround shipped in #336. Verified that
  data written by lancedb 0.32.0 (lance v6) reads correctly under
  0.34.0 (lance v8), so existing deployments upgrade cleanly. Never
  widen the floor below 0.34 -- older lance cannot read v8-format data.
- Ceiling <0.35.0: 0.35 embeds lance-rust v9 (large encoding jump, not
  yet stable-released); it must pass the soak harness before we allow
  it.

Resolved version is unchanged (still 0.34.0); this only tightens the
declared constraint.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant