Skip to content

test: useLiveQuery conformance suite across all five framework adapters (RFC #1623)#1636

Merged
kevin-dp merged 15 commits into
mainfrom
test/live-query-conformance-suite
Jul 7, 2026
Merged

test: useLiveQuery conformance suite across all five framework adapters (RFC #1623)#1636
kevin-dp merged 15 commits into
mainfrom
test/live-query-conformance-suite

Conversation

@kevin-dp

@kevin-dp kevin-dp commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Draft — first slice of RFC #1623 (framework-binding platform). Discussion: #1623

Scope — this is test infrastructure only

This PR is entirely test-only — no src/ changes, no shared production code. It does not extract the duplicated adapter logic into a shared observer/module; that's the next milestone (RFC step 2+). This PR builds the safety net that must exist before that extraction, so we can rip shared logic out of five adapters and prove behavior is unchanged.

The per-framework files here (packages/*/tests/conformance.test.*) are small test harnesses, not new production abstractions. Each just wraps its framework's existing useLiveQuery (mount → read → flush → unmount) so one shared spec can run against all five. (In the code the harness interface is named LiveQueryDriver — "driver" in the JUnit/conformance-testing sense of "the thing that drives the tests," not a runtime adapter.)

What this is

A shared, framework-agnostic conformance suite for the useLiveQuery adapters. One behavioral spec (packages/db/tests/conformance/suite.ts, 24 scenarios) runs against all five adapters through a thin per-framework test harness implementing a common contract (packages/db/tests/conformance/contract.ts).

This is deliberately the lowest-risk, highest-consensus piece of #1623: it documents current behavior, commits to no architecture, and becomes the safety net for the later observer/adapter refactor.

How it's sourced

Bottom-up from the union of the existing adapter test suites — every fixed bug already left a regression test behind, so those tests are the real contract. The diff between adapters' suites is the drift #1623 is about, so folding them into one spec surfaces it. The RFC's aspirational list contributes only the small additive tail (the #1601 order-only-move case, a universal expected-fail).

Design notes

  • Realm-safe injection. The per-framework harness — not the scenarios — creates collections and supplies query operators, both from the adapter's @tanstack/db. Scenarios never import @tanstack/db, avoiding the dual-package instanceof CollectionImpl mismatch.
  • Per-adapter knownGaps, populated empirically. Each scenario runs as it or it.fails per adapter based on knownGaps. A key is added only when it actually fails on a run — the matrix says where to look, the run says what's broken. When a gap closes, it.fails errors ("expected to fail but passed") prompting removal. So drift and bugs are documented without the suite going red, and the knownGaps lists double as the follow-up to-do list.
  • apply(fn) handle primitive. A cross-framework "run a mutation inside the framework's update scope, then settle" (React act / Vue nextTick / Svelte flushSync / Solid batch) — needed for the optimistic-mutation scenario.

Coverage (24 scenarios)

Query/where/select, live insert/update/delete, orderBy, join, groupBy/aggregate, nested aggregates, .includes subqueries, findOne cardinality (+ reactive update/delete), disabled + enable/disable transitions, deferred readiness / eager-visible-while-loading / ready-with-no-data, param recompilation, optimistic mutation, pre-created & config-object inputs, error status, and the #1601 order-only-move tail.

Results — all five adapters covered

Each passes 24/24 (documented knownGaps run as expected-fail). The suite surfaced three real cross-adapter divergences/bugs:

Adapter knownGaps Finding Fix
React reference harness, clean
Vue clean — and closes Vue's findOne coverage gap (vue-db had zero findOne tests; it works, was just untested)
Svelte disabled-explicit, disabled-transition Bug: toValue() (added for the reactive () => collection form) calls a () => null disabled query as a getter, unwraps it to null, and passes {...null} into createLiveQueryCollection → crash in getQueryIR. The disabled short-circuit is unreachable; svelte-db has no disabled tests. #1637
Solid error-status Divergence: errors are surfaced by throwing (CollectionStateError) through the createResource/Suspense path for an <ErrorBoundary>, not via a readable isError flag like the others. #1639
Angular config-object-input Divergence: the plain { query } config path calls createLiveQueryCollection(opts) without injecting startSync: true (its query-fn path does), so a bare config never syncs. The others auto-start; Angular requires explicit startSync: true. #1638

Also confirmed not drift: every adapter eagerly starts a pre-created collection on mount (an earlier scenario wrongly tested this as React-specific; corrected to build over a not-ready source so it measures the actual contract — isReady stays false only when the source never readies).

Out of scope for this suite (by design)

Framework-specific tests (Solid proxy identity, Svelte runes, React overloads, Angular DI lifecycle) stay per-adapter. Separate hook families (useLiveInfiniteQuery, useLiveSuspenseQuery, useLiveQueryEffect) get their own suites later.

Follow-up PRs (stacked on this branch)

Each clears one knownGap and is stacked on this branch (re-target to main once this lands):

Each removes its knownGap entry, so the previously-it.fails scenario runs as a normal passing test — the suite itself proves the fix.

Next milestone (NOT this PR)

The actual duplication cleanup — extracting the ~7 items each adapter copies (input normalization, disabled handling, cardinality, status derivation, ready-race, etc.) into a shared observer/module in @tanstack/db, then slimming each adapter's src/useLiveQuery down to it — is a separate follow-up. That's the PR that will show src/ changes and shrink duplication. This suite exists to keep it honest: it must stay green through that refactor.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features
    • Added a shared live-query conformance test suite.
    • Added adapter drivers for Angular, React, Solid, Svelte, and Vue so the same query scenarios run consistently across frameworks.
    • Expanded coverage for filtering, inserts/updates/deletes, ordering, joins, grouping, deferred loading, disabled queries, errors, and optimistic updates.
  • Bug Fixes
    • Added expected-fail tracking to keep known framework differences from breaking the suite.
    • Improved cleanup so mounted queries are torn down after each scenario.

kevin-dp and others added 2 commits July 1, 2026 13:32
Introduce a shared, framework-agnostic conformance harness for the
`useLiveQuery` adapters (RFC #1623, Phase 1). Each adapter supplies a thin
driver that implements a common contract; the shared suite runs one
behavioral spec against all of them.

- contract.ts: the LiveQueryDriver contract (realm-safe injection of
  collection factories + query operators, so scenarios never import
  @tanstack/db directly and avoid the dual-package instanceof mismatch).
- suite.ts: 24 scenarios sourced bottom-up from the union of the existing
  adapter test suites (spine + gap-closers), plus the #1601 order-only-move
  case as a universal expected-fail. Per-adapter knownGaps mark behaviors an
  adapter does not yet satisfy (populated empirically, not from the matrix).
- react-db: React reference driver. Its only knownGap beyond the universal
  #1601 is precreated-not-syncing-isready-false — React eagerly starts sync
  on mount, real cross-adapter drift the suite surfaced.

Covers query/liveness, join/groupBy/aggregate/includes, findOne cardinality,
disabled + transitions, deferred readiness/eager, param recompile, optimistic
mutation, pre-created/config-object inputs, and error status.

Vue/Svelte/Solid/Angular drivers to follow, one per PR.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Wire the vue-db adapter into the shared live-query conformance suite. Vue
composables run inside an effectScope so unmount disposes via scope.stop().

Vue passes all 24 scenarios (knownGaps empty) — including findOne cardinality,
which vue-db had zero tests for. The suite closes that coverage gap.

Also correct the `precreated-not-syncing-isready-false` scenario: it now builds
the live query over a not-ready (deferred) source rather than a ready one. The
original wrongly asserted an eager-start implementation detail — both React and
Vue eagerly start a pre-created collection on mount, so isReady is only false
when the source itself never readies. React's spurious knownGap is dropped.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

Introduces a shared cross-framework live-query conformance test contract and suite (packages/db/tests/conformance/contract.ts, suite.ts) defining common types and scenarios, along with six per-framework driver test files (Angular, React, Solid, Svelte, Vue) that implement the driver interface and register with runSuite to validate live-query behavior consistently across adapters.

Changes

Shared Conformance Contract and Suite

Layer / File(s) Summary
Driver contract types
packages/db/tests/conformance/contract.ts
Defines Row, SourceHandle, DeferredSourceHandle, DbOps, ConformanceResult, QueryBuild, LiveQueryHandle, ControllableHandle, and LiveQueryDriver interfaces shared by all adapters.
Suite orchestration and fixtures
packages/db/tests/conformance/suite.ts
runSuite tracks mounted handles, derives expected-fail keys from knownGaps/UNIVERSAL_EXPECTED_FAIL, and defines seed fixtures.
Conformance scenarios
packages/db/tests/conformance/suite.ts
Adds scenarios for basic queries, live mutations, ordering, findOne, disabled/unmount behavior, joins, aggregates, subqueries, deferred readiness, controllable params, optimistic mutations, precreated collections, config-object input, error status, and a meta knownGaps validation test.

Framework Driver Implementations

Layer / File(s) Summary
React conformance driver
packages/react-db/tests/conformance.test.tsx
Implements source factories, useLiveQuery mount helpers, a handle adapter, and reactDriver registration.
Angular conformance driver
packages/angular-db/tests/conformance.test.ts
Implements source factories, injectLiveQuery mounting via EnvironmentInjector, a handle adapter, and angularDriver registration.
Vue conformance driver
packages/vue-db/tests/conformance.test.ts
Implements source factories, effectScope-based mounting, a handle adapter, and vueDriver registration.
Svelte conformance driver
packages/svelte-db/tests/conformance.svelte.test.ts
Implements source factories, $effect.root-based mounting, a handle adapter with flushSync settling, and svelteDriver registration.
Solid conformance driver
packages/solid-db/tests/conformance.test.tsx
Implements source factories, signal-based controllable mounting, a handle adapter, and solidDriver registration.

Estimated code review effort: 3 (Moderate) | ~30 minutes

Sequence Diagram(s)

sequenceDiagram
  participant TestRunner
  participant runSuite
  participant FrameworkDriver
  participant LiveQueryHandle
  TestRunner->>runSuite: register scenario
  runSuite->>runSuite: resolve expected-fail via knownGaps/UNIVERSAL_EXPECTED_FAIL
  runSuite->>FrameworkDriver: mount/mountDeferred/mountControllable(...)
  FrameworkDriver->>LiveQueryHandle: return handle
  runSuite->>LiveQueryHandle: current()
  runSuite->>LiveQueryHandle: flush()/apply()
  runSuite->>LiveQueryHandle: unmount()
Loading

Suggested reviewers: samwillis

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Description check ⚠️ Warning The description is detailed, but it doesn't follow the required template sections and omits the checklist and release-impact items. Rewrite it using the repo template sections: Changes, Checklist, and Release Impact, and fill in the required testing/release status items.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly states the new shared useLiveQuery conformance suite across framework adapters.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch test/live-query-conformance-suite

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@pkg-pr-new

pkg-pr-new Bot commented Jul 2, 2026

Copy link
Copy Markdown
More templates

@tanstack/angular-db

npm i https://pkg.pr.new/@tanstack/angular-db@1636

@tanstack/browser-db-sqlite-persistence

npm i https://pkg.pr.new/@tanstack/browser-db-sqlite-persistence@1636

@tanstack/capacitor-db-sqlite-persistence

npm i https://pkg.pr.new/@tanstack/capacitor-db-sqlite-persistence@1636

@tanstack/cloudflare-durable-objects-db-sqlite-persistence

npm i https://pkg.pr.new/@tanstack/cloudflare-durable-objects-db-sqlite-persistence@1636

@tanstack/db

npm i https://pkg.pr.new/@tanstack/db@1636

@tanstack/db-ivm

npm i https://pkg.pr.new/@tanstack/db-ivm@1636

@tanstack/db-sqlite-persistence-core

npm i https://pkg.pr.new/@tanstack/db-sqlite-persistence-core@1636

@tanstack/electric-db-collection

npm i https://pkg.pr.new/@tanstack/electric-db-collection@1636

@tanstack/electron-db-sqlite-persistence

npm i https://pkg.pr.new/@tanstack/electron-db-sqlite-persistence@1636

@tanstack/expo-db-sqlite-persistence

npm i https://pkg.pr.new/@tanstack/expo-db-sqlite-persistence@1636

@tanstack/node-db-sqlite-persistence

npm i https://pkg.pr.new/@tanstack/node-db-sqlite-persistence@1636

@tanstack/offline-transactions

npm i https://pkg.pr.new/@tanstack/offline-transactions@1636

@tanstack/powersync-db-collection

npm i https://pkg.pr.new/@tanstack/powersync-db-collection@1636

@tanstack/query-db-collection

npm i https://pkg.pr.new/@tanstack/query-db-collection@1636

@tanstack/react-db

npm i https://pkg.pr.new/@tanstack/react-db@1636

@tanstack/react-native-db-sqlite-persistence

npm i https://pkg.pr.new/@tanstack/react-native-db-sqlite-persistence@1636

@tanstack/rxdb-db-collection

npm i https://pkg.pr.new/@tanstack/rxdb-db-collection@1636

@tanstack/solid-db

npm i https://pkg.pr.new/@tanstack/solid-db@1636

@tanstack/svelte-db

npm i https://pkg.pr.new/@tanstack/svelte-db@1636

@tanstack/tauri-db-sqlite-persistence

npm i https://pkg.pr.new/@tanstack/tauri-db-sqlite-persistence@1636

@tanstack/trailbase-db-collection

npm i https://pkg.pr.new/@tanstack/trailbase-db-collection@1636

@tanstack/vue-db

npm i https://pkg.pr.new/@tanstack/vue-db@1636

commit: 3f6a9e5

@github-actions

github-actions Bot commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Size Change: 0 B

Total Size: 125 kB

ℹ️ View Unchanged
Filename Size
packages/db/dist/esm/collection/change-events.js 1.43 kB
packages/db/dist/esm/collection/changes.js 1.38 kB
packages/db/dist/esm/collection/cleanup-queue.js 810 B
packages/db/dist/esm/collection/events.js 434 B
packages/db/dist/esm/collection/index.js 3.62 kB
packages/db/dist/esm/collection/indexes.js 1.99 kB
packages/db/dist/esm/collection/lifecycle.js 1.69 kB
packages/db/dist/esm/collection/mutations.js 2.47 kB
packages/db/dist/esm/collection/state.js 5.4 kB
packages/db/dist/esm/collection/subscription.js 3.74 kB
packages/db/dist/esm/collection/sync.js 2.88 kB
packages/db/dist/esm/collection/transaction-metadata.js 144 B
packages/db/dist/esm/deferred.js 207 B
packages/db/dist/esm/errors.js 5.1 kB
packages/db/dist/esm/event-emitter.js 748 B
packages/db/dist/esm/index.js 3.1 kB
packages/db/dist/esm/indexes/auto-index.js 829 B
packages/db/dist/esm/indexes/base-index.js 767 B
packages/db/dist/esm/indexes/basic-index.js 2.06 kB
packages/db/dist/esm/indexes/btree-index.js 2.19 kB
packages/db/dist/esm/indexes/index-registry.js 820 B
packages/db/dist/esm/indexes/reverse-index.js 557 B
packages/db/dist/esm/local-only.js 916 B
packages/db/dist/esm/local-storage.js 2.12 kB
packages/db/dist/esm/optimistic-action.js 359 B
packages/db/dist/esm/paced-mutations.js 496 B
packages/db/dist/esm/proxy.js 3.75 kB
packages/db/dist/esm/query/builder/functions.js 1.47 kB
packages/db/dist/esm/query/builder/index.js 5.84 kB
packages/db/dist/esm/query/builder/ref-proxy.js 1.24 kB
packages/db/dist/esm/query/compiler/evaluators.js 1.89 kB
packages/db/dist/esm/query/compiler/expressions.js 430 B
packages/db/dist/esm/query/compiler/group-by.js 3.56 kB
packages/db/dist/esm/query/compiler/index.js 6.67 kB
packages/db/dist/esm/query/compiler/joins.js 2.5 kB
packages/db/dist/esm/query/compiler/lazy-targets.js 923 B
packages/db/dist/esm/query/compiler/order-by.js 1.74 kB
packages/db/dist/esm/query/compiler/select.js 1.53 kB
packages/db/dist/esm/query/effect.js 4.77 kB
packages/db/dist/esm/query/expression-helpers.js 1.43 kB
packages/db/dist/esm/query/ir.js 1.25 kB
packages/db/dist/esm/query/live-query-collection.js 360 B
packages/db/dist/esm/query/live/collection-config-builder.js 9.1 kB
packages/db/dist/esm/query/live/collection-registry.js 264 B
packages/db/dist/esm/query/live/collection-subscriber.js 1.93 kB
packages/db/dist/esm/query/live/internal.js 145 B
packages/db/dist/esm/query/live/utils.js 1.81 kB
packages/db/dist/esm/query/optimizer.js 2.92 kB
packages/db/dist/esm/query/predicate-utils.js 2.97 kB
packages/db/dist/esm/query/query-once.js 359 B
packages/db/dist/esm/query/subset-dedupe.js 960 B
packages/db/dist/esm/scheduler.js 1.3 kB
packages/db/dist/esm/SortedMap.js 1.3 kB
packages/db/dist/esm/strategies/debounceStrategy.js 247 B
packages/db/dist/esm/strategies/queueStrategy.js 428 B
packages/db/dist/esm/strategies/throttleStrategy.js 246 B
packages/db/dist/esm/transactions.js 3.04 kB
packages/db/dist/esm/utils.js 927 B
packages/db/dist/esm/utils/array-utils.js 273 B
packages/db/dist/esm/utils/browser-polyfills.js 304 B
packages/db/dist/esm/utils/btree.js 5.61 kB
packages/db/dist/esm/utils/comparison.js 1.11 kB
packages/db/dist/esm/utils/cursor.js 457 B
packages/db/dist/esm/utils/index-optimization.js 2.39 kB
packages/db/dist/esm/utils/type-guards.js 157 B
packages/db/dist/esm/utils/uuid.js 449 B
packages/db/dist/esm/virtual-props.js 360 B

compressed-size-action::db-package-size

@github-actions

github-actions Bot commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Size Change: 0 B

Total Size: 4.26 kB

ℹ️ View Unchanged
Filename Size
packages/react-db/dist/esm/index.js 249 B
packages/react-db/dist/esm/useLiveInfiniteQuery.js 1.32 kB
packages/react-db/dist/esm/useLiveQuery.js 1.37 kB
packages/react-db/dist/esm/useLiveQueryEffect.js 355 B
packages/react-db/dist/esm/useLiveSuspenseQuery.js 567 B
packages/react-db/dist/esm/usePacedMutations.js 401 B

compressed-size-action::react-db-package-size

kevin-dp and others added 5 commits July 2, 2026 08:53
Wire svelte-db into the shared suite. Svelte composables run inside a
persistent $effect.root (disposed on unmount); reads happen after flushSync().

The suite caught a real bug: svelte-db's toValue() unwrapping — added to support
the reactive `() => collection` input form — calls a disabled query fn like
`() => null` as if it were a getter, unwraps it to null, and passes {...null}
into createLiveQueryCollection, crashing in getQueryIR. The disabled
short-circuit is unreachable for this case, and svelte-db has no disabled tests.
Recorded as knownGaps (disabled-explicit, disabled-transition) until fixed.

All other 22 scenarios pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Wire solid-db into the shared suite. Each mount runs inside createRoot (disposed
on unmount); Solid auto-tracks signals so controllable inputs read a signal in
the query fn, and collection/config inputs are passed as accessors per Solid's
arity-based input detection.

The suite surfaced a divergence: solid-db routes errors through its
createResource/Suspense path, which throws CollectionStateError for an
<ErrorBoundary> to catch, rather than exposing a readable isError flag like
React/Vue/Svelte. Recorded as a knownGap (error-status).

All other 23 scenarios pass, including findOne, optimistic reconcile, and
disabled transitions.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Wire angular-db into the shared suite. Each mount runs injectLiveQuery inside a
child EnvironmentInjector created off TestBed's, disposed on unmount to fire the
DestroyRef cleanup. Controllable inputs use Angular's reactive { params, query }
form driven by a signal.

The suite surfaced a divergence: angular-db's plain `{ query }` config-object
path calls createLiveQueryCollection(opts) without injecting startSync:true (its
query-fn path does), so a bare config never syncs and returns empty —
React/Vue/Svelte/Solid all auto-start config objects. Recorded as a knownGap
(config-object-input).

All other 23 scenarios pass, including error status (Angular exposes isError
directly, unlike Solid's ErrorBoundary throw).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
makeHandle typed its param as ReturnType<typeof renderHook>
(RenderHookResult<unknown, unknown>), but mountControllable passes a typed
hook (RenderHookResult<..., { param: P }>), which fails vitest's project
typecheck on rerender's contravariant props. Widen the param to
RenderHookResult<any, any>. Reproduced locally via `vitest --run` (typecheck
enabled); now green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@kevin-dp kevin-dp changed the title test: cross-adapter live-query conformance suite (RFC #1623) test: live-query conformance suite with React/Vue/Svelte/Solid/Angular drivers (RFC #1623) Jul 2, 2026
@kevin-dp kevin-dp changed the title test: live-query conformance suite with React/Vue/Svelte/Solid/Angular drivers (RFC #1623) test: useLiveQuery conformance suite across all five framework adapters (RFC #1623) Jul 2, 2026

@KyleAMathews KyleAMathews left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the conformance-suite shape is good, and I’m approving this direction. I did a pass over an external review and validated a few test-infrastructure issues that seem worth addressing, either in this PR or a quick follow-up:

  1. The React conformance driver currently derives normalized isEnabled from status !== 'disabled', so the suite doesn’t actually catch a broken React isEnabled value. I verified by intentionally returning isEnabled: true for disabled React queries: the conformance test still passed. Mapping React’s normalized field from r.isEnabled makes the disabled conformance scenarios catch it. For adapters that don’t expose a real isEnabled field/signal, deriving from status is fine, but the normalized field should probably be named/treated honestly as status-derived.

  2. Expected-fail scenarios can leak mounted framework roots because h.unmount() is after assertions. I instrumented Solid roots and saw leaks from the expected-fail error-status and universal order-only-move scenarios. Wrapping mounted handles in try/finally { h.unmount() } fixes it.

  3. knownGaps should be validated against registered scenario keys. I added a bogus React gap and the suite still passed. A small scenarioKeys set plus validation test catches stale/misspelled gaps.

  4. includes-subquery is under-asserted. The suggested stronger assertion is directionally right, though johnSmith.issues is a child collection rather than [], so the strengthened assertion should read child collection contents through the collection API.

  5. The suite.ts header is stale: it says engine-heavy scenarios are “ported next,” but this PR already includes them.

I also checked for bidi/Trojan Source controls in the added conformance files and didn’t find any, so I wouldn’t block on that.

kevin-dp and others added 6 commits July 7, 2026 10:11
The React driver derived normalized `isEnabled` from `status !== 'disabled'`, so
the suite could not catch a broken react-db `isEnabled` (returning a wrong value
still passed). Map it from `r.isEnabled` instead. The other four adapters expose
no `isEnabled`, so they keep the status-derived value with an explicit comment.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…iew #2)

Expected-fail scenarios throw at the assertion, before their `h.unmount()`, so
the mounted framework root leaked (observed with Solid on error-status and the
universal order-only-move). Wrap the driver's mount* methods to track handles
and tear them all down in a finally around each scenario, so teardown runs
regardless of whether the scenario throws.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…eview #3)

A stale or misspelled knownGap silently no-ops (a bogus key just doesn't mark
any scenario), which is a live footgun while gaps churn across the follow-up
fixes. Register every scenario key and add a test asserting each driver's
knownGaps and UNIVERSAL_EXPECTED_FAIL reference a real scenario.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The includes scenario only checked that `john.issues` was defined. It's a child
collection, so read its contents through the collection API and assert John
(id 1) has exactly issues i1 and i3.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The header said engine-heavy scenarios were "ported next"; they've been in the
suite for a while. Replace the stale STATUS note with the actual coverage.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@kevin-dp

kevin-dp commented Jul 7, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for the careful pass, Kyle — all five were legit. Addressed each as its own commit on this branch:

  1. React isEnabled was status-derived, masking a broken value (e5c131fb) — the React driver now maps the normalized field from r.isEnabled, so the disabled scenarios actually catch it. The other four adapters don't expose an isEnabled, so they keep the status-derived value with an explicit comment noting that.

  2. Expected-fail scenarios leaked mounted roots (457d1099) — the mount* methods are now wrapped to track every handle, and each scenario tears them all down in a finally, so teardown runs even when a scenario throws before its own h.unmount(). This covers both the universal order-only-move and any per-adapter knownGap (e.g. Solid error-status).

  3. knownGaps weren't validated (596bba2c) — every scenario key is now registered, and a test asserts each driver's knownGaps (and UNIVERSAL_EXPECTED_FAIL) reference a real key. I confirmed it catches a bogus/misspelled gap. Handy since gaps are churning across the follow-up fix PRs.

  4. includes-subquery was under-asserted (4c117967) — you're right that john.issues is a child collection, so it now reads the contents through the collection API and asserts John (id 1) has exactly i1 and i3.

  5. Stale header (bed14e62) — replaced the "ported next" STATUS note with the actual coverage list.

All five suites stay green at 25/25 (the +1 is the new knownGaps validation). No changes needed for the bidi/Trojan-source note. The rest of the stack (#1641/#1642 and the three fix PRs) has been rebased onto this and re-verified.

Remove references to the (uncommitted) RFC and to GitHub issue numbers from the
conformance suite comments and driver headers; the behavior descriptions stay.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@kevin-dp kevin-dp marked this pull request as ready for review July 7, 2026 10:15
@kevin-dp kevin-dp merged commit 3ec3e7a into main Jul 7, 2026
10 of 11 checks passed
@kevin-dp kevin-dp deleted the test/live-query-conformance-suite branch July 7, 2026 10:20

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (4)
packages/db/tests/conformance/suite.ts (3)

88-94: 🩺 Stability & Availability | 🔵 Trivial | ⚡ Quick win

Silent teardown catch-all may mask real driver bugs.

Every handle.unmount() failure is swallowed unconditionally. Since the whole point of this suite is to surface adapter-specific defects, an unmount that throws for a genuine bug (not just idempotent double-unmount) will be silently hidden rather than surfaced as a signal for someone to investigate.

♻️ Suggested tweak: log instead of fully silencing
         for (const handle of handles) {
           try {
             handle.unmount()
-          } catch {
-            // teardown is best-effort / idempotent
+          } catch (err) {
+            // teardown is best-effort / idempotent, but still surface unexpected
+            // failures for diagnosis instead of hiding them entirely.
+            console.warn(`[conformance] handle.unmount() failed during teardown:`, err)
           }
         }
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/db/tests/conformance/suite.ts` around lines 88 - 94, The teardown
loop in the conformance suite is swallowing every `handle.unmount()` error,
which can hide real adapter bugs. Update the `handles` cleanup block in
`suite.ts` so `handle.unmount()` failures are not silently ignored; instead, log
the caught error with enough context to identify which handle failed, while
still keeping teardown best-effort and idempotent.

104-176: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Repeated makeSource + mount(select id) + flush boilerplate across scenarios.

Several scenarios (basic-select, live-insert, live-delete, orderby, live-update, etc.) repeat the same driver.makeSource(SEED)driver.mount(...select id...)await h.flush() setup. Extracting a small helper (e.g. mountIdSelect(source)) would reduce duplication and keep each scenario body focused on its assertion.

As per coding guidelines, "Extract common logic into utility functions when identical or near-identical code blocks appear in multiple places."

Also applies to: 358-400

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/db/tests/conformance/suite.ts` around lines 104 - 176, The test
scenarios repeat the same driver.makeSource(SEED) and driver.mount(...) setup
with an id-only select, so extract that shared initialization into a small
helper and reuse it across the conformance scenarios. Add a utility such as a
mount helper near the suite-level test setup, then update the affected scenarios
like basic-select, live-insert, live-delete, and orderby to call it and keep
only the scenario-specific assertions and mutations.

Source: Coding guidelines


104-676: 📐 Maintainability & Code Quality | 🔵 Trivial | 🏗️ Heavy lift

Pervasive any typing in scenario callbacks.

Nearly every query-builder callback throughout the file is typed as (q: any) / ({ items }: any) / ({ items, persons }: any), etc. (e.g. Lines 112-113, 130-131, 244-251, 296-313, 332-339). This runs counter to the guideline requiring unknown + type guards over any. Given each adapter passes its own framework-specific generic query builder, fully eliminating any may require nontrivial generic plumbing through contract.ts's QueryBuild type — worth considering for a follow-up rather than blocking this draft suite.

As per coding guidelines, "Avoid using any types; use unknown instead when the type is truly unknown, and provide proper type annotations for return values."

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/db/tests/conformance/suite.ts` around lines 104 - 676, The scenario
callbacks in this conformance suite rely on pervasive any annotations in
query-builder lambdas, which conflicts with the typing guideline. Replace these
callback parameters in the suite’s scenario definitions and query expressions
(such as the mounts using q, items, persons, issues, ic) with properly typed
generics or unknown plus narrowing, using the relevant driver/query-builder
types instead of any. If full typing is not practical here, wire the generic
query-builder type through the shared QueryBuild/contract layer so the callbacks
can infer concrete shapes without any.

Source: Coding guidelines

packages/react-db/tests/conformance.test.tsx (1)

41-121: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Identical boilerplate duplicated across all 5 driver files.

writer, makeSource, makeDeferredSource, makePrecreated, and makeErrorSource are byte-for-byte identical (modulo the id prefix string) in the Angular, Vue, Svelte, and Solid driver files. Since each only needs createCollection, createLiveQueryCollection, mockSyncCollectionOptions, and mockSyncCollectionOptionsNoInitialState (all realm-specific), this can be extracted into a shared parameterized factory in packages/db/tests/conformance/ that each driver calls with its own realm imports and prefix.

The PR notes state shared-logic extraction is deferred to a later stacked refactor, so this is a good-to-have rather than a blocker for this draft.

♻️ Illustrative extraction (outside current file's line range)
// packages/db/tests/conformance/sourceFactories.ts
export function createSourceFactories<Realm extends {
  createCollection: typeof import('`@tanstack/db`').createCollection
  createLiveQueryCollection: typeof import('`@tanstack/db`').createLiveQueryCollection
}>(realm: Realm, mocks: {
  mockSyncCollectionOptions: typeof mockSyncCollectionOptions
  mockSyncCollectionOptionsNoInitialState: typeof mockSyncCollectionOptionsNoInitialState
}, prefix: string) {
  let sourceSeq = 0
  function writer<T extends { id: string }>(collection: any) { /* shared body */ }
  function makeSource<T extends { id: string }>(initialData: ReadonlyArray<T>) { /* shared body */ }
  function makeDeferredSource<T extends { id: string }>() { /* shared body */ }
  function makePrecreated(build: QueryBuild, opts?: { startSync?: boolean }) { /* shared body */ }
  function makeErrorSource() { /* shared body */ }
  return { makeSource, makeDeferredSource, makePrecreated, makeErrorSource }
}

Then this file's lines 41-121 collapse to a single call:

-let sourceSeq = 0
-
-function writer<T extends { id: string }>(collection: any) { ... }
-function makeSource<T extends { id: string }>(...) { ... }
-function makeDeferredSource<T extends { id: string }>() { ... }
-function makePrecreated(build: QueryBuild, opts?: { startSync?: boolean }) { ... }
-function makeErrorSource() { ... }
+const { makeSource, makeDeferredSource, makePrecreated, makeErrorSource } =
+  createSourceFactories(
+    { createCollection, createLiveQueryCollection },
+    { mockSyncCollectionOptions, mockSyncCollectionOptionsNoInitialState },
+    `conformance-react`,
+  )

As per coding guidelines, "Extract common logic into utility functions when identical or near-identical code blocks appear in multiple places."

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/react-db/tests/conformance.test.tsx` around lines 41 - 121, The
conformance test helpers here are duplicated across the driver files, so extract
the shared boilerplate into a parameterized factory under
packages/db/tests/conformance and have this file consume it instead of defining
writer, makeSource, makeDeferredSource, makePrecreated, and makeErrorSource
locally. Keep the realm-specific pieces (createCollection,
createLiveQueryCollection, mockSyncCollectionOptions,
mockSyncCollectionOptionsNoInitialState, and the id prefix/sourceSeq handling)
as injected dependencies so each driver can reuse the same implementation.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@packages/db/tests/conformance/suite.ts`:
- Around line 88-94: The teardown loop in the conformance suite is swallowing
every `handle.unmount()` error, which can hide real adapter bugs. Update the
`handles` cleanup block in `suite.ts` so `handle.unmount()` failures are not
silently ignored; instead, log the caught error with enough context to identify
which handle failed, while still keeping teardown best-effort and idempotent.
- Around line 104-176: The test scenarios repeat the same
driver.makeSource(SEED) and driver.mount(...) setup with an id-only select, so
extract that shared initialization into a small helper and reuse it across the
conformance scenarios. Add a utility such as a mount helper near the suite-level
test setup, then update the affected scenarios like basic-select, live-insert,
live-delete, and orderby to call it and keep only the scenario-specific
assertions and mutations.
- Around line 104-676: The scenario callbacks in this conformance suite rely on
pervasive any annotations in query-builder lambdas, which conflicts with the
typing guideline. Replace these callback parameters in the suite’s scenario
definitions and query expressions (such as the mounts using q, items, persons,
issues, ic) with properly typed generics or unknown plus narrowing, using the
relevant driver/query-builder types instead of any. If full typing is not
practical here, wire the generic query-builder type through the shared
QueryBuild/contract layer so the callbacks can infer concrete shapes without
any.

In `@packages/react-db/tests/conformance.test.tsx`:
- Around line 41-121: The conformance test helpers here are duplicated across
the driver files, so extract the shared boilerplate into a parameterized factory
under packages/db/tests/conformance and have this file consume it instead of
defining writer, makeSource, makeDeferredSource, makePrecreated, and
makeErrorSource locally. Keep the realm-specific pieces (createCollection,
createLiveQueryCollection, mockSyncCollectionOptions,
mockSyncCollectionOptionsNoInitialState, and the id prefix/sourceSeq handling)
as injected dependencies so each driver can reuse the same implementation.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 6fd94bb8-a36a-4a1d-b201-622045a3d01b

📥 Commits

Reviewing files that changed from the base of the PR and between f7da776 and 3f6a9e5.

📒 Files selected for processing (7)
  • packages/angular-db/tests/conformance.test.ts
  • packages/db/tests/conformance/contract.ts
  • packages/db/tests/conformance/suite.ts
  • packages/react-db/tests/conformance.test.tsx
  • packages/solid-db/tests/conformance.test.tsx
  • packages/svelte-db/tests/conformance.svelte.test.ts
  • packages/vue-db/tests/conformance.test.ts

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants