Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/fix-sqlite-expression-index-planning.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@tanstack/db-sqlite-persistence-core': patch
---

Use canonical SQL literals for serialized ref JSON paths so runtime predicates match persisted expression indexes and SQLite can use them.
2 changes: 2 additions & 0 deletions docs/contributing/oracle-coverage.md
Original file line number Diff line number Diff line change
Expand Up @@ -111,6 +111,7 @@ comment and the current API/architecture contract before extending its model.
| Electric and TrailBase | [Electric histories](https://github.com/TanStack/db/blob/main/packages/electric-db-collection/tests/electric-oracle.property.test.ts), [PostgreSQL semantics](https://github.com/TanStack/db/blob/main/packages/electric-db-collection/e2e/sql-predicate-semantics.e2e.test.ts), [TrailBase contract](https://github.com/TanStack/db/blob/main/packages/trailbase-db-collection/tests/ORACLE.md) | Installed SDK delivery/framing, independent predicates, exact subscription arguments and late errors. SDK fixtures and a real service test earn different credit. |
| PowerSync | [tests](https://github.com/TanStack/db/tree/main/packages/powersync-db-collection/tests), `tests/correctness-oracle.test.ts` | Applied receipt positions crossed with held peers, native SQLite/SDK and cleanup evidence. Run the focused owner with the package's `test:oracles` command. A timeout mutant proves a progress failure, not every value assertion. |
| SQLite persistence and native hosts | [persisted histories](https://github.com/TanStack/db/blob/main/packages/db-sqlite-persistence-core/tests/persisted.test.ts), [driver contracts](https://github.com/TanStack/db/blob/main/packages/db-sqlite-persistence-core/tests/contracts/sqlite-driver-contract.ts), [browser OPFS lifecycle](https://github.com/TanStack/db/blob/main/packages/browser-db-sqlite-persistence/tests/opfs-page-lifecycle-oracle.test.ts), [worker diagnostics](https://github.com/TanStack/db/blob/main/packages/browser-db-sqlite-persistence/tests/opfs-worker-diagnostics-oracle.test.ts), [113-law manifest](https://github.com/TanStack/db/blob/main/packages/db-collection-e2e/src/fixtures/persisted-conformance-manifest.ts) | Cache/remote rejection/peer/reopen histories, exact driver results, controlled page/worker ownership, and diagnostic-cause retention. Fake workers and synthetic page events do not prove native handle release or real bfcache admission. The manifest excludes progressive and move suites; registration and shim runs are not device execution. |
| SQLite expression-index planning | [Node expression-index oracle](https://github.com/TanStack/db/blob/main/packages/node-db-sqlite-persistence/tests/expression-index-oracle.test.ts) | RFC #1659 invariant 8 owns matching persisted-index and runtime-predicate JSON expression shapes. Generated paths use 1-6 character identifiers starting with `a/m/p/t/x` and continuing with `a/b/e/i/n/r/s/0/1`, plus up to three identifier or `0..3` index tails. Values are integer targets `-10_000..10_000` with adjacent distractors, `target-'${suffix}` strings with `before-${suffix}`/`after-${suffix}` distractors for suffixes `0..10_000`, and booleans with duplicated opposite distractors. `_root`/`Upper_2`, fractional-number, and empty-string examples are omitted from generation but same-path audit probes are GREEN; no permanent matrix is claimed. An independent full scan judges matching keys while captured real-driver SQL and bindings are replayed through `EXPLAIN QUERY PLAN` for a named-index search. Run the package's `test:oracles` campaign. Arbitrary raw SQL, result ordering, date/bigint/null/tagged-value coercions, and native-host execution are outside this law. |
| Offline execution | [scheduler](https://github.com/TanStack/db/blob/main/packages/offline-transactions/tests/KeyScheduler.property.test.ts), [leadership](https://github.com/TanStack/db/blob/main/packages/offline-transactions/tests/leadership-replay.property.test.ts), [settlement](https://github.com/TanStack/db/blob/main/packages/offline-transactions/tests/transaction-settlement.property.test.ts), [serialization](https://github.com/TanStack/db/blob/main/packages/offline-transactions/tests/transaction-serializer.property.test.ts) | Declarative FIFO eligibility, per-transaction outcomes, durable state and typed wire trees. Issued work may finish after ownership loss, but new work must not start. Exactly-once network execution is not promised. |
| Frameworks | [React conformance](https://github.com/TanStack/db/blob/main/packages/react-db/tests/conformance.test.tsx), [React pagination](https://github.com/TanStack/db/blob/main/packages/react-db/tests/infinite-query-conformance.test.tsx), [shared suites](https://github.com/TanStack/db/tree/main/packages/db-collection-e2e/src/suites) | Exact exposed rows/pages and each framework's own lifecycle cuts. A React witness does not prove Vue/Solid/Angular/Svelte scheduling. Preserve their receiving registrations. |
| Structural values and ordered primitives | [hash values](https://github.com/TanStack/db/blob/main/packages/db-ivm/tests/hash.property.test.ts), [hash graphs](https://github.com/TanStack/db/blob/main/packages/db-ivm/tests/hash-graph.property.test.ts), [mixed hash graphs](https://github.com/TanStack/db/blob/main/packages/db-ivm/tests/hash-mixed-graph.property.test.ts), [hash retry](https://github.com/TanStack/db/blob/main/packages/db-ivm/tests/hash-failure-retry.property.test.ts), [comparison](https://github.com/TanStack/db/blob/main/packages/db/tests/comparison.property.test.ts), [deep equality](https://github.com/TanStack/db/blob/main/packages/db/tests/utils.property.test.ts), [cursor](https://github.com/TanStack/db/blob/main/packages/db/tests/cursor.property.test.ts), [indexes](https://github.com/TanStack/db/blob/main/packages/db/tests/index-update.property.test.ts), [query identity](https://github.com/TanStack/db/blob/main/packages/db/tests/query/identity-output-shape-oracle.test.ts) | Independent flat values, graph topology, algebraic laws, Map/group/sort recomputation, expression denotation, and compiled output bags. Hash collision freedom is not promised. Unsupported composite cursors reject. |
Expand Down Expand Up @@ -155,6 +156,7 @@ pnpm --filter @tanstack/db-ivm build
pnpm --filter @tanstack/db build
pnpm --filter @tanstack/db test:oracles
pnpm --filter @tanstack/powersync-db-collection test:oracles
pnpm --filter @tanstack/node-db-sqlite-persistence test:oracles
# Service-dependent: requires Electric and PostgreSQL to be running.
pnpm --filter @tanstack/electric-db-collection test:e2e
pnpm run typecheck:tests
Expand Down
15 changes: 10 additions & 5 deletions packages/db-sqlite-persistence-core/src/sqlite-core-adapter.ts
Original file line number Diff line number Diff line change
Expand Up @@ -594,18 +594,23 @@ function compileComparisonSql(
function compileRefExpressionSql(jsonPath: string): CompiledSqlFragment {
const typePath = `${jsonPath}.${PERSISTED_TYPE_TAG}`
const taggedValuePath = `${jsonPath}.${PERSISTED_VALUE_TAG}`
// createJsonPath has already validated every segment. Keep these paths as
// canonical SQL literals so runtime refs match persisted expression indexes.
const typePathSql = toSqliteLiteral(typePath)
const taggedValuePathSql = toSqliteLiteral(taggedValuePath)
const jsonPathSql = toSqliteLiteral(jsonPath)

return {
supported: true,
sql: `(CASE json_extract(value, ?)
WHEN 'bigint' THEN CAST(json_extract(value, ?) AS NUMERIC)
WHEN 'date' THEN json_extract(value, ?)
sql: `(CASE json_extract(value, ${typePathSql})
WHEN 'bigint' THEN CAST(json_extract(value, ${taggedValuePathSql}) AS NUMERIC)
WHEN 'date' THEN json_extract(value, ${taggedValuePathSql})
WHEN 'nan' THEN NULL
WHEN 'infinity' THEN NULL
WHEN '-infinity' THEN NULL
ELSE json_extract(value, ?)
ELSE json_extract(value, ${jsonPathSql})
END)`,
params: [typePath, taggedValuePath, taggedValuePath, jsonPath],
params: [],
valueKind: `unknown`,
}
}
Expand Down
1 change: 1 addition & 0 deletions packages/node-db-sqlite-persistence/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@
"dev": "vite build --watch",
"lint": "eslint . --fix",
"test": "vitest --run",
"test:oracles": "vitest --run tests/expression-index-oracle.test.ts",
"test:e2e": "pnpm --filter @tanstack/db-ivm build && pnpm --filter @tanstack/db build && pnpm --filter @tanstack/db-sqlite-persistence-core build && pnpm --filter @tanstack/node-db-sqlite-persistence build && vitest --config vitest.e2e.config.ts --run"
},
"type": "module",
Expand Down
Loading
Loading