Skip to content

fix(orb): index orb_relay_pending by created_at so the relay-drain prune stops full-scanning - #7451

Closed
bitfathers94 wants to merge 1 commit into
JSONbored:mainfrom
bitfathers94:fix/issue-7430
Closed

fix(orb): index orb_relay_pending by created_at so the relay-drain prune stops full-scanning#7451
bitfathers94 wants to merge 1 commit into
JSONbored:mainfrom
bitfathers94:fix/issue-7430

Conversation

@bitfathers94

Copy link
Copy Markdown
Contributor

Summary

  • pruneRelayPending runs on every pull-mode relay-drain (POST /v1/orb/relay/pull, fleet-wide, every 30s) and filters orb_relay_pending by a bare created_at predicate. Both existing indexes lead with installation_id, so neither can serve it — the SELECT and DELETE full-scan the table and can exceed the pull request's AbortSignal.timeout(30_000) budget, surfacing as the recurring TimeoutError in Sentry since 2026-07-15.
  • Adds a (created_at, delivery_id) index (idx_orb_relay_pending_created_at) — declared in src/db/schema.ts and created in migrations/0167_orb_relay_pending_created_at_index.sql. It turns both the SELECT (WHERE created_at < ? ORDER BY created_at, delivery_id) and the DELETE (WHERE created_at < ?) into index range scans, and lets the SELECT satisfy its ORDER BY from the index with no temp-B-tree sort.

Closes #7430

Scope

  • The PR title follows type(scope): short summary Conventional Commit format.
  • This PR is focused and does not mix unrelated backend, UI, MCP, docs, dependency, and deploy changes.
  • This follows CONTRIBUTING.md and does not reintroduce GitHub Pages, VitePress, site/, or CNAME.
  • I linked a currently open issue this PR resolves (Closes #7430).

Validation

  • git diff --check
  • npm run db:migrations:check (contiguous 0001..0167, no new duplicates)
  • npm run db:schema-drift:check (schema.ts matches migrations/)
  • npm run typecheck
  • New regression test test/unit/orb-relay-pending-index.test.ts asserts the index exists and that both of pruneRelayPending's queries EXPLAIN QUERY PLAN as a SEARCH via idx_orb_relay_pending_created_at (not a SCAN, and with no temp-B-tree sort). It fails before this change (full scan) and passes after.
  • test/integration/orb-relay.test.ts (83 tests) still green.
  • codecov/patch: the only measured diff line is the schema.ts index declaration, executed at module import like its sibling index declarations; the migration and test files are not coverage-measured.

Safety

  • No secrets, wallet details, hotkeys, coldkeys, PATs, private keys, trust scores, or private evidence exposed.
  • Pure DB-index change: no behavior change to query results, only their access path — existing relay tests confirm identical outcomes.

Notes

  • The index mirrors the SELECT's ORDER BY (created_at, delivery_id) column order so the drain prune needs neither a full scan nor a sort step.

…une stops full-scanning

pruneRelayPending runs on every pull-mode relay-drain (fleet-wide, every 30s) and filters
orb_relay_pending by a bare created_at predicate:

  SELECT ... WHERE created_at < ? ORDER BY created_at, delivery_id LIMIT ?
  DELETE     WHERE created_at < ?

Both existing indexes (idx_orb_relay_pending_install, idx_orb_relay_pending_coalesce) lead with
installation_id, so neither serves a created_at-only predicate — the SELECT and DELETE full-scan
the table and can blow the pull request's AbortSignal.timeout(30_000) budget, surfacing as the
recurring TimeoutError in Sentry.

Add a (created_at, delivery_id) index: it turns both queries into range scans and lets the SELECT
satisfy its ORDER BY (created_at, delivery_id) straight from the index with no temp-B-tree sort.

Closes JSONbored#7430
@bitfathers94
bitfathers94 requested a review from JSONbored as a code owner July 20, 2026 10:52
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@codecov

codecov Bot commented Jul 20, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 91.35%. Comparing base (25decd9) to head (64d518b).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #7451   +/-   ##
=======================================
  Coverage   91.35%   91.35%           
=======================================
  Files         716      716           
  Lines       72990    72990           
  Branches    21628    21627    -1     
=======================================
  Hits        66678    66678           
  Misses       5272     5272           
  Partials     1040     1040           
Flag Coverage Δ
shard-1 33.69% <ø> (+1.01%) ⬆️
shard-2 40.35% <ø> (+4.23%) ⬆️
shard-3 28.76% <ø> (-5.77%) ⬇️
shard-4 39.57% <ø> (-3.32%) ⬇️
shard-5 35.72% <ø> (-1.25%) ⬇️
shard-6 34.83% <ø> (+1.28%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
src/db/schema.ts 72.97% <ø> (ø)

@loopover-orb loopover-orb Bot added the gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. label Jul 20, 2026
@loopover-orb

loopover-orb Bot commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Tip

✅ LoopOver review result - approve/merge recommended

Review updated: 2026-07-20 11:02:40 UTC

3 files · 1 AI reviewer · no blockers · readiness 100/100 · CI green · clean

✅ Suggested Action - Approve/Merge

  • safe to merge

Review summary
Adds a (created_at, delivery_id) index on orb_relay_pending and a matching migration, correctly turning pruneRelayPending's SELECT and DELETE from full scans into index range scans, with the SELECT also satisfying its ORDER BY from the index without a temp B-tree sort. The migration is additive (CREATE INDEX IF NOT EXISTS), schema.ts and the migration file agree on column order, and the new unit test directly asserts SEARCH-not-SCAN via EXPLAIN QUERY PLAN against the real queries used in orb/relay.ts, which is a solid regression test tied to the actual production path rather than a fabricated one.

Nits — 4 non-blocking
  • The `sqlite_master` reference in test/unit/orb-relay-pending-index.test.ts:8 is SQLite's actual system table name, so it can't be renamed without breaking the query — the external brief's suggestion to replace it doesn't apply here.
  • src/db/schema.ts is flagged as a large file (~888 lines) by the size-smell scan, but this diff only adds 4 lines to an existing table definition, so it's pre-existing size, not something introduced here.
  • Confirm idx_orb_relay_pending_created_at is picked up by the existing `db:schema-drift:check` in CI (already checked per the PR description) so future schema edits stay in sync.
  • Consider also asserting index cardinality/row-count behavior in the new test (e.g. planner picks the index even with a non-trivial row count) if SQLite's planner ever needs ANALYZE stats to prefer it over a table scan in production-sized tables.

Decision drivers

  • ✅ Code review — No blockers (1 reviewer)
  • ✅ Gate result — Passing (No configured blocker found.)
Context & advisory signals — never blocks the verdict
Signal Result Evidence
Linked issue ✅ Linked #7430
Related work ✅ No active overlap found No same-issue or scoped active PR overlap found.
Change scope ✅ 20/20 Low review scope from cached public metadata (1 linked issue).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 45 registered-repo PR(s), 29 merged, 0 issue(s).
Contributor context ✅ Confirmed Gittensor contributor bitfathers94; Gittensor profile; 45 PR(s), 0 issue(s).
Improvement ✅ Minor risk: clean · value: minor · LLM: moderate
Linked issue satisfaction

Addressed
The PR adds a (created_at, delivery_id) index in both the migration and schema.ts that directly targets the bare created_at predicate used by pruneRelayPending's SELECT and DELETE, matching the issue's root-cause diagnosis, and includes a regression test verifying EXPLAIN QUERY PLAN shows an index SEARCH rather than a SCAN.

Review context
  • Author: bitfathers94
  • Role context: outside_contributor
  • Public audience mode: oss maintainer
  • Lane context: Repository is configured for direct PR review.
  • Public profile languages: not available
  • Official Gittensor activity: 45 PR(s), 0 issue(s).
  • PR-specific overlap: none found.
Contributor next steps
  • Keep the PR focused and include validation evidence before maintainer review.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.
🧪 Chat with LoopOver

Ask LoopOver a question about this PR directly in a comment — grounded only in the same cached, public-safe facts shown above, never a new claim.

  • @loopover ask &lt;question&gt; answers contribution-quality Q&A with source citations and freshness.
  • @loopover chat &lt;question&gt; answers in natural prose from cached decision-pack facts via local inference (maintainer/collaborator; read-only).
  • A plain-language @loopover mention with a real question is routed to the closest matching read-only command automatically — no exact syntax required.

Full command reference: https://loopover.ai/docs/loopover-commands

🧪 Experimental — new and may change.

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by LoopOver, a quiet PR intelligence layer for OSS maintainers.

  • Re-run LoopOver review

@loopover-orb

loopover-orb Bot commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

LoopOver is closing this pull request on the maintainer's behalf (Linked issue #7430 is assigned to the maintainer (@JSONbored) — that work is reserved for the maintainer, so this PR cannot be auto-accepted.). This is an automated maintenance action — to pursue this change, please open a new pull request with the issues resolved. Closed PRs may be analyzed later to improve review accuracy, but they are not automatically reopened or re-reviewed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(orb): orb-relay-drain pull times out — pruneRelayPending full-scans orb_relay_pending

1 participant