Skip to content

fix: prevent global coverage timeouts on large archives#146

Merged
steipete merged 1 commit into
openclaw:mainfrom
hannesrudolph:codex/fix-global-coverage-timeout
Jul 21, 2026
Merged

fix: prevent global coverage timeouts on large archives#146
steipete merged 1 commit into
openclaw:mainfrom
hannesrudolph:codex/fix-global-coverage-timeout

Conversation

@hannesrudolph

Copy link
Copy Markdown
Member

Problem

A scheduled incremental snapshot refresh completed its import successfully, then failed closed during post-maintenance validation because global coverage --json exceeded its internal deadline. The affected archive was 8.39 GB with about 1.39 million messages and 27.8k channels. The failed run preserved all 2,323,757 preexisting identities, added only 2 messages, and reported query channel coverage: context deadline exceeded. An immediate warm retry completed successfully.

Upstream main currently gives every coverage query the same 2-minute deadline and uses a channel-first aggregate for both filtered and global reports. On a large global archive, that shape can repeatedly walk message indexes per channel and leave too little cold-cache margin.

Fix

  • Use the existing channel-first query for guild-scoped reports.
  • Use one materialized sequential message scan for global reports, then join the aggregate to channels.
  • Keep guild-scoped coverage bounded at 2 minutes.
  • Give global coverage an 8-minute bound, leaving 2 minutes under the caller's 10-minute validation timeout.
  • Preserve empty channels and continue excluding deleted messages.

Validation

  • go test ./...
  • go vet ./...
  • go test ./internal/store -run TestCoverage -count=1
  • Built the candidate and ran read-only coverage --json against the live 8.39 GB archive: exit 0 in 27.93 seconds.
  • Query-plan test verifies the global report materializes a sequential messages scan rather than selecting either channel index.

No crawl, sync, import, publish, vacuum, checkpoint, integrity scan, schema change, or archive mutation is introduced by this patch.

@steipete
steipete merged commit 68585e1 into openclaw:main Jul 21, 2026
9 checks passed
@steipete

Copy link
Copy Markdown
Collaborator

Landed as 68585e120c33ca8bb009825528e5241241909e0c.

Verification on exact PR head b9423033fa1eb185d08ab1aee1befe059814bdf2:

  • GOWORK=off go test ./internal/store -run '^TestCoverage' -count=1 -v — pass, including global/filtered parity, empty/deleted rows, deadlines/cancellation, and the sequential query plan.
  • GOWORK=off go test ./... — pass.
  • GOWORK=off go vet ./... — pass.
  • gofmt -d internal/store/coverage.go internal/store/coverage_test.go and git diff --check origin/main...HEAD — clean.
  • Built the candidate and ran read-only coverage --json against a disposable schema-upgraded clone of the local representative archive (8.41 GB, 1,536,377 messages, 27,571 channels): exit 0 in 29.32s with all counts preserved. The original archive was never opened writable.
  • Executed the old and candidate SQL shapes against that archive: byte-identical row-stream SHA-256 60fb9b5b6d814c313b0f651814a6be6074f5b32202114d3efbd346af5faf2157; old shape 41.48s, candidate shape 25.81s.
  • .agents/skills/autoreview/scripts/autoreview --mode branch --base origin/main --stream-engine-output — clean, no accepted/actionable findings; overall correctness 0.96.

Hosted checks were green on the exact PR head:

  • CI run 29775171394: lint 88462704908, test 88462704911, deps 88462704922, release-check 88462704881, secrets 88462704916.
  • CodeQL run 29775171393: analyze 88462704648; CodeQL result check 88462954657.
  • Docker run 29775171464: job 88462704859.
  • Secret-scanning run 29775171454: job 88462704948.

No dependency files changed. No release or deployment action was performed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants