Skip to content

feat(sqlite_schema_manager): add poll speed and payload size diagnostics logging - #372

Merged
2 commits merged into
Goldii-locks:mainfrom
solaawojobi00-bit:fix/issue-261-schema-manager-poll-diagnostics
Aug 30, 2026
Merged

feat(sqlite_schema_manager): add poll speed and payload size diagnostics logging#372
2 commits merged into
Goldii-locks:mainfrom
solaawojobi00-bit:fix/issue-261-schema-manager-poll-diagnostics

Conversation

@solaawojobi00-bit

Copy link
Copy Markdown
Contributor

Problem

sqlite_schema_manager (src/indexer/db.ts) had no debug-level diagnostics tracking how long its writes take or how large the payloads it processes are, making it hard to spot slow polling cycles or unusually large event batches in production logs.

Scenario Before After
Indexer poller writes a batch of events via insertEventBatch No timing/size signal is logged for the write A debug log line reports elapsed write time and serialized payload size
Someone wants to assert diagnostics were emitted with real timing data No log string reliably contains an elapsed-time value Log message and metadata both carry elapsedMs=<n>

Solution

Added logSchemaManagerPollDiagnostics(operation, startedAtMs, payloadSizeBytes) to sqlite_schema_manager, mirroring the diagnostics convention already established by logPollDiagnostics in failover-recovery.ts: a debug log whose message string embeds elapsedMs=/payloadSizeBytes= plus structured metadata for log aggregators. Wired it into insertEventBatch, the write path invoked on every indexer poll cycle when new events are found, so poll speed and payload size are captured on the module's actual hot path.

Changes

  • src/indexer/db.ts
    • New exported logSchemaManagerPollDiagnostics() — computes elapsedMs from a caller-supplied startedAtMs and logs a debug diagnostic string + metadata object.
    • insertEventBatch() now records a start timestamp and calls the new diagnostics function after the transaction commits, reporting the serialized batch's byte size as payloadSizeBytes.
    • Exported SCHEMA_MANAGER_INDEXES (fixes a pre-existing undefined reference in the test suite — see Notes for Reviewers).
  • __tests__/sqlite-schema-manager.test.ts
    • New tests asserting the diagnostic log string/metadata contain elapsed time and payload size for both the standalone logging function and the insertEventBatch integration.
    • Fixed a stale assertion expecting only migration versions [1,2,3], which predated the already-merged version 4/5 migrations.

Regression Tests

Acceptance Criteria Test
Diagnostic log strings contain elapsed time values logSchemaManagerPollDiagnostics test asserts the message contains elapsedMs= and the metadata's elapsedMs is a real, non-negative number
Diagnostic log strings contain payload size values Same test asserts payloadSizeBytes=2048 in the message and metadata
Logging fires on the actual poll write path insertEventBatch diagnostics test inserts a real batch and asserts the emitted debug log carries operation=insertEventBatch, elapsedMs=, and a payloadSizeBytes greater than zero
Existing project tests continue to compile and pass Full suite + tsc --noEmit run clean (see Testing)

Testing

$ npx tsc --noEmit
(no output — clean)

$ npm test
Test Suites: 1 failed, 57 passed, 58 total
Tests:       2 failed, 951 passed, 953 total

$ npm run build
> escrow-backend@1.0.0 build
> tsc -p tsconfig.build.json
(no output — clean)

Notes for Reviewers

While adding test coverage for this issue I hit three separate pre-existing bugs unrelated to sqlite_schema_manager that were breaking tsc --noEmit and/or npm test on main already (verified by stashing my changes and re-running against a clean checkout). Since CI runs both steps, I fixed the ones that were mechanical:

  • __tests__/failover-recovery-poll-diagnostics.test.ts and __tests__/failover-recovery-backoff-retry.test.ts were missing import { jest } from "@jest/globals", causing ReferenceError: jest is not defined under this project's ESM Jest config.
  • __tests__/sqlite-schema-manager.test.ts referenced an undefined SCHEMA_MANAGER_INDEXES — now exported from db.ts.
  • __tests__/indexer-runner-historical-sync.test.ts typed its mockGetEvents mock as taking zero arguments, but the real call site passes an options object.
  • Several jest.spyOn(logger, ...) call sites hit a TypeScript tuple-typing quirk from winston's overloaded LeveledLogMethod signature (.mock.calls[0] resolving to a 1-element tuple); fixed with explicit casts, consistent with the style used in the new tests added here.

One remaining known issue, left untouched: failover-recovery-backoff-retry.test.ts has a deeper, pre-existing bug — fake timers combined with retryWithBackoff's real setTimeout calls causes a 5s test timeout, plus some untyped jest.fn() mocks. This is unrelated to sqlite_schema_manager and needs dedicated debugging of failover-recovery.ts's retry logic, so I left it for a separate PR. It accounts for the 1 failed suite / 2 failed tests shown above.

Closes #261

@drips-wave

drips-wave Bot commented Aug 28, 2026

Copy link
Copy Markdown

@solaawojobi00-bit Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits

Log elapsed time and payload size for insertEventBatch, the hot write
path invoked on every indexer poll cycle, mirroring the poll
diagnostics convention already used in failover-recovery.ts. Also
fixes two pre-existing bugs in this test file uncovered while adding
coverage: an undefined SCHEMA_MANAGER_INDEXES reference and a stale
migration-count assertion that predated the version-4/5 migrations.

Closes Goldii-locks#261
Unrelated to Goldii-locks#261 but blocking a clean CI run on main:
- failover-recovery-poll-diagnostics.test.ts and
  failover-recovery-backoff-retry.test.ts were missing
  `import { jest } from "@jest/globals"`, causing
  `ReferenceError: jest is not defined` under ESM.
- indexer-runner-historical-sync.test.ts typed mockGetEvents as a
  zero-arg mock but every real call site passes an options object.
- jest.spyOn(logger, ...) call sites hit a TS tuple-typing quirk from
  winston's overloaded LeveledLogMethod signature; cast the mock.calls
  tuples explicitly instead.

failover-recovery-backoff-retry.test.ts still has a separate,
pre-existing bug (fake timers vs. retryWithBackoff's real setTimeout
causing a test timeout) left unfixed here as it requires debugging
failover-recovery.ts's retry logic, which is out of scope for this PR.
@solaawojobi00-bit
solaawojobi00-bit force-pushed the fix/issue-261-schema-manager-poll-diagnostics branch from 6d389fc to ca766f9 Compare August 30, 2026 03:00
@godamongstmen897 godamongstmen897 closed this pull request by merging all changes into Goldii-locks:main in 9e42e6f Aug 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Write polling diagnostics logs for sqlite_schema_manager

2 participants