Skip to content

Watcher: Emit a startup heartbeat before the initial directory scan - #66

Merged
wasimxyz merged 1 commit into
stagingfrom
wa/initial-scan-heartbeats
May 13, 2026
Merged

Watcher: Emit a startup heartbeat before the initial directory scan#66
wasimxyz merged 1 commit into
stagingfrom
wa/initial-scan-heartbeats

Conversation

@wasimxyz

Copy link
Copy Markdown
Member

Summary

  • HeartbeatLoop.start() now sends a synchronous status="watching" heartbeat and flushes the event reporter before spawning the loop thread. Without this, the loop's first Event.wait(timeout=interval) deferred the first beat by a full 60s — and the just-queued WATCHER_STARTED event with it — so the dashboard saw nothing for the entire initial scan window. On headless Windows services that made the watcher indistinguishable from a hung process.
  • Bumps the watcher to 0.2.7 (lockfile regenerated via uv sync).

Why this is the minimum-blast-radius fix

  • The API route at web-app/app/api/v1/watchers/[watcherId]/heartbeat/route.ts and the watcher_status Postgres enum in web-app/lib/db/schema.ts only accept ["registered", "watching", "stopped"]. A new status ("starting" / "scanning") would have required a DB migration + dashboard work for no signal the existing "watching" doesn't already convey at this point — the watcher process is up and intends to watch; the initial scan is part of normal startup.
  • For very long scans the existing daemon heartbeat thread already fires every 60s (the scan's path.stat() / path.iterdir() calls release the GIL). The only gap was the initial silence — closed by the synchronous startup beat.
  • _send_heartbeat and EventReporter.flush() both swallow network errors internally, so a startup with the API unreachable still proceeds; the loop thread starts and the first successful tick emits the standard heartbeat_recovered event.

Test plan

  • New TestHeartbeatStartupBeat class in watcher/tests/test_heartbeat.py covering:
    • Immediate status="watching" heartbeat is sent on start().
    • EventReporter.flush() is invoked from the main thread before the loop thread becomes alive (so WATCHER_STARTED ships before the initial scan).
    • A startup-time ApiError does not prevent the loop thread from spawning, and the failure is recorded so the next successful tick emits heartbeat_recovered.
    • End-to-end timing guard: with a 1s interval, the first heartbeat lands in well under 100ms — i.e. it does not go through the loop's interval wait.
  • uv run pytest watcher/tests/test_heartbeat.py watcher/tests/test_runtime.py -v → 40/40 pass (4 new + 36 pre-existing).
  • make check-all → ruff, ruff-format, pyright, prettier, eslint, tsc all clean.
  • Manual verification on a Windows lab PC: install the new watcher as a service, restart it, confirm a heartbeat + WATCHER_STARTED event appear in Data Hub within seconds (not after the previous ~60s gap).

Made with Cursor

`HeartbeatLoop.start()` now sends a synchronous `status="watching"`
heartbeat and flushes the event reporter before spawning the loop
thread. Without this, the loop's first `Event.wait(timeout=interval)`
deferred the first beat by a full 60s — and the just-queued
`WATCHER_STARTED` event with it — so the dashboard saw nothing for
the entire initial scan window. On headless Windows services that
made the watcher indistinguishable from a hung process.

Bumps the watcher to 0.2.7.

Co-authored-by: Cursor <cursoragent@cursor.com>
@vercel

vercel Bot commented May 13, 2026

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
data-hub Ready Ready Preview, Comment May 13, 2026 7:28pm

Request Review

@wasimxyz
wasimxyz marked this pull request as ready for review May 13, 2026 19:29
@wasimxyz wasimxyz self-assigned this May 13, 2026
@wasimxyz
wasimxyz merged commit d180548 into staging May 13, 2026
4 checks passed
@wasimxyz
wasimxyz deleted the wa/initial-scan-heartbeats branch May 13, 2026 19:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant