Skip to content

ops: heartbeat v2 — programmatic health checks, zero tokens when healthy#127

Merged
benvinegar merged 4 commits into
mainfrom
baudbot/heartbeat-v2-zero-token
Feb 23, 2026
Merged

ops: heartbeat v2 — programmatic health checks, zero tokens when healthy#127
benvinegar merged 4 commits into
mainfrom
baudbot/heartbeat-v2-zero-token

Conversation

@baudbot-agent

Copy link
Copy Markdown
Collaborator

Summary

Rewrote heartbeat.ts to perform all health checks in pure Node.js — zero LLM tokens consumed when everything is healthy (which is ~95% of the time).

Before (v1)

  • Every 10 minutes: inject the full HEARTBEAT.md checklist as an LLM prompt
  • Agent reads checklist, runs tool calls (list_sessions, curl), responds
  • Cost: ~$0.50/heartbeat on Opus → ~$72/day
  • Context bloat: each heartbeat adds ~800 tokens to conversation history

After (v2)

  • Every 10 minutes: extension runs health checks in Node.js (no LLM)
  • Only injects a prompt when something fails, describing just the failures
  • Cost: $0/day when healthy, ~$0.50 per incident
  • Context stays lean — no routine heartbeat messages

Checks performed (all in Node.js)

  1. Session liveness — verify .alias symlinks resolve to live .sock files
  2. Slack bridge — HTTP POST to localhost:7890/send returns 400
  3. Stale worktrees — dirs in ~/workspace/worktrees/ with no matching in-progress todo
  4. Stuck todosin-progress for >2h with no matching dev-agent session
  5. Orphaned dev-agentsdev-agent-* sessions with no matching todo

Configuration

  • HEARTBEAT_EXPECTED_SESSIONS — comma-separated aliases to check (default: sentry-agent)
  • HEARTBEAT_INTERVAL_MS / HEARTBEAT_ENABLED — unchanged from v1

Tests

  • Updated test file with 50 tests covering v2 functions (parseTodo, clampInt, getExpectedSessions, etc.)
  • All passing

@greptile-apps

greptile-apps Bot commented Feb 22, 2026

Copy link
Copy Markdown

Greptile Summary

Rewrote heartbeat extension to perform health checks programmatically in Node.js instead of consuming LLM tokens. When all 5 checks pass (session liveness, Slack bridge, stale worktrees, stuck todos, orphaned dev-agents), zero tokens are consumed. Only injects a targeted prompt when failures are detected.

Major changes:

  • Removed HEARTBEAT.md file dependency and readHeartbeatFile() function
  • Added 5 programmatic checks: checkSessions(), checkBridge(), checkWorktrees(), checkStuckTodos(), plus orphaned dev-agent detection
  • New config: HEARTBEAT_EXPECTED_SESSIONS (default: sentry-agent)
  • Added parseTodo() function to parse JSON front matter from todo files
  • Updated 50 tests to cover v2 logic (removed v1 file-reading tests)
  • Updated SKILL.md documentation with v2 behavior

Issues found:

  • parseTodo() JSON parser doesn't handle braces inside string values - will break on todos like {"title": "Fix {bug}"}
  • hoursStuck calculation has parentheses in wrong place - computes (age / 3600000 * 10) / 10 instead of ((age / 3600000) * 10) / 10

Token savings: $72/day → $0/day when healthy ($0.50 per incident)

Confidence Score: 4/5

  • Safe to merge with two logic bugs fixed - zero-token design is sound
  • Two logic bugs found: JSON parser can't handle braces in strings (will fail on real todos), and hoursStuck calculation has misplaced parentheses. Both are fixable before merge. The core architectural change is excellent.
  • Fix parseTodo JSON parser (line 210) and hoursStuck calculation (line 260) in pi/extensions/heartbeat.ts

Important Files Changed

Filename Overview
pi/extensions/heartbeat.ts Complete rewrite from LLM-based to programmatic health checks. Implements 5 checks (sessions, bridge, worktrees, stuck todos, orphans) with zero-token consumption when healthy. Minor issue: parseTodo doesn't handle strings inside JSON objects with braces.
pi/extensions/heartbeat.test.mjs Tests updated to cover v2 functions: clampInt, getExpectedSessions, parseTodo, stuck todo detection. Removed old v1 tests. Comprehensive coverage with 50+ tests for new programmatic checks.
pi/skills/control-agent/SKILL.md Documentation updated to describe v2 heartbeat behavior - programmatic checks with zero tokens when healthy, targeted prompts only on failures. Adds orphaned dev-agent check documentation.

Last reviewed commit: 2b2db3f

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

3 files reviewed, 2 comments

Edit Code Review Agent Settings | Greptile

Comment on lines +210 to +221
let depth = 0;
let jsonEnd = -1;
for (let i = 0; i < trimmed.length; i++) {
if (trimmed[i] === "{") depth++;
else if (trimmed[i] === "}") {
depth--;
if (depth === 0) {
jsonEnd = i + 1;
break;
}
}
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

JSON parsing doesn't handle escaped braces in strings - will fail on todos with { or } in string values

Suggested change
let depth = 0;
let jsonEnd = -1;
for (let i = 0; i < trimmed.length; i++) {
if (trimmed[i] === "{") depth++;
else if (trimmed[i] === "}") {
depth--;
if (depth === 0) {
jsonEnd = i + 1;
break;
}
}
}
// Find the closing brace for the top-level JSON object
let depth = 0;
let jsonEnd = -1;
let inString = false;
let escapeNext = false;
for (let i = 0; i < trimmed.length; i++) {
const char = trimmed[i];
if (escapeNext) {
escapeNext = false;
continue;
}
if (char === '\\') {
escapeNext = true;
continue;
}
if (char === '"') {
inString = !inString;
continue;
}
if (!inString) {
if (char === "{") depth++;
else if (char === "}") {
depth--;
if (depth === 0) {
jsonEnd = i + 1;
break;
}
}
}
}
Prompt To Fix With AI
This is a comment left during a code review.
Path: pi/extensions/heartbeat.ts
Line: 210-221

Comment:
JSON parsing doesn't handle escaped braces in strings - will fail on todos with `{` or `}` in string values

```suggestion
    // Find the closing brace for the top-level JSON object
    let depth = 0;
    let jsonEnd = -1;
    let inString = false;
    let escapeNext = false;
    for (let i = 0; i < trimmed.length; i++) {
      const char = trimmed[i];
      if (escapeNext) {
        escapeNext = false;
        continue;
      }
      if (char === '\\') {
        escapeNext = true;
        continue;
      }
      if (char === '"') {
        inString = !inString;
        continue;
      }
      if (!inString) {
        if (char === "{") depth++;
        else if (char === "}") {
          depth--;
          if (depth === 0) {
            jsonEnd = i + 1;
            break;
          }
        }
      }
    }
```

How can I resolve this? If you propose a fix, please make it concise.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch — the naive brace counter broke on strings like "Fix {bug}". Fixed in bd63682: the scanner now tracks inString/escapeNext state to skip braces inside quoted values. Added 3 tests covering braces-in-strings and escaped quotes.

Comment thread pi/extensions/heartbeat.ts Outdated

if (!hasAgent) {
const title = todo.title || todoId;
const hoursStuck = Math.round(age / (60 * 60 * 1000) * 10) / 10;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Parentheses placed incorrectly - dividing by milliseconds after multiplying by 10

Suggested change
const hoursStuck = Math.round(age / (60 * 60 * 1000) * 10) / 10;
const hoursStuck = Math.round((age / (60 * 60 * 1000)) * 10) / 10;
Prompt To Fix With AI
This is a comment left during a code review.
Path: pi/extensions/heartbeat.ts
Line: 260

Comment:
Parentheses placed incorrectly - dividing by milliseconds after multiplying by 10

```suggestion
          const hoursStuck = Math.round((age / (60 * 60 * 1000)) * 10) / 10;
```

How can I resolve this? If you propose a fix, please make it concise.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in bd63682. The original code happened to produce the correct result (JS evaluates age / 3600000 * 10 left-to-right since / and * are same precedence), but the explicit parens make the intent unambiguous.

Comment thread pi/extensions/heartbeat.ts Outdated
@baudbot-agent

Copy link
Copy Markdown
Collaborator Author

Addressed the hasMatchingInProgressTodo false-positive bug in 799b63d.

Problem: content.includes(worktreeName) was a naive substring match — a worktree named fix would match any todo containing "fix" anywhere, preventing stale worktree detection.

Fix: Now uses two-pronged matching:

  1. Path pattern: checks for worktrees/<name> (the stored path format from dev-agent spawning)
  2. Word boundary regex: matches the name preceded/followed by whitespace, quotes, backticks, or line boundaries — so fix/sentry-alert in prose matches, but prefix or Refixing does not.

Added 7 tests covering path context, boundary matching, backtick/quote contexts, and false-positive rejection. 60 tests total, all passing.

Comment thread pi/extensions/heartbeat.ts
@baudbot-agent

Copy link
Copy Markdown
Collaborator Author

Fixed in ea2a701.

Bug: The catch block in checkSessions silently reported ok: true when readlinkSync threw — masking broken sessions (e.g. alias file isn't a symlink (EINVAL), permission denied (EACCES), or other I/O errors).

Fix: Now catches the error and reports ok: false with the specific error message, so broken symlinks surface in the heartbeat alert and get escalated.

Baudbot added 4 commits February 22, 2026 18:37
Rewrote heartbeat.ts to perform all health checks in pure Node.js:
- Session liveness (alias + socket existence)
- Slack bridge responsiveness (HTTP 400 check)
- Stale worktrees (cross-ref with in-progress todos)
- Stuck todos (in-progress >2h with no dev-agent)
- Orphaned dev-agent sessions

Only injects an LLM prompt when something actually fails.
Saves ~$72/day on Opus vs the old approach that injected the
full HEARTBEAT.md checklist every 10 minutes regardless of health.

Updated tests to cover v2 functions (parseTodo, clampInt,
getExpectedSessions, etc.). All 50 tests pass.

Updated SKILL.md to document new heartbeat behavior.
- parseTodo JSON scanner now tracks inString/escapeNext state so
  braces inside string values (e.g. "Fix {bug}") don't break parsing
- Added explicit parens to hoursStuck for clarity (same result, safer)
- Added 3 new tests for braces-in-strings and escaped quotes (53 total)
hasMatchingInProgressTodo used content.includes(worktreeName) which
caused false positives — a worktree named 'fix' would match any todo
containing the word 'fix' anywhere in its body. Stale worktrees would
never be detected and cleaned up.

Now checks for:
1. Path pattern: 'worktrees/<name>' (the stored path format)
2. Word boundary: name preceded/followed by whitespace, quotes, or
   backticks (catches 'fix/branch-name' in prose)

Added 7 tests for boundary matching (60 total).
The catch block in checkSessions silently reported ok:true when
readlinkSync threw (EINVAL, EACCES, etc.), masking broken sessions.
Now reports ok:false with the error message so failures surface
in the heartbeat alert.
@benvinegar benvinegar force-pushed the baudbot/heartbeat-v2-zero-token branch from ea2a701 to 53ca277 Compare February 22, 2026 23:37
@benvinegar benvinegar merged commit 27cc89c into main Feb 23, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants