Skip to content

Fix: Pulse crash-loops when any unwatchable node exists under LIFEOS/ - #1683

Open
lmbagley wants to merge 1 commit into
danielmiessler:mainfrom
lmbagley:fix/pulse-watcher-crash-on-unwatchable-node
Open

Fix: Pulse crash-loops when any unwatchable node exists under LIFEOS/#1683
lmbagley wants to merge 1 commit into
danielmiessler:mainfrom
lmbagley:fix/pulse-watcher-crash-on-unwatchable-node

Conversation

@lmbagley

Copy link
Copy Markdown

Problem

startWatchers() in the wiki module watches LIFEOS_DIR recursively. A recursive watch walks every node beneath its root, and some cannot be opened — unix sockets, FIFOs, dead symlinks, unreadable directories. Those surface as an asynchronous 'error' event on the watcher. The surrounding try/catch only sees synchronous throws, so the event goes unhandled and terminates the process.

Under a supervisor, Pulse then crash-loops for as long as the node exists. Because the walk happens during startup, it presents as "Pulse won't start" rather than "one file is unwatchable", and the only clue is an ENXIO naming a file with nothing to do with the wiki:

error: ENXIO: no such device or address, open '.../MEMORY/<tool>/<name>.sock'
      at emitError (node:events:51:13)
      at #onEvent (node:fs:47:16)

It is easy to miss because it is latent. A long-running Pulse is unaffected when the socket appears — the walk already happened. It only bites on the next restart, which can be weeks later and looks unrelated to whatever created the file. Any tool that puts a socket under LIFEOS/MEMORY/ (a bot, an IPC helper, an editor server) is enough to trigger it.

Changes

1. Narrow the watch roots to what the reindexer actually consumes — DOCUMENTATION_DIR, KNOWLEDGE_DIR, ALGORITHM_DIR.

Watching all of LIFEOS_DIR was simultaneously too broad and too narrow:

  • Too broadMEMORY/ is the busiest subtree in an install (constant .jsonl appends) and the place where non-regular files appear.
  • Too narrowskills/ and hooks/ live under ~/.claude, not under LIFEOS/, so changes there were never watched anyway.

This removes the cause and cuts a lot of pointless wakeup churn.

2. Attach an 'error' handler so any future unopenable node degrades to "no live reindex" instead of taking Pulse down. The comment in the existing catch already says watching is best-effort; this makes that true.

The same async-error gap exists in user-index.ts, which watches USER_DIR with an identical try/catch, so it gets the same handler. Its initial scan has already built the index, so losing live updates is a soft degradation.

Verification

Linux, bun 1.3.13:

  • Before — with a socket present under MEMORY/, Pulse crash-looped and /healthz never came up.
  • After — starts clean, logs the watcher warning once when applicable, serves normally.
  • No new typecheck errors in either file (baseline error counts identical before/after: wiki 15→15, user-index 18→18; those are module-resolution noise from checking files in isolation).

Notes

Both changes are independent — the narrowing prevents the crash, the handler survives it. If you would rather take only one, the handler alone is the minimal fix, though the broad watch root is the underlying cause.

`startWatchers()` in the wiki module watches `LIFEOS_DIR` recursively. A recursive
watch walks every node beneath its root, and some cannot be opened — unix sockets,
FIFOs, dead symlinks, unreadable directories. Those surface as an asynchronous
'error' event on the watcher. The surrounding `try/catch` only sees synchronous
throws, so the event goes unhandled and terminates the process.

Under a supervisor Pulse then crash-loops for as long as the node exists. Because
the walk happens during startup, it presents as "Pulse won't start" rather than
"one file is unwatchable", and the only clue is an ENXIO naming a file that has
nothing to do with the wiki:

    error: ENXIO: no such device or address, open '.../MEMORY/<tool>/<name>.sock'
      at emitError (node:events:51:13)
      at #onEvent (node:fs:47:16)

It is easy to miss because it is latent: a long-running Pulse is unaffected when
the socket appears, since the walk already happened. It only bites on the next
restart, which can be weeks later and looks unrelated to whatever created the file.
Any tool that puts a socket under LIFEOS/MEMORY/ — a bot, an IPC helper, an editor
server — is enough to trigger it.

Two changes:

1. Narrow the watch roots to what the reindexer actually consumes
   (DOCUMENTATION, KNOWLEDGE, ALGORITHM). Watching all of LIFEOS_DIR was both too
   broad and too narrow — too broad because MEMORY/ is the busiest subtree in an
   install and the place where non-regular files appear, too narrow because
   skills/ and hooks/ live under ~/.claude rather than LIFEOS/, so changes there
   were never watched anyway. This removes the cause and cuts a large amount of
   pointless wakeup churn from .jsonl appends.

2. Attach an 'error' handler to the watcher so any future unopenable node degrades
   to "no live reindex" instead of taking Pulse down. The comment in the existing
   catch already says watching is best-effort; this makes that true.

The same async-error gap exists in `user-index.ts`, which watches USER_DIR with an
identical `try/catch`, so it gets the same handler. Its initial scan has already
built the index, so losing live updates is a soft degradation.

Verified on Linux (bun 1.3.13): before, Pulse crash-looped with a socket present
and `/healthz` never came up; after, it starts clean, logs the watcher warning once
when applicable, and serves normally. No new typecheck errors in either file.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants