Skip to content

hackbrowser: worker has no crash safety net — one uncaught throw kills the whole crawl #117

Description

@badchars

Summary

The hackbrowser worker subprocess has no process-level crash safety net. Any uncaught exception or unhandled promise rejection anywhere in the worker terminates the process (exit 1) and kills the entire multi-page crawl. #116 demonstrated the impact: one malformed element on one page (an invalid label[for] selector) threw inside page.evaluate, the rejection was unhandled, and a 50-page crawl died at page 20. Point-fixing each source (as #116 did) is whack-a-mole; the worker needs a safety net so no single stray throw can abort the whole crawl.

Root cause

Fix

Part 1 — source guard (agent.ts): attach a detached .catch() when storing the pendingUI promise, so a rejection can't go unhandled. The consumer still awaits the original promise inside its existing try/catch, so behavior is unchanged.

Part 2 — safety net (hackbrowser-worker.ts): install global handlers at worker startup:

  • unhandledRejection → log (via the IPC "log" message) and CONTINUE. Such a rejection escaped every await, so the main runCrawl chain never depended on it — swallowing it is safe and lets the crawl run to natural completion.
  • uncaughtException → log and continue, with a circuit breaker: if more than N exceptions occur within a short window, emit a terminal error and exit (a genuinely corrupted browser/session), preserving the captures already streamed to the parent.

Why "log + continue" is sufficient

Captured requests are ingested to the parent live during the crawl, so continuing loses no data. The main BFS loop already has per-page try/catch and browser-death detection, so it terminates cleanly on real failure. The handler needs no access to runCrawl internals.

Not in scope (separate)

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions