fix(launcher): reap the fingerprint records the proxy leaves behind

[codeslake]

Draft. The review loop is still running on this branch; I will mark it ready when it exits clean.

What leaks

publishFingerprint(port) writes cache-fix-proxy-<port>.sha256 into the temp dir before each spawn, and nothing removed it. The port is ephemeral wherever the OS picks one, so the records accumulate one per proxy start, without bound.

Measured on a long-lived container: 6,743 records, none older than 14.6 days only because the box had not run this code that long.

That rate is not a production leak rate, and it would be wrong to read it as one. Counted by hour on a dev box, creation is bursty and tracks test runs: 1,443 of 2,145 records in a single hour, the hour a full suite ran repeatedly. What the number establishes is that nothing removes them and the directory only grows — not how fast a user's machine fills. A production host spawns a proxy when a session starts, not a thousand times an hour.

runningOurCode() calls an unreadable record ordinary because "the record lives in /tmp, which systemd-tmpfiles sweeps". That host has zsh as PID 1, no systemd, and no sweeper at all. The comment describes a deployment assumption, not a guarantee, and it is false wherever the proxy runs in a container. This PR replaces that sentence too.

Why it is worth code

Not the 284 KB. The launcher's own scratch-CA reaper walks readdirSync(tmpdir()) on every start; that scan measured 117 ms at 74,493 entries, and this litter is what fills the directory. The leak taxes the startup path that exists to clean up after it.

The shape, and why each part is the way it is

Seven days, matching SCRATCH_PREFIX's reaper. publishFingerprint has exactly one call site — the spawn path — and nothing republishes. So an unrestarted holder's record mtime is its launch time, and a shorter gate deletes a live holder's own record: runningOurCode() → null → takeOver() exits 0, which is the "swept /tmp turned a deploy into a no-op that read as a success" failure the launcher already documents. Age is not the only discriminator, and relying on it alone is what the second commit had to undo — see below.

Driven from holdPort, not from publishFingerprint. That function opens with if (!fp) return;, so reaping from there stops exactly when publishing is persistently broken — the mistake the sibling CA reaper spells out. It also runs on every respawn, on a ladder this file measures at 51 spawns in 1.2 s.

Deferred with setTimeout(..., 0).unref(). The scan is not free (135–193 ms at this box's 68,533 entries) and nothing waits on its result. Run inline it delays the bind. Interleaved against the merge base at one load:

arm	failures
this diff, scan inline	2 / 10
this diff, scan body disabled	0 / 5
merge base	0 / 14

test/proxy-held-port.test.mjs, a different case failing each time — the signature of a slower startup rather than a broken branch. unref so the timer can never hold the process open.

A port that still answers outranks the clock. Age alone made the seven-day gate a deadline, not a margin: nothing republishes a record, so a holder that neither respawns nor is redeployed for a week is fully live with an over-age record, and the next launcher deleted it. Measured end to end before the fix — a scratch TMPDIR holding an 8-day-old record for a live holder's port, one unrelated run-service start, record gone.

So a record is removed only when it is both over-age and its port does not answer. portFree() binds the port with net.createServer() on bindAddr(); a non-integer port resolves false and is kept.

Asked by binding, not by lsof. The first version of that check shelled out, and upstream CI went red on Node 18 — not because of the Node version, but because that runner had no lsof. Reproduced locally by removing lsof from PATH: 3 of 6 cases fail, the same failure CI reported. The fail-closed branch then kept every record, so the reaper became a silent no-op on any host without lsof — exactly where nobody would look for it. A bind asks the kernel directly and is also more exact: lsof sees only this uid.

Two things the bind check does NOT do, both measured and both now stated in the code: it tests LISTEN rather than ownership, so a holder in this file's deliberate bound-but-not-listening state reads as free; and the loop does not yield — 176 ms of held event loop at 3,000 over-age records.

RECORD_PREFIX shared by the writer and the reaper. The first version derived the reaper's prefix from fingerprintPath("") at runtime via lastIndexOf("/"). That is -1 where the separator differs, so the whole absolute path became the prefix and no basename from readdirSync ever matched — not destructive, but a silent no-op on a supported target.

Verified end to end

Real launcher, isolated TMPDIR, ephemeral port: an 8-day-old record removed, a fresh one kept, a neighbouring cache-fix-ca-scratch-keepme.sha256 kept, and its own record published. The deferred timer does fire.

Full suite interleaved against the merge base, 3 rounds each: 1949 pass / 0 fail on the branch, 1946 / 0 on the base. CI green on Node 18, 20 and 22.

Tests

test/proxy-fingerprint-reap.test.mjs lifts the shipped function out of the launcher and runs it with tmpdir() rebound to a scratch dir, the way proxy-held-port.test.mjs lifts holderPidOn. It asserts on every slice, so a rename fails the test instead of quietly testing nothing.

Mutation-checked, each break made fresh against the final code:

mutation	result
RECORD_PREFIX = ""	fails
RECORD_PREFIX = "cache-fix-" (one segment short)	fails — takes the neighbouring CA name
gate back to 1 day	fails — a 3-day-old record is reaped
gate Infinity	fails — the stale record survives
call deleted from holdPort	fails
call moved back into publishFingerprint	fails
fingerprintPath back to a literal	fails
reap made synchronous again	fails
port guard removed	fails — a live holder's record is deleted
non-numeric port inverted to reap	fails
lsof-failure fallback changed to an empty Set (before the bind rewrite)	fails

The negative control is named cache-fix-ca-scratch-keepme.sha256 on purpose: with any other suffix the endsWith(".sha256") check alone would save it and an empty prefix would pass.

Also carried here: a crash this branch surfaced but did not cause

test/proxy-held-port.test.mjs has seven inline health probes. Six resolve `ERR:${e.code}` — a string carrying the prefix classify() tests for. The seventh, at the forced-kill case, resolves a bare r.statusCode on success; its caller filters out 200 and hands the rest to classify(), which opens with .startsWith. A 502 arrives as a Number and the case dies.

CI Node 22 hit it on this branch, which touches neither that file nor classify(). The path only opens when a non-200 is actually observed during the forced kill — every local run and three earlier CI matrices had none.

Fix is if (typeof body !== "string") return null; plus a unit case pinning four answers (a status code is a reply, not an outage). Measured: removing the guard reds that case with the exact CI string.

It rides along rather than waiting behind its own PR because leaving this branch red would mean explaining the red in a comment and fixing the order the two had to merge in, for a five-line guard.

Anti-bloat numbers

Per AGENTS.md, stated rather than asserted:

production LOC	83 (26 code + 54 comment + 3 blank)
test LOC	230
test:production	2.8x
new files	1 (test/proxy-fingerprint-reap.test.mjs)
new exports / env vars / on-disk paths	0 / 0 / 0
comment:code on the production diff	2.08:1

The smallest change that fixes the stated defect is a ~10-line sweep loop; the port discriminator, the bind fallback and the shared prefix are the rest, and this is 26 lines of code. The comments carry why seven days, why holdPort and not publishFingerprint, and why deferred — the three things a reader would otherwise undo. The second commit deletes the investigation that had accumulated around them; the evidence lives here and in the commit messages instead.

Ref #304 — the record and the systemd-tmpfiles comment both arrived in 84ba7c2.

— Proxy Builder

🤖 Generated with Claude Code