diff --git a/CHANGELOG.md b/CHANGELOG.md index 15fe89d..3e1e220 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,72 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +## [0.3.0] - 2026-07-19 + +### Added + +- `gemcatch export [--tag ] [--status ] [--format md|json] [-o ]` — + concatenate finished results into one document, each under a heading with its + prompt, id and date. Markdown by default (or JSON for `jq`), to stdout or a + file. This is the "gather" step that pairs with `batch`'s "scatter": where + `get` prints one result at a time, `export` collects a whole tag at once. +- `gemcatch digest --tag ` — feed a tag's completed results back through a + single Gemini call to synthesise one summary. Submits like `research` and + watches to completion; the summary lands under `-digest`. +- `GEMCATCH_WATCH_MAX_FAILS` (default 10) — the consecutive-poll-failure bound + at which `watch` and `batch -w` give up rather than loop forever. + +### Fixed + +- `research --watch` no longer marks a **successfully submitted, server-running** + task `failed` when a poll errors during the watch. A transient poll failure (a + 5xx past its retries, a network blip) or an expiry mid-watch would propagate to + the submit handler and overwrite the status to `failed`, dropping the task from + the active set so the daemon abandoned it and the result was lost. The watch + loop now rides out poll errors (retrying on the next interval), and only a + failed *submit* — a task with no `interaction_id` yet — is ever marked `failed`. +- A wedged or expired interaction no longer keeps a task in flight forever. When a + poll returns **404** (the free tier drops interactions after 24h, or one was + deleted), the task is retired locally to `incomplete` with a recorded reason, so + it leaves the active set and `daemon --exit-when-idle` converges. `watch` and + `batch -w` additionally stop after `GEMCATCH_WATCH_MAX_FAILS` consecutive poll + failures (or a stalled batch), with a clear message and a non-zero exit, instead + of spinning. +- A **completed-but-empty** result is now served from the local cache. `get` and + `watch` gated the cache hit on the result being truthy, so a task that completed + with empty text (`''`) skipped the cache, re-polled, and 404'd after 24h — the + exact loss the daemon exists to prevent. The gate is now on presence + (`result != null`), not truthiness. +- `prune -d ` (or a non-numeric `--days`) is rejected instead of putting + the cutoff in the future and deleting **every** finished task. `--days` must now + be a non-negative number. +- `batch` no longer silently drops a prompt line that starts with `#`. A `#` is a + comment only when followed by whitespace (`# like this`); a line such as + `#1 cause of X?` is a real prompt and survives. When comment or blank lines are + skipped, a one-line count is noted on stderr. +- `watch -i` / `daemon -i` reject a non-positive interval (`-i -5` busy-looped, + `-i 0` silently fell back to the default). `list -n 0` now returns zero rows + instead of all of them, and `-n` rejects negatives (which SQLite reads as "no + limit"). +- A second `Ctrl-C` to the daemon now force-exits (130) instead of doing nothing + while a long paced pass finishes. +- One-shot commands close the SQLite store on exit, so they no longer leave + `-wal`/`-shm` sidecar files lingering next to `tasks.db`. +- Colour written to **stderr** (the status chatter from `watch`, `daemon` and + `research -w`) is now keyed to `process.stderr.isTTY`, not stdout's. Redirecting + one stream no longer strips colour from the other, nor leaks raw ANSI into a + redirected file. + +### Notes + +- The default `@google/genai` SDK transport is now covered by the offline suite + (previously every test forced `GEMCATCH_FORCE_REST=1`): a stubbed client drives + submit → poll → completed and one unwrapped SDK error, confirming `shape()` + reads an SDK-shaped response and `friendly()` surfaces Google's real message. +- `GEMCATCH_RPM` pacing is **per process**. Two concurrent `gemcatch` processes + each keep their own counter and can together exceed the ceiling; run a single + daemon if the limit must hold. Documented in the README. + ## [0.2.0] - 2026-07-19 ### Added @@ -96,7 +162,8 @@ seen a task complete, the text is cached locally and survives that expiry — bu something has to poll inside that window for it to be seen at all, which is what `gemcatch daemon` exists to do. -[Unreleased]: https://github.com/Booyaka101/gemcatch/compare/v0.2.0...HEAD +[Unreleased]: https://github.com/Booyaka101/gemcatch/compare/v0.3.0...HEAD +[0.3.0]: https://github.com/Booyaka101/gemcatch/compare/v0.2.0...v0.3.0 [0.2.0]: https://github.com/Booyaka101/gemcatch/compare/v0.1.1...v0.2.0 [0.1.1]: https://github.com/Booyaka101/gemcatch/compare/v0.1.0...v0.1.1 [0.1.0]: https://github.com/Booyaka101/gemcatch/releases/tag/v0.1.0 diff --git a/README.md b/README.md index 60692f3..7124c16 100644 --- a/README.md +++ b/README.md @@ -83,6 +83,8 @@ This week in AI: ... | `gemcatch status ` | Polls the API and prints the current state. | | `gemcatch get ` | Prints the full response if complete, otherwise the current status. | | `gemcatch list` | All tasks, newest first: id, age, status, prompt. | +| `gemcatch export` | Concatenates finished results, each under its prompt, to stdout or a file (Markdown or JSON). | +| `gemcatch digest` | Feeds a tag's completed results through one Gemini call into a single summary. | | `gemcatch watch ` | Polls until the task finishes, then prints the result. | | `gemcatch sync` | Refreshes every in-flight task in one pass. | | `gemcatch daemon` | Keeps polling in-flight tasks on a loop, so results are cached before they expire. | @@ -102,9 +104,12 @@ Useful flags: | `-t, --tag ` | `research`, `batch`, `list` | Label tasks and filter them. | | `-w, --watch` | `research`, `batch` | Submit and wait, in one command. | | `--separator ` | `batch` | Split the file on this delimiter line for multi-line prompts. | -| `-i, --interval ` | `watch`, `daemon` | Poll rate. Default 10s for `watch`, 300s for `daemon`. | +| `-i, --interval ` | `watch`, `daemon` | Poll rate in seconds; must be > 0. Default 10s for `watch`, 300s for `daemon`. | | `--exit-when-idle` | `daemon` | Stop once nothing is left in flight. | -| `-n, --limit ` | `list` | Cap the rows. | +| `--status ` | `list`, `export` | Only tasks in this status. | +| `-n, --limit ` | `list` | Cap the rows (non-negative; `0` shows none). | +| `--format ` | `export` | Output format. Default `md`. | +| `-o, --out ` | `export` | Write to a file instead of stdout. | | `--dry-run` | `batch`, `prune` | Show what would go; submit/delete nothing. | | `--raw` | `get` | Dump the raw interaction JSON. | @@ -117,10 +122,15 @@ Statuses come straight from the API: `in_progress`, `requires_action`, `complete ```bash # Fire off a whole file of prompts in one command, then collect later. # Every task shares one auto-generated tag (batch-xxxxxx), printed on submit. -$ gemcatch batch questions.txt # one prompt per line; # and blanks skipped +$ gemcatch batch questions.txt # one prompt per line; "# " and blanks skipped $ gemcatch daemon --exit-when-idle # keep polling until they're all in $ gemcatch list --tag batch-1a2b3c --status completed +# Collect a whole batch into one document (the "gather" for batch's "scatter"). +$ gemcatch export --tag batch-1a2b3c -o results.md # Markdown, one section per prompt +$ gemcatch export --tag batch-1a2b3c --format json | jq -r '.[].result' +$ gemcatch digest --tag batch-1a2b3c # or synthesize them into one summary + # Multi-line prompts: split the file on a delimiter line instead of per-line $ gemcatch batch briefs.md --separator --- $ gemcatch batch - < questions.txt # or pipe the list in on stdin @@ -182,10 +192,14 @@ $ gemcatch daemon --exit-when-idle -i 30 $ gemcatch list --tag batch1 --status completed ``` +If a task's interaction has vanished server-side — the free tier dropped it after 24h, or it was deleted — polling it returns a 404. Rather than chase a task that can never resolve, `gemcatch` retires it locally to `incomplete`, so it leaves the in-flight set and `--exit-when-idle` still converges. `watch` and `batch -w` also give up after a bounded run of consecutive poll failures (`GEMCATCH_WATCH_MAX_FAILS`, default 10) instead of looping forever. + ## Rate limits and retries The free tier allows roughly 15 requests a minute, which a wide `gemcatch sync` or a busy daemon would otherwise blow straight through. Every outbound call is paced to `GEMCATCH_RPM` (default 15) — set it higher on a paid key, or `0` to disable pacing entirely. +Pacing is **per process**: each `gemcatch` invocation keeps its own counter, so two running at once (a `daemon` in one terminal and a one-off `sync` in another) can together exceed the ceiling. If you need the limit to hold, run a single daemon and let it do the polling. + Transient failures are retried with exponential backoff and full jitter, honouring `Retry-After` when the server sends it. A rate limit, a timeout or a 5xx gets `GEMCATCH_MAX_RETRIES` more attempts (default 4); a 4xx does not, because a bad key or a bad model id fails identically forever and retrying it only burns your quota. ## Environment variables @@ -199,6 +213,7 @@ Transient failures are retried with exponential backoff and full jitter, honouri | `GEMCATCH_DAEMON_S` | `daemon` interval in seconds. Default `300`. | | `GEMCATCH_RPM` | Requests/minute ceiling. Default `15` (the free tier). `0` disables pacing. | | `GEMCATCH_MAX_RETRIES` | Extra attempts on a transient failure. Default `4`. `0` disables retries. | +| `GEMCATCH_WATCH_MAX_FAILS` | Consecutive poll failures before `watch`/`batch -w` give up. Default `10`. | | `GEMCATCH_BASE_URL` | Override the API endpoint (proxy/gateway/testing). | | `GEMCATCH_FORCE_REST` | `1` bypasses the SDK and uses raw `fetch`. | | `NO_COLOR` | Disable colour output. | diff --git a/db.js b/db.js index 1c1ef0e..77b8c6b 100644 --- a/db.js +++ b/db.js @@ -125,7 +125,10 @@ function listTasks(opts) { let sql = 'SELECT * FROM tasks'; if (where.length) sql += ` WHERE ${where.join(' AND ')}`; sql += ' ORDER BY created_at DESC'; - if (o.limit) { + // Presence, not truthiness: `--limit 0` is a real cap (return nothing), so it + // must not be treated the same as "no limit given". The caller validates that + // it is a non-negative integer before we get here. + if (o.limit != null) { sql += ' LIMIT @limit'; params.limit = o.limit; } diff --git a/index.js b/index.js index e06539f..fabe923 100644 --- a/index.js +++ b/index.js @@ -13,20 +13,41 @@ const DEFAULT_POLL_MS = Number(process.env.GEMCATCH_POLL_MS) || 10000; // comfortably faster than that. Five minutes is far inside the margin and // costs a handful of requests an hour. const DEFAULT_DAEMON_S = Number(process.env.GEMCATCH_DAEMON_S) || 300; +// A watch loop must not spin forever on a task the server can no longer resolve +// -- a wedged in_progress, or transient poll errors that never clear. `watch` +// and `batch -w` give up after this many *consecutive* poll failures (a clean +// poll resets the run), surfacing a clear message and a non-zero exit instead +// of hanging. The daemon, meant to run for days, is bounded differently: a 404 +// retires the task locally (see refresh) so it simply leaves the active set. +const WATCH_MAX_FAILS = Number(process.env.GEMCATCH_WATCH_MAX_FAILS) || 10; const ALL_STATUSES = [PENDING].concat(ACTIVE, TERMINAL); // --- output --------------------------------------------------------------- -const useColor = process.stdout.isTTY && !process.env.NO_COLOR; -const paint = (code, s) => (useColor ? `[${code}m${s}` : s); +// Colour is decided per stream. Progress and status chatter go to stderr +// (watch/daemon/research -w); results and tables go to stdout. Each stream keys +// its ANSI on its *own* TTY-ness, so redirecting one (`gemcatch watch x > out.txt`) +// neither strips colour from the other nor leaks raw escape codes into the +// redirected file. NO_COLOR disables both. +const NO_COLOR = !!process.env.NO_COLOR; +const useColor = process.stdout.isTTY && !NO_COLOR; // stdout-bound colour +const useColorErr = process.stderr.isTTY && !NO_COLOR; // stderr-bound colour + +const wrap = (on) => (code, s) => (on ? `[${code}m${s}` : s); +const paint = wrap(useColor); // paints for stdout +const epaint = wrap(useColorErr); // paints for stderr const dim = (s) => paint('2', s); - -function colorStatus(s) { - if (isSuccess(s)) return paint('32', s); // green - if (s === 'in_progress' || s === PENDING) return paint('36', s); // cyan - if (s === 'requires_action') return paint('33', s); // yellow - return paint('31', s); // red: failed/cancelled/incomplete/budget_exceeded +const edim = (s) => epaint('2', s); + +// Status colour keyed to a given painter, so one rule set serves both streams. +function tint(pnt, s) { + if (isSuccess(s)) return pnt('32', s); // green + if (s === 'in_progress' || s === PENDING) return pnt('36', s); // cyan + if (s === 'requires_action') return pnt('33', s); // yellow + return pnt('31', s); // red: failed/cancelled/incomplete/budget_exceeded } +const colorStatus = (s) => tint(paint, s); // for stdout +const ecolorStatus = (s) => tint(epaint, s); // for stderr const hhmmss = () => new Date().toISOString().slice(11, 19); @@ -100,7 +121,22 @@ async function resolvePrompt(arg, opts) { // Poll one task and persist whatever came back. async function refresh(task) { if (!task.interaction_id) return { status: task.status, text: null, usage: null }; - const r = await gemini.poll(task.interaction_id); + let r; + try { + r = await gemini.poll(task.interaction_id); + } catch (err) { + // A 404 is genuine and permanent: the interaction is gone -- dropped after + // the free tier's 24h retention, or deleted -- and it will 404 identically + // forever (a 4xx never retries). Retire the task locally so it leaves the + // active set, instead of the daemon or a watch loop polling a ghost until + // the end of time. Any other error (5xx, network) is transient and is + // re-thrown for the caller to retry on its next pass. + if (err && err.httpStatus === 404) { + store.setStatus(task.id, 'incomplete', { error: 'interaction not found (expired or deleted)' }); + return { status: 'incomplete', text: null, usage: null, raw: null }; + } + throw err; + } const extra = {}; if (isDone(r.status)) { if (isSuccess(r.status)) extra.result = r.text; @@ -162,7 +198,7 @@ program // Under --watch the submit line is progress, not the answer, so it // goes to stderr -- `gemcatch research -w "..." > out.txt` then captures // only the result. - if (!opts.json) console.error(dim(`Task ${id} submitted.`)); + if (!opts.json) console.error(edim(`Task ${id} submitted.`)); await watchTask(store.getTask(id), DEFAULT_POLL_MS, opts.json); return; } @@ -170,16 +206,29 @@ program console.log(`Task ${id} submitted. Run: gemcatch get ${id} when ready.`) ); } catch (err) { - if (id) store.setStatus(id, 'failed', { error: err.message }); + // Only a failed *submit* should mark the task failed. Once it has an + // interaction_id it is live on the server, and a later watch/poll error + // must never overwrite it to failed -- that would drop it from the active + // set and the daemon would abandon a task whose result is still coming. + // Leave it active; the daemon (or a later `get`) collects it. + if (id) { + const t = store.getTask(id); + if (!t || !t.interaction_id) store.setStatus(id, 'failed', { error: err.message }); + } die(err); } }); // --- batch ---------------------------------------------------------------- -// Turn a prompts file into a list of prompts. Default: one per line, skipping -// blank lines and `#` comments. With --separator, split the whole file on that +// Turn a prompts file into a list of prompts, plus a count of the lines it +// dropped so the caller can note them. Default: one per line, skipping blank +// lines and `#` comments. With --separator, split the whole file on that // delimiter line instead, so a single prompt can span multiple lines. +// +// A `#` is a comment only when followed by whitespace (`# like this`). A line +// such as `#1 cause of X?` is a real prompt, not a comment, and must survive -- +// treating every leading `#` as a comment silently swallowed those. function parsePrompts(text, separator) { if (separator) { const blocks = []; @@ -193,12 +242,20 @@ function parsePrompts(text, separator) { } } blocks.push(cur.join('\n').trim()); - return blocks.filter(Boolean); + const prompts = blocks.filter(Boolean); + return { prompts, skipped: blocks.length - prompts.length }; + } + // The file almost always ends in a newline; that trailing empty line is not a + // blank the user wrote, so it does not count towards the skipped tally. + const lines = text.split(/\r?\n/).map((l) => l.trim()); + while (lines.length && lines[lines.length - 1] === '') lines.pop(); + const prompts = []; + let skipped = 0; + for (const l of lines) { + if (!l || /^#\s/.test(l)) skipped += 1; + else prompts.push(l); } - return text - .split(/\r?\n/) - .map((l) => l.trim()) - .filter((l) => l && !l.startsWith('#')); + return { prompts, skipped }; } // Poll just this batch until nothing tagged with it is still in flight, then @@ -207,11 +264,25 @@ function parsePrompts(text, separator) { async function watchBatch(tag, intervalMs, json) { const inFlight = () => store.listTasks({ tag }).filter((t) => t.interaction_id && !isDone(t.status)); let pending = inFlight(); + let stalls = 0; // consecutive passes that resolved nothing while (pending.length) { // A poll that throws keeps the task's old status; the next pass retries it. + // A 404 retires the task inside refresh, so it drops out of `inFlight`. await mapLimit(pending, 4, (t) => refresh(t).catch(() => {})); - pending = inFlight(); - if (pending.length) await new Promise((r) => setTimeout(r, intervalMs)); + const next = inFlight(); + // Forward progress = the in-flight set shrank. A pass that resolves nothing + // -- every poll erroring, or a wedged in_progress that never moves -- is a + // stall; enough of those in a row means give up rather than loop forever. + stalls = next.length < pending.length ? 0 : stalls + 1; + pending = next; + if (!pending.length) break; + if (stalls >= WATCH_MAX_FAILS) { + const msg = `Batch ${tag}: gave up after ${stalls} passes with no progress; ${pending.length} task(s) unresolved.`; + emit(json, { tag, error: msg, unresolved: pending.length }, () => console.error(edim(msg))); + process.exitCode = 1; + return; + } + await new Promise((r) => setTimeout(r, intervalMs)); } const tasks = store.listTasks({ tag }); const completed = tasks.filter((t) => isSuccess(t.status)).length; @@ -235,8 +306,13 @@ program .action(async (file, opts) => { try { const text = file === '-' ? await readStdin() : fs.readFileSync(file, 'utf8'); - const prompts = parsePrompts(text, opts.separator); + const { prompts, skipped } = parsePrompts(text, opts.separator); if (!prompts.length) throw new Error(`no prompts found in ${file === '-' ? 'stdin' : file}`); + // A one-line heads-up so a swallowed prompt (or a stray comment) is never a + // silent mystery. Goes to stderr so it can't corrupt --json on stdout. + if (skipped) { + console.error(edim(`(skipped ${skipped} blank/comment line${skipped === 1 ? '' : 's'})`)); + } // Auto-tag so the batch is collectable as a unit; a user tag wins. const tag = opts.tag || `batch-${crypto.randomUUID().slice(0, 6)}`; @@ -321,10 +397,13 @@ program const task = needTask(id); try { // Completed tasks are served from SQLite -- no network, and it still - // works after the free tier drops the interaction at 24h. - if (isSuccess(task.status) && task.result && !opts.raw) { + // works after the free tier drops the interaction at 24h. Gate on the + // result being *present*, not truthy: a task that completes with empty + // text stores `''`, which is exactly the case the cache must still serve + // -- re-polling it would 404 after 24h, the very thing we cache to avoid. + if (isSuccess(task.status) && task.result != null && !opts.raw) { emit(opts.json, { id: task.id, status: task.status, result: task.result }, () => - console.log(task.result) + console.log(task.result || '(empty response)') ); return; } @@ -360,6 +439,11 @@ program .option('--json', 'machine-readable output') .description('all tasks, newest first') .action((opts) => { + // `-n 0` is a valid cap (show nothing); a negative would become SQLite's + // "no limit" (LIMIT -1 = all rows), so reject anything but a non-negative int. + if (opts.limit != null && (!Number.isInteger(opts.limit) || opts.limit < 0)) { + return die(new Error(`--limit must be a non-negative integer (got ${opts.limit})`)); + } const tasks = store.listTasks({ status: opts.status, tag: opts.tag, limit: opts.limit }); if (opts.json) return console.log(JSON.stringify(tasks, null, 2)); if (!tasks.length) { @@ -378,6 +462,117 @@ program } }); +// --- export --------------------------------------------------------------- + +// Collect many finished results into one document -- the "gather" that pairs +// with `batch`'s "scatter". Where `get` prints one result at a time, `export` +// concatenates a whole tag (or status) under prompt headings, to stdout or a +// file, as Markdown (default) or JSON. +program + .command('export') + .option('-t, --tag ', 'only this tag') + .addOption(new Option('--status ', 'only this status').choices(ALL_STATUSES).default('completed')) + .addOption(new Option('--format ', 'output format').choices(['md', 'json']).default('md')) + .option('-o, --out ', 'write to a file instead of stdout') + .description('concatenate finished results, each under its prompt, to stdout or a file') + .action((opts) => { + const tasks = store.listTasks({ tag: opts.tag, status: opts.status }); + // Newest-first suits a listing, but an export reads top-to-bottom like a + // document, so oldest-first is the natural order here. + tasks.reverse(); + // Only rows that actually carry a result are worth exporting: a status + // filter other than `completed` can match tasks that never stored text. + const rows = tasks.filter((t) => t.result != null); + if (!rows.length) { + // Nothing to write isn't an error, but say why so an empty -o file (or an + // empty pipe) isn't a mystery. The note goes to stderr, never the output. + console.error(`No ${opts.status} results to export${opts.tag ? ` for tag '${opts.tag}'` : ''}.`); + return; + } + + let output; + if (opts.format === 'json') { + output = JSON.stringify( + rows.map((t) => ({ + id: t.id, + tag: t.tag, + status: t.status, + prompt: t.prompt, + result: t.result, + created_at: t.created_at, + })), + null, + 2 + ); + } else { + output = rows + .map((t) => { + const when = new Date(t.created_at).toISOString().replace('T', ' ').slice(0, 16); + const head = (t.prompt || '(no prompt)').replace(/\s+/g, ' ').trim(); + const body = t.result && t.result.trim() ? t.result : '_(empty result)_'; + return `## ${head}\n\n\`${t.id}\` · ${t.status} · ${when} UTC\n\n${body}`; + }) + .join('\n\n---\n\n'); + } + + if (opts.out) { + fs.writeFileSync(opts.out, output.endsWith('\n') ? output : `${output}\n`); + console.error(`Wrote ${rows.length} result(s) to ${opts.out}.`); + } else { + console.log(output); + } + }); + +// --- digest --------------------------------------------------------------- + +// One step past `export`: instead of concatenating a tag's results, feed them +// back through a single Gemini call and synthesise one summary. It is `research` +// with a prompt built from what you have already collected, so it submits, then +// watches to completion just like `research -w`. +program + .command('digest') + .requiredOption('-t, --tag ', 'synthesize the completed results under this tag') + .option('-m, --model ', 'model to use', gemini.DEFAULT_MODEL) + .option('-s, --system ', 'system instruction for the synthesis') + .option('--json', 'machine-readable output') + .description("feed a tag's completed results through one Gemini call into a single summary") + .action(async (opts) => { + let id; + try { + const done = store + .listTasks({ tag: opts.tag, status: 'completed' }) + .filter((t) => t.result != null && t.result.trim()); + if (!done.length) { + throw new Error( + `no completed results tagged '${opts.tag}' to digest.` + + ' Collect them first: gemcatch daemon --exit-when-idle' + ); + } + done.reverse(); // oldest first, so the sources read in submission order + const sources = done + .map((t, i) => `## Source ${i + 1}: ${(t.prompt || '').replace(/\s+/g, ' ').trim()}\n\n${t.result}`) + .join('\n\n'); + const prompt = + `Synthesize the following ${done.length} research result(s) into one coherent summary.` + + ' Note where they agree and disagree, and do not simply repeat each verbatim.\n\n' + + sources; + // The digest is itself a task, tagged so it is findable but kept out of + // the source tag so a later digest never digests its own output. + id = store.createTask({ prompt, model: opts.model, systemInstruction: opts.system, tag: `${opts.tag}-digest` }); + const r = await gemini.submit(prompt, { model: opts.model, systemInstruction: opts.system }); + store.setInteraction(id, r.interactionId, r.status); + if (!opts.json) console.error(edim(`Digesting ${done.length} result(s) tagged ${opts.tag} -> task ${id}.`)); + await watchTask(store.getTask(id), DEFAULT_POLL_MS, opts.json); + } catch (err) { + // Same rule as `research`: only a failed *submit* marks the task failed. + if (id) { + const t = store.getTask(id); + if (!t || !t.interaction_id) store.setStatus(id, 'failed', { error: err.message }); + } + die(err); + } + }); + // --- sync ----------------------------------------------------------------- // One refresh pass over everything in flight. Never throws: a task that fails @@ -422,12 +617,18 @@ program .option('--json', 'newline-delimited JSON events on stdout') .description('poll in-flight tasks on a loop so results are cached before they expire') .action(async (opts) => { - const intervalMs = Math.max(1000, (opts.interval || DEFAULT_DAEMON_S) * 1000); + if (!Number.isFinite(opts.interval) || opts.interval <= 0) { + return die(new Error(`--interval must be a positive number of seconds (got ${opts.interval})`)); + } + const intervalMs = Math.max(1000, opts.interval * 1000); let stopping = false; let wake = null; // Finish the pass in progress, then exit cleanly -- never leave a polled - // result unwritten because someone hit Ctrl-C. + // result unwritten because someone hit Ctrl-C. A *second* signal, though, + // means "I don't want to wait for this pass" -- force-exit immediately with + // the conventional 130 (128 + SIGINT) so a long paced pass can't trap you. const stop = () => { + if (stopping) process.exit(130); stopping = true; if (wake) wake(); }; @@ -440,7 +641,7 @@ program if (!opts.json) { console.error( - dim(`gemcatch daemon: polling every ${intervalMs / 1000}s. Store: ${store.DB_PATH}. Ctrl-C to stop.`) + edim(`gemcatch daemon: polling every ${intervalMs / 1000}s. Store: ${store.DB_PATH}. Ctrl-C to stop.`) ); } event({ event: 'start', interval_s: intervalMs / 1000, db: store.DB_PATH }); @@ -455,7 +656,7 @@ program // the next pass may well succeed, and a daemon that dies silently is // worse than one that complains. if (opts.json) event({ event: 'error', error: err.message }); - else console.error(`${dim(`[${hhmmss()}]`)} Error: ${err.message}`); + else console.error(`${edim(`[${hhmmss()}]`)} Error: ${err.message}`); } // Quiet by default: only transitions and failures are worth a line. @@ -465,7 +666,7 @@ program event({ event: r.error ? 'error' : 'update', id: r.id, status: r.status, error: r.error || null }); } else { console.error( - dim(`[${hhmmss()}] ${r.id}: `) + colorStatus(r.status) + (r.error ? ` ${dim(r.error)}` : '') + edim(`[${hhmmss()}] ${r.id}: `) + ecolorStatus(r.status) + (r.error ? ` ${edim(r.error)}` : '') ); } } @@ -488,7 +689,7 @@ program } event({ event: 'stop' }); - if (!opts.json) console.error(dim('gemcatch daemon: stopped.')); + if (!opts.json) console.error(edim('gemcatch daemon: stopped.')); store.close(); }); @@ -496,12 +697,32 @@ program async function watchTask(task, intervalMs, json) { let last = null; + let fails = 0; for (;;) { - const r = await refresh(task); + let r; + try { + r = await refresh(task); + fails = 0; // a clean poll resets the failure run + } catch (err) { + // A poll error must not sink a live task: keep its old status and try + // again next interval, exactly like watchBatch. Give up only once the + // failures pile up, so a task the server can't answer for can't hang the + // watch forever. (A 404 doesn't reach here -- refresh retires it and + // returns a terminal status, handled below.) + fails += 1; + if (fails >= WATCH_MAX_FAILS) { + const msg = `Gave up watching ${task.id} after ${fails} consecutive poll failures: ${err.message}`; + emit(json, { id: task.id, status: task.status, error: msg }, () => console.error(edim(msg))); + process.exitCode = 1; + return; + } + await new Promise((r2) => setTimeout(r2, intervalMs)); + continue; + } // Status chatter goes to stderr so `gemcatch watch x > out.txt` captures only // the result. if (r.status !== last && !json) { - console.error(dim(`[${new Date().toISOString().slice(11, 19)}] ${task.id}: `) + colorStatus(r.status)); + console.error(edim(`[${new Date().toISOString().slice(11, 19)}] ${task.id}: `) + ecolorStatus(r.status)); last = r.status; } if (isSuccess(r.status)) { @@ -512,7 +733,7 @@ async function watchTask(task, intervalMs, json) { } if (isDone(r.status)) { emit(json, { id: task.id, status: r.status, error: r.text || null }, () => { - console.error(`Task ${task.id} ended: ${colorStatus(r.status)}`); + console.error(`Task ${task.id} ended: ${ecolorStatus(r.status)}`); if (r.text) console.log(r.text); }); process.exitCode = 1; @@ -531,12 +752,17 @@ program .action(async (id, opts) => { const task = needTask(id); try { - if (isSuccess(task.status) && task.result) { + // Serve a completed result from cache -- present, not merely truthy, so an + // empty-text completion is served instead of re-polled (and lost at 24h). + if (isSuccess(task.status) && task.result != null) { emit(opts.json, { id: task.id, status: task.status, result: task.result }, () => - console.log(task.result) + console.log(task.result || '(empty response)') ); return; } + if (opts.interval != null && (!Number.isFinite(opts.interval) || opts.interval <= 0)) { + return die(new Error(`--interval must be a positive number of seconds (got ${opts.interval})`)); + } await watchTask(task, opts.interval ? opts.interval * 1000 : DEFAULT_POLL_MS, opts.json); } catch (err) { die(err); @@ -579,7 +805,7 @@ program } catch (err) { // Free-tier interactions vanish after 24h, so a missing remote is // normal -- never block the local delete on it. - console.error(dim(` (remote delete failed for ${task.id}: ${err.message})`)); + console.error(edim(` (remote delete failed for ${task.id}: ${err.message})`)); } } if (store.removeTask(task.id)) removed += 1; @@ -595,6 +821,12 @@ program .option('--dry-run', 'list what would go, delete nothing') .description('drop old finished tasks (in-flight work is never touched)') .action((opts) => { + // A negative (or non-numeric) --days puts the cutoff in the *future*, which + // would match every finished task and quietly wipe the lot. Refuse it: the + // cutoff must be at or before now. + if (!Number.isFinite(opts.days) || opts.days < 0) { + return die(new Error(`--days must be a non-negative number (got ${opts.days})`)); + } const cutoff = Date.now() - opts.days * 86400000; const doomed = store.prunableTasks(cutoff); if (!doomed.length) { @@ -626,4 +858,16 @@ program }); }); +// Close the store on the way out so a one-shot command doesn't leave the +// SQLite -wal/-shm sidecars lingering. The store opens lazily, so if a command +// never touched it this is a no-op; the daemon closes explicitly too, and a +// second close is harmless. +process.on('exit', () => { + try { + store.close(); + } catch (_) { + /* best effort on the way out */ + } +}); + program.parseAsync(process.argv).catch(die); diff --git a/package-lock.json b/package-lock.json index 0f2e989..0ee8db2 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1,12 +1,12 @@ { "name": "gemcatch", - "version": "0.2.0", + "version": "0.3.0", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "gemcatch", - "version": "0.2.0", + "version": "0.3.0", "license": "MIT", "dependencies": { "@google/genai": "^2.12.0", diff --git a/package.json b/package.json index 90b878d..d46447a 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "gemcatch", - "version": "0.2.0", + "version": "0.3.0", "description": "Fire-and-forget CLI for Gemini's Interactions API background execution. Submit long-running research prompts, close your laptop, collect results later.", "main": "index.js", "bin": { diff --git a/test-offline.js b/test-offline.js index 87dcf46..19f6071 100644 --- a/test-offline.js +++ b/test-offline.js @@ -56,14 +56,18 @@ const server = http.createServer((req, res) => { interactions.set(id, { status: 'in_progress', prompt: body.input, - // SLOW stays in_progress for a couple of polls; FAIL and FLAKY resolve - // on the first successful one so both paths are reachable in a single - // `get`. - pollsLeft: /SLOW/.test(body.input) ? 2 : /FAIL|FLAKY/.test(body.input) ? 0 : 1, - text: /FAIL/.test(body.input) ? '' : ANSWER, + // SLOW stays in_progress for a couple of polls; FAIL/FLAKY/EMPTY and the + // wedge cases resolve on the first successful one so both paths are + // reachable in a single `get`. + pollsLeft: /SLOW/.test(body.input) ? 2 : /FAIL|FLAKY|EMPTY|WATCHWEDGE|HARDFAIL/.test(body.input) ? 0 : 1, + // FAIL and EMPTY both complete with no text; only FAIL is an error. + text: /FAIL|EMPTY/.test(body.input) ? '' : ANSWER, fails: /FAIL/.test(body.input), // FLAKY answers the first two polls with a 503 before behaving. flakyLeft: /FLAKY/.test(body.input) ? 2 : 0, + // WATCHWEDGE 500s a few times then recovers; HARDFAIL 500s forever. Both + // drive the watch/daemon consecutive-failure safety bound. + hardFailLeft: /HARDFAIL/.test(body.input) ? Infinity : /WATCHWEDGE/.test(body.input) ? 4 : 0, model: body.model, system: body.system_instruction, }); @@ -92,6 +96,13 @@ const server = http.createServer((req, res) => { getHits += 1; const it = interactions.get(idPart); if (!it) return send(404, { error: { code: 404, message: 'Interaction not found.' } }); + // WATCHWEDGE/HARDFAIL: a 500 that, with retries off, surfaces straight to + // the watch loop and drives its consecutive-failure bound. Infinity never + // recovers; a finite count recovers once it hits zero. + if (it.hardFailLeft > 0) { + it.hardFailLeft -= 1; + return send(500, { error: { code: 500, message: 'Internal error. Please try again.' } }); + } // Retry-After: 0 keeps the suite fast while still exercising the header. if (it.flakyLeft > 0) { it.flakyLeft -= 1; @@ -169,6 +180,34 @@ function cli(args, extra) { }); } +// Like cli(), but never hangs the suite: if the process doesn't exit within +// `ms` it is killed and the result carries `timedOut: true` for the caller to +// assert on. Used to prove the watch/daemon safety bounds actually fire. +function cliTimeout(args, extra, ms) { + const o = extra || {}; + return new Promise((resolve, reject) => { + const child = spawn(process.execPath, [path.join(__dirname, 'index.js')].concat(args), { + env: testEnv(o.env), + stdio: ['pipe', 'pipe', 'pipe'], + }); + let stdout = ''; + let stderr = ''; + let timedOut = false; + const timer = setTimeout(() => { + timedOut = true; + child.kill('SIGKILL'); + }, ms); + child.stdout.on('data', (d) => (stdout += d)); + child.stderr.on('data', (d) => (stderr += d)); + child.on('error', reject); + child.on('close', (code) => { + clearTimeout(timer); + resolve({ stdout, stderr, code, timedOut }); + }); + child.stdin.end(o.stdin === undefined ? '' : o.stdin); + }); +} + const out = async (args, extra) => (await cli(args, extra)).stdout; const idOf = (text) => (text.match(/^Task (\w+) submitted\./m) || [])[1]; const ok = (name) => console.log(` ok ${name}`); @@ -541,6 +580,217 @@ async function submit(prompt, args, extra) { assert(/GEMINI_API_KEY is not set/.test(noKey.stderr), `expected no-key message: ${noKey.stderr}`); ok('a missing key explains how to get one'); + // ======================================================================== + // Audit fixes (0.3.0) + // ======================================================================== + + // ---- #1: a poll error mid research --watch never marks a submitted task + // failed; it stays active and a later poll still completes it. ---- + const wwEnv = { GEMCATCH_HOME: path.join(HOME, 'watchwedge') }; + const ww = await cliTimeout( + ['research', 'WATCHWEDGE research me', '-w'], + { env: Object.assign({}, wwEnv, { GEMCATCH_MAX_RETRIES: '0', GEMCATCH_WATCH_MAX_FAILS: '2' }) }, + 15000 + ); + assert.strictEqual(ww.timedOut, false, 'the watch must give up, not hang, on a wedged poll'); + assert.strictEqual(ww.code, 1, 'giving up on a watch exits non-zero'); + assert(/[Gg]ave up watching/.test(ww.stderr), `expected a give-up message: ${ww.stderr}`); + const wwRow = new Database(path.join(wwEnv.GEMCATCH_HOME, 'tasks.db')).prepare('SELECT id, status FROM tasks').get(); + assert.notStrictEqual(wwRow.status, 'failed', 'a watch poll error must never mark the submitted task failed'); + assert.strictEqual(wwRow.status, 'in_progress', `the task stays active for the daemon: ${wwRow.status}`); + // Full retries (a later get / the daemon) recover it once the blips clear. + assert.strictEqual((await out(['get', wwRow.id], { env: wwEnv })).trim(), ANSWER, 'a later get still collects the result'); + ok('#1 a poll error during research --watch leaves the task active (not failed); a later get completes it'); + + // ---- #2: a 404 on poll retires the task to incomplete, so + // `daemon --exit-when-idle` converges instead of spinning forever. ---- + const exEnv = { GEMCATCH_HOME: path.join(HOME, 'expire404') }; + const exId = await submit('SLOW expire me', [], { env: exEnv }); + const exIid = new Database(path.join(exEnv.GEMCATCH_HOME, 'tasks.db')) + .prepare('SELECT interaction_id FROM tasks WHERE id = ?') + .get(exId).interaction_id; + interactions.delete(exIid); // the free tier drops it; every poll now 404s + const exDaemon = await cliTimeout(['daemon', '-i', '0.05', '--exit-when-idle', '--json'], { env: exEnv }, 15000); + assert.strictEqual(exDaemon.timedOut, false, 'daemon --exit-when-idle must converge once the 404 retires the task'); + const exRow = new Database(path.join(exEnv.GEMCATCH_HOME, 'tasks.db')) + .prepare('SELECT status, error FROM tasks WHERE id = ?') + .get(exId); + assert.strictEqual(exRow.status, 'incomplete', `a 404 poll should retire the task to incomplete: ${JSON.stringify(exRow)}`); + assert(/not found/.test(exRow.error || ''), 'the retire reason should be recorded'); + ok('#2 a 404 poll retires the task to incomplete and daemon --exit-when-idle converges'); + + // ---- #2 (bound): a permanently-failing poll hits the safety bound instead + // of looping forever. ---- + const hfEnv = { GEMCATCH_HOME: path.join(HOME, 'hardfail') }; + const hfId = await submit('HARDFAIL forever', [], { env: hfEnv }); + const hf = await cliTimeout( + ['watch', hfId], + { env: Object.assign({}, hfEnv, { GEMCATCH_MAX_RETRIES: '0', GEMCATCH_WATCH_MAX_FAILS: '3', GEMCATCH_POLL_MS: '20' }) }, + 15000 + ); + assert.strictEqual(hf.timedOut, false, 'a persistently failing watch must stop at the bound, not loop forever'); + assert.strictEqual(hf.code, 1, 'hitting the safety bound exits non-zero'); + assert(/consecutive poll failures/.test(hf.stderr), `expected the bound message: ${hf.stderr}`); + ok('#2 a permanently-failing watch stops at the consecutive-failure bound'); + + // ---- #4: prune -d is rejected and deletes nothing. ---- + const p4Env = { GEMCATCH_HOME: path.join(HOME, 'prune-neg') }; + const p4 = await submit('prune neg victim', [], { env: p4Env }); + await cliTimeout(['watch', p4, '-i', '0.02'], { env: p4Env }, 15000); // drive to completed + const p4db = new Database(path.join(p4Env.GEMCATCH_HOME, 'tasks.db')); + p4db.prepare('UPDATE tasks SET created_at = ? WHERE id = ?').run(Date.now() - 999 * 86400000, p4); + p4db.close(); + const p4rej = await cli(['prune', '-d', '-5'], { env: p4Env }).catch((e) => e); + assert.strictEqual(p4rej.code, 1, 'prune -d -5 should be rejected'); + assert(/--days must be a non-negative number/.test(p4rej.stderr), `expected a clear rejection: ${p4rej.stderr}`); + assert((await out(['list'], { env: p4Env })).includes(p4), 'a rejected prune must delete nothing, even an ancient finished task'); + ok('#4 prune -d is rejected and deletes nothing'); + + // ---- #5: a completed-but-empty result is served from cache, not re-polled. ---- + const emEnv = { GEMCATCH_HOME: path.join(HOME, 'empty') }; + const emId = await submit('EMPTY result please', [], { env: emEnv }); + const em1 = await cliTimeout(['watch', emId, '-i', '0.02'], { env: emEnv }, 15000); + assert.strictEqual(em1.code, 0, `an empty completion is still a success: ${em1.stderr}`); + const beforeEmpty = getHits; + const em2 = await cli(['get', emId], { env: emEnv }); + assert.strictEqual(getHits, beforeEmpty, 'an empty completed result must come from SQLite, not the API'); + assert(/empty response/.test(em2.stdout), `an empty result reads as (empty response): ${JSON.stringify(em2.stdout)}`); + ok('#5 a completed-but-empty result is served from cache without re-polling'); + + // ---- #6: `#` is a comment only with trailing whitespace; "#1 ..." survives, + // and the skipped count is reported on stderr. ---- + const c6 = path.join(HOME, 'batch-hash.txt'); + fs.writeFileSync(c6, '# a comment\n#1 cause of failures?\nkeep two\n\n# another comment\n'); + const r6 = await cli(['batch', c6, '--json']); + const j6 = JSON.parse(r6.stdout); + assert.deepStrictEqual( + j6.submitted.map((s) => s.prompt), + ['#1 cause of failures?', 'keep two'], + 'a "#1 ..." line is a prompt, not a comment; "# ..." lines are skipped' + ); + assert(/skipped 3 blank\/comment lines/.test(r6.stderr), `the skipped count should be noted on stderr: ${r6.stderr}`); + ok('#6 batch keeps "#1 ..." prompts, skips "# ..." comments, and reports the count'); + + // ---- #7: list -n 0 returns zero rows (not all); watch -i <=0 is rejected. ---- + assert.strictEqual(JSON.parse(await out(['list', '--json', '-n', '0'])).length, 0, '-n 0 caps to zero rows, not all'); + const i7Env = { GEMCATCH_HOME: path.join(HOME, 'interval') }; + const i7 = await submit('SLOW interval victim', [], { env: i7Env }); + const ivRej = await cli(['watch', i7, '-i', '-5'], { env: i7Env }).catch((e) => e); + assert.strictEqual(ivRej.code, 1, 'watch -i -5 should be rejected'); + assert(/--interval must be a positive number/.test(ivRej.stderr), `expected interval validation: ${ivRej.stderr}`); + ok('#7 list -n 0 returns zero rows; watch -i <=0 is rejected'); + + // ---- #8: export gathers a tag's completed prompts + results (md/json/-o). ---- + const xpEnv = { GEMCATCH_HOME: path.join(HOME, 'export') }; + const xp1 = await submit('first export question', ['-t', 'expt'], { env: xpEnv }); + const xp2 = await submit('second export question', ['-t', 'expt'], { env: xpEnv }); + await cliTimeout(['watch', xp1, '-i', '0.02'], { env: xpEnv }, 15000); + await cliTimeout(['watch', xp2, '-i', '0.02'], { env: xpEnv }, 15000); + const md = await out(['export', '--tag', 'expt', '--format', 'md'], { env: xpEnv }); + assert(md.includes('## first export question') && md.includes('## second export question'), `each prompt should be a heading: ${md}`); + const answerHits = (md.match(new RegExp(ANSWER.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'), 'g')) || []).length; + assert.strictEqual(answerHits, 2, 'both completed results should appear in the export'); + assert(md.includes(xp1) && md.includes(xp2), 'each section carries its task id'); + assert(md.indexOf('first export question') < md.indexOf('second export question'), 'export reads oldest-first'); + const jx = JSON.parse(await out(['export', '--tag', 'expt', '--format', 'json'], { env: xpEnv })); + assert.strictEqual(jx.length, 2, 'json export has one object per result'); + assert.strictEqual(jx[0].result, ANSWER, 'json export carries the result text'); + const xpFile = path.join(xpEnv.GEMCATCH_HOME, 'out.md'); + const xpo = await cli(['export', '--tag', 'expt', '-o', xpFile], { env: xpEnv }); + assert(fs.existsSync(xpFile) && fs.readFileSync(xpFile, 'utf8').includes(ANSWER), '-o writes the export to a file'); + assert(/Wrote 2 result/.test(xpo.stderr), 'the -o confirmation goes to stderr, not the file'); + ok('#8 export gathers a tag\'s completed prompts + results as md/json and to a file'); + + // ---- digest: synthesize a tag's completed results through one Gemini call. ---- + const dg = await cliTimeout(['digest', '--tag', 'expt'], { env: xpEnv }, 15000); + assert.strictEqual(dg.code, 0, `digest should succeed: ${dg.stderr}`); + assert.strictEqual(dg.stdout.trim(), ANSWER, 'digest prints the synthesized result on stdout'); + assert(/Digesting 2 result/.test(dg.stderr), 'digest reports what it is summarizing on stderr'); + assert.strictEqual( + JSON.parse(await out(['list', '--json', '--tag', 'expt-digest'], { env: xpEnv })).length, + 1, + 'the digest lands under -digest, not the source tag' + ); + ok('digest feeds a tag\'s completed results through one Gemini call into a single summary'); + + // ---- #3: the default SDK transport, exercised against a stubbed @google/genai. + // Every test above forces GEMCATCH_FORCE_REST=1, so sdkInteractions() is + // otherwise never covered. Inject a stub client and drive it directly. ---- + { + const genaiPath = require.resolve('@google/genai'); + const geminiPath = require.resolve('./gemini'); + const sdkState = new Map(); + let sdkSeq = 0; + class StubGenAI { + constructor(cfg) { + assert(cfg && cfg.apiKey, 'the SDK client must be built with an apiKey'); + this.interactions = { + create: async (body) => { + assert.strictEqual(body.background, true, 'SDK submit must pass background:true'); + assert.strictEqual(typeof body.input, 'string', 'SDK input must be a plain string'); + const id = `sdk_${++sdkSeq}`; + sdkState.set(id, { polls: /SLOW/.test(body.input) ? 1 : 0, boom: /BOOM/.test(body.input) }); + return { id, status: 'in_progress' }; + }, + get: async (id) => { + const s = sdkState.get(id); + if (s.boom) { + // Mimic the SDK's real failure shape: a useless stub .message plus + // Google's true payload as a JSON string on .body -- exactly what + // friendly() unwraps into a readable error. + const e = new Error('400 API error occurred: {"httpMeta":{}}'); + e.status = 400; + e.body = JSON.stringify({ error: { code: 400, message: 'Unknown model id via SDK.' } }); + throw e; + } + if (s.polls > 0) { + s.polls -= 1; + return { id, status: 'in_progress' }; + } + // The SDK synthesises output_text; return one so shape() prefers it. + return { id, status: 'completed', output_text: ANSWER, usage: { total_tokens: 7 } }; + }, + }; + } + } + const saved = { + key: process.env.GEMINI_API_KEY, + force: process.env.GEMCATCH_FORCE_REST, + rpm: process.env.GEMCATCH_RPM, + }; + process.env.GEMINI_API_KEY = 'TEST_KEY'; + delete process.env.GEMCATCH_FORCE_REST; // let the SDK path win over the fallback + process.env.GEMCATCH_RPM = '0'; // no pacing for these in-process calls + require.cache[genaiPath] = { id: genaiPath, filename: genaiPath, loaded: true, exports: { GoogleGenAI: StubGenAI } }; + delete require.cache[geminiPath]; // reload so sdkInteractions() memoizes the stub + const sdk = require('./gemini'); + + const s = await sdk.submit('SLOW via the sdk', { model: 'gemini-3.1-flash' }); + assert.strictEqual(s.status, 'in_progress', 'SDK submit returns in_progress'); + assert(/^sdk_/.test(s.interactionId), `shape() must read id off the SDK response: ${s.interactionId}`); + let p = await sdk.poll(s.interactionId); + assert.strictEqual(p.status, 'in_progress', 'first SDK poll is still running'); + p = await sdk.poll(s.interactionId); + assert.strictEqual(p.status, 'completed', 'second SDK poll completes'); + assert.strictEqual(p.text, ANSWER, 'shape() prefers the SDK-synthesised output_text'); + assert.strictEqual(p.usage.total_tokens, 7, 'shape() carries usage through the SDK path'); + + const boom = await sdk.submit('BOOM').then((r) => sdk.poll(r.interactionId)).catch((e) => e); + assert(/Unknown model id via SDK/.test(boom.message), `friendly() must surface the SDK's real payload: ${boom.message}`); + assert.strictEqual(boom.httpStatus, 400, 'friendly() maps the SDK error to its HTTP status'); + + // Restore: nothing after this should see the stub or the fake key. + delete require.cache[geminiPath]; + delete require.cache[genaiPath]; + if (saved.key === undefined) delete process.env.GEMINI_API_KEY; + else process.env.GEMINI_API_KEY = saved.key; + if (saved.force === undefined) delete process.env.GEMCATCH_FORCE_REST; + else process.env.GEMCATCH_FORCE_REST = saved.force; + if (saved.rpm === undefined) delete process.env.GEMCATCH_RPM; + else process.env.GEMCATCH_RPM = saved.rpm; + ok('#3 SDK transport: submit -> poll -> completed and a friendly() error, via a stubbed @google/genai'); + } + server.close(); fs.rmSync(HOME, { recursive: true, force: true }); console.log('\nAll offline tests passed.');