Repository navigation
[wip] cf-os api: drive a running instance through its API - #19
Maximo-Guk wants to merge 13 commits into
Conversation
Maximo-Guk
changed the title
![ask-bonk[bot]](https://avatars.githubusercontent.com/in/2454580?s=60&v=4){kind=small}
| const disposables: Disposable[] = []; | ||
| const emit = ndjsonEvents((line) => io.stdout.write(line)); | ||
| const args = calls.map((call) => call.args.map((arg) => { | ||
| if (arg instanceof StreamArg) return io.openStream(arg.path); |
There was a problem hiding this comment.
[P2] Clean up input streams when connection setup fails
Stream arguments are opened before connect() validates credentials, but the cleanup never cancels these inputs. For import-blueprint - with a missing CFOS_API_TOKEN, the command reports the authentication error yet keeps reading stdin; if the producer has not closed its pipe, the CLI hangs instead of exiting. I reproduced this with stdin kept open after supplying some input. Track and cancel/destroy opened inputs on every exit, including failures before the stream is handed to Cap'n Web, and put argument setup inside the cleanup-protected region.
There was a problem hiding this comment.
This doesn't reproduce in the CLI itself. With stdin held open through a FIFO and CFOS_API_TOKEN unset, cf-os api import-blueprint - --url http://localhost:1 prints the authentication error and exits 1 straight away: finish() in main.ts ends any failed run with process.exit, so an input still being read can't keep it alive. The hang only shows when the command is driven in-process, where nothing exits.
The one opening left is a run that exits 0 without consuming its stream argument (its reader going away, or a null capability part way through the chain), since a successful run is left to the event loop. That's narrow enough to leave as it is for now.
| // Whatever moved, the pin or the generator with the merge, the committed API files should be what | ||
| // generation writes now. It reads the submodule's installed packages, so it waits for the install. | ||
| if (result.installable && hasScript(root, API_GENERATE)) { | ||
| result.api = result.installed ? generateApi(ctx, deps, result) : { status: "not-run" }; |
There was a problem hiding this comment.
[P2] Compare API commands against the pre-update schema
For a normal starter update, merge() has already replaced the generated schema.json before this runs. The generator reads that file as its comparison baseline (generator/generate.ts:59), so it compares the new API with itself: commands removed or changed by the merge are reported as unchanged. When the generated files already match the new pin, the brief even says no API command changed and apiStep() omits the caller-migration step. Preserve the schema from before the merge and compare against it independently of whether regeneration rewrites any files, so breaking changes in ordinary updates are included in the brief.
There was a problem hiding this comment.
Agreed, and fixed in 7e57608. The generator takes --since <commit> and compares the commands with the schema committed at that commit; update passes the commit it started from. When the merge brought the files already regenerated, the brief still lists the commands removed and changed, adds the step for their callers, and the update no longer counts as clean. Tested both in generate.test.ts, against a scratch repository, and in update.test.ts.
|
Review: 2 findings. Posted 2 actionable inline comments in the review, verified with targeted reproductions. |
The Workshop's Cap'n Web API has no schema to generate a client from: capnweb stubs are Proxies with no reflection, and the one model of the API's shapes, capnweb-validate's, is internal and has no parameter names or docs. So generator/extract.ts reads cloudflare-os's workshop-shared/src/api.ts with the TypeScript compiler API (the typescript6 alias, as upstream does), classifies types by capnweb-validate's rules (values, capabilities, callbacks, streams, any tails) and writes src/api/generated/schema.json. generator/emit.ts turns that schema into src/api/generated/commands.ts, a table of the CLI's decisions whose types are src/api/table.ts. It is a pure generate(schema) -> files with no I/O, shaped so it can later be wrapped as a forge plugin. generate.ts does the I/O, prints what changed as JSON for `cf-os update`, and with --check fails on stale files, which `pnpm lint` now runs as api:check. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
After moving the pin and installing, `update` runs `pnpm run api:generate` when the root package.json has that script, so `cf-os api`'s commands follow the pin. The regenerated files are left unstaged beside the staged pin for review, and the brief and the JSON result list the API commands added, removed and changed. A failed generation is reported there without undoing the pin. The first build workflow: on pull requests and pushes to main, check out with the submodule, install with the frozen lockfile (whose postinstall installs the submodule's workspace), then `pnpm lint`, which includes api:check, and `pnpm test`. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`cf-os api <command> [arguments] [<command> [arguments]]...` walks argv against the generated table: each segment is a method on the capability reached so far, required parameters positional and optional ones flags. The whole chain is sent pipelined in one Cap'n Web session, its result printed as JSON, and every stub and the session disposed however the run ends (result, error, --timeout, Ctrl-C, or stdout closing). It signs in through Cloudflare Access (CF_ACCESS_TOKEN or cloudflared), a token or a password hashed as the frontend hashes it, with secrets only ever from the environment. Subscriptions print their callbacks as NDJSON, built with every method the API declares, so a new callback cannot quietly end them. Streams go to and from files, and untyped capabilities take `call <method>`. A chain that ends on a capability sends nothing and says so, exiting 2 like any other usage error. `api` is project-only and delegated to the project's copy, so its commands are the pin's; main.ts hands it its arguments unparsed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`cf-os api search "<task>"` ranks every command in the table against the task's words and prints the best five as JSON, each a whole command line with placeholders for the arguments along the way. It reads the table only. The paths come from src/api/paths.ts, now shared with the generator, which reserves `search` on the root as it does `public`. The help of a command that returns a capability lists that capability's commands; the root help opens with a three-line pointer at search for agents, and every other help ends with one line pointing at the API options. Docs read in the CLI's words: `method()` becomes its command, and TODO paragraphs are left out. Events key their arguments by parameter name. An api usage error prints its message alone, and a void result ends the chain as "nothing". Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A doc naming a method of another capability keeps its owner
("GadgetClient's `bind`"), so GatekeeperClient's help no longer points at
a command it does not have, and a call inside a longer code span is left
alone. Search folds add, create and make into new, splits CamelCase
capability names into words, and scores and prints the doc's whole first
sentence, which "e.g." no longer ends. node:util's own flag errors become
one sentence with a pointer at the help, like every other api usage error.
`cf-os --help` and the README's command table mention search.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A JSON parameter's help spells out each named object or union it is made of, so `--filter` shows ConnectedAccountsFilter's fields rather than the name alone. A command named in a doc says which capability it is on as "`bind` (on GadgetClient)", which no longer runs into the word before it, and a code span is rewritten only when it is the call alone. In search an exact verb outranks the synonym it is folded with. A usage error's help pointer names the help of the segment at fault, the root's for a runtime option, quoted as a shell needs it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A command's result is spelled out in its help as a JSON parameter's type is, so get-metadata's help shows the workspace id among its fields; a named type too long to read in help (AiChatMessage) stays a name. A help pointer stops before a positional that only `--` let through, since no `--help` can follow it. cliDoc pairs code spans within a paragraph, so a stray backtick leaves the next paragraph alone. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A command typed where a positional goes (`open-gadget new-gatekeeper --help`) is taken for that positional; the help it prints now says so and gives the command for the help that was meant. The help of a command that returns a capability ends with how to reach the help of one of that capability's commands. Help pointers and the examples of an unfinished chain share one rule for tokens only `--` lets through (flag-like ones and `--` itself): an example puts the `--` back, so it walks as suggested, and a help pointer stops before them, since no `--help` can follow a `--`. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The pin brings the admin API's update checks and changes the chat and Gatekeeper-listing types. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
| } | ||
|
|
||
| const disposables: Disposable[] = []; | ||
| const emit = ndjsonEvents((line) => io.stdout.write(line)); |
There was a problem hiding this comment.
[P2] Backpressure subscription events when stdout is full
For a long-running subscription piped to a slow or temporarily stalled consumer, this discards stdout.write() returning false, while callbackTarget() immediately returns and acknowledges the event. The server can therefore keep delivering events while Node buffers all the output, growing memory without bound. I reproduced this with the generated ConsoleLogSubscriber: all 4,096 sequentially awaited 4 KiB log events were acknowledged despite a stalled 1 KiB-high-water-mark output, leaving about 17 MB queued. Await drain when a write returns false, and propagate that promise through the callback methods so event delivery can backpressure the reader.
There was a problem hiding this comment.
Fixed in fc22d19. When a write leaves stdout full, the event's callback now returns a promise that settles on drain, so a server that awaits its calls is held back, the same way a returned byte stream already waits. callbacks.test.ts checks it with a stalled writer: the server's call doesn't return until the reader takes the line.
|
Review: 1 findings. Posted 1 new actionable inline comment in the review, verified with a targeted reproduction. The two earlier review threads remain open. All 38 targeted parser, codec, callback, and emitter tests passed. |
A starter merge brings the generated files already regenerated for its pin, so api:generate compared the new schema with itself: the brief said no API command changed, and left out the step for their callers. The generator now takes --since <commit>, the schema committed there being the baseline, and update passes the commit it started from. Commands removed or changed with nothing left to rewrite are listed, get the callers step, and keep the update from counting as clean. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An event's write ignored stdout being full, and its callback returned at once, so a server awaiting its calls kept delivering while Node buffered every line. The callback now returns a promise for the drain when the write leaves stdout full, as a returned byte stream already waits for it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Since last review: 3 resolved, 0 still open, 0 new. LGTM! |
WIP
Stacked on #15.
cf-os apicalls a running Cloudflare OS instance's Cap'n Web API, the one its frontend uses. Its commands are generated from the pinnedcloudflare-os/packages/workshop-shared/src/api.ts, so they always match the pin.packages/cf-os/generator): extracts the API's capability graph with the TypeScript compiler API and writes a command table tosrc/api/generated.pnpm api:generaterewrites it,pnpm lintchecks it is current (api:check), andcf-os updateregenerates it when the pin moves.src/api): walks argv as a capability chain (open-gadget <id> get-chat-history <chatId>), sent pipelined in one WebSocket session. It signs in with Cloudflare Access, a token or a password. Results print as JSON, subscription events as JSON lines, and streams as bytes. Everything the chain made is disposed.cf-os api search "<task>"prints the five best matches as whole command lines.Also bumps
cf-osto0.2.0-beta.0, to publish under thebetatag.Testing:
pnpm lintandpnpm testpass. Agents with no prior context then did four real tasks against a test instance, using only the CLI:All four completed without guessing a command.