Skip to content

Latest commit

 

History

History
153 lines (135 loc) · 37.6 KB

File metadata and controls

153 lines (135 loc) · 37.6 KB

This project is building a platform for "vibe coded" personal applications and AI agents that run inside a strong sandbox.

The following files are commonly important to reference:

  • packages/workshop-shared/node_modules/capnweb/README.md: Explains how to use Cap'n Web RPC, which is used extensively for client-server communications.
  • packages/workshop-shared/src/api.ts: Defines the RPC API used between the frontend and backend.

The project structure is:

  • packages/workshop-frontend: The Gadgets Workshop UI.
    • This is a pure single-page app, running entirely client-side.
    • It speaks to the backend using an RPC API over a persistent WebSocket connection.
    • Uses React, Kumo UI (https://kumo-ui.com/api/component-registry), Phosphor icons, and Vite.
  • packages/workshop-backend: The Gadgets Workshop server.
    • Runs on Cloudflare Workers.
    • This is the kernel: it defines the architecture and is held to a higher bar than UI/gatekeeper code. Reviewers read every line of workshop-backend and of API changes in workshop-shared, so keep diffs here small and elegant. Concretely: doc-comment every exported member of the workshop-shared public API (types, consts, and functions — not just interfaces); never introduce a hand-written interface that mirrors an RPC interface plus an as unknown as cast (derive from the real type instead, or rethink the design); and prefer reusing existing mechanisms over adding parallel ones. Capability-based security note: a resource becomes "ambient" (auto-injected) only by user/admin configuration — a gatekeeper must never assert its own ambience. When a change to this package is large, split it by concern into separate PRs (and at minimum group commits so workshop-backend/workshop-shared can be reviewed apart from UI), since fewer kernel lines = easier review.
    • format-blueprints/ holds the output format blueprints the deployment ships with, committed as reviewable source: each <name>/ contains blueprint.json plus the gadget code under files/. scripts/build-format-blueprints.ts reconstructs their archive representation (override the source directory with FORMAT_BLUEPRINTS_DIR) into the gitignored src/generated/format-blueprints.ts, so build and test both run the generator first. Replace one with pnpm import:format-blueprint <export.gadget> <blueprintId>, or add one with pnpm import:format-blueprint <export.gadget> --new <name>; never edit a blueprintId after deploy, since the install and promotion are keyed on it and a rename orphans the old entry. See format-blueprints/README.md.
  • packages/workshop-shared: Shared API definitions between client and server.
    • This defines the application's RPC interface.
    • The RPC protocol is Cap'n Web, which has similar semantics to Cloudflare's Worker-to-Worker RPC system, while being able to run in a browser over WebSocket. Read the readme for details.
  • packages/configurator-ui: Type-only component helpers used by optional gatekeeper resource configurator UI modules.
    • Gatekeeper configurator UI modules are compiled by scripts/build-gatekeeper-configurator.ts as part of package builds.
  • packages/gatekeeper-*: Gatekeeper workers for external service integrations.
    • Each gatekeeper runs as a separate Cloudflare Worker — with one exception the prefix does not capture: a gatekeeper-* package with no wrangler.jsonc is a library, not a worker (gatekeeper-kit here; gatekeeper-shared in the internal repo). Deployable discovery is config-gated, not name-gated — readDeployablePackages in scripts/release/manifest-lib.ts keys solely on the presence of wrangler.jsonc, and run-dev-server.ts requires it too — so adding one to a library package is what would make it deployable, at which point workerKind would classify it a gatekeeper by prefix and the deploy wizard would demand CLIENT_ID/CLIENT_SECRET for it. manifest-lib.test.ts fails first if that ever happens.
    • Gatekeepers handle OAuth flows and provide sandboxed access to external APIs.
    • A gatekeeper may declare VendorDescription.autoProvisionsAccount: it can mint a connected account with no OAuth flow (via GatekeeperVendor.createAccount(), which takes no user identity). For such gatekeepers the deployment admin picks a per-vendor mode in the admin Gatekeepers panel — disabled / optional / enabled (default optional) — resolved in provisioning-policy.ts: enabled auto-provisions the account for every user (forced, and hidden from the Connectors list), optional lets each user opt in from the Connectors page, and disabled offers it to no one (existing accounts go dormant). The Workshop persists the account in the user DO like any connected account (the account capability — not an asserted identity — is the authority thereafter). The account (a GatekeeperUser) declares in its AccountDescription whether it provides an agent singleton (singleton: { tsType }) and/or a management UI (providesUi). The Workshop auto-provides the singleton to the owner's workspaces as an ambient gatekeeper record, folded into each chat's env as a named chat binding (named by the gatekeeper's suggestedBindingName; see prepareChatBindings in overseer.ts) that the agent reads in executeCode (getSession/getAgentCatalog), each read recorded as an observation. It is not bound to any gadget by default — most gadgets never call it programmatically — but the agent may wire it into a gadget's binding list with setGadgetBinding when the gadget's persistent code needs it. The UI is hosted at /gatekeepers/$appId (the gatekeeper's vendor id, e.g. /gatekeepers/context) via startAppUi({ isAdmin }). The two are orthogonal — an account can declare either, both, or neither.
  • packages/mcp-shared: Shared implementation behind the two MCP gatekeepers — gatekeeper-mcp (endpoints a user pastes) and gatekeeper-mcp-portal (one admin-configured portal). Not a Worker; a library both import, holding the MCP client, the OAuth chain, the account DO base, the resource-URL scope grammar, and the queued-action store. See packages/mcp-shared/README.md and each connector's README.
    • The trust boundary is tools.ts, and nothing outside it reads a tool's annotations: a tool the server declares readOnlyHint: true runs as an observation, everything else is queued for approval, and auto-applying a write additionally requires a vetted endpoint — which only the portal can produce, via MCP_PORTAL_TRUST_ANNOTATIONS.
    • OAuth uses the official @modelcontextprotocol/client; always give SDK OAuth operations sdkFetch(...) so every request and redirect retains endpoint and SSRF checks.
  • packages/gatekeeper-context: The Context Library — a gatekeeper whose account provides a singleton read session + a management UI, for authoring collections of context documents that agents read as observations. Collections have one of two visibilities: private (owned by a single account, readable/writable only by that account) and public (created/edited only by deployment admins, readable by everyone and auto-enabled for all users). It owns its state in three Durable Objects (ContextCollectionDurableObject for content, UserLibraryDurableObject for each account's own private collections, LibraryRegistryDurableObject for the domain's public set) plus a KV namespace. All data is namespaced by a sharingDomain (from the binding's props, see domain.ts) so multiple workshops sharing one gatekeeper instance stay isolated.
    • Its GatekeeperVendor entrypoint (bound as GATEKEEPER_CONTEXT) declares autoProvisionsAccount and mints a ContextAccount via createAccount() (no user identity is passed in; the account keys its private data by its own generated accountId). The account exposes the agent read session (getSession()), collection discovery metadata (getAgentCatalog()), and a management UI (startAppUi({ isAdmin })). The UI is a single-file React SPA in app/ (Vite + Tailwind + Kumo) bundled by build-app.mjs into src/generated/app.txt.
  • packages/gatekeeper-scheduler: Scheduled Tasks — an auto-provisioned gatekeeper whose account provides an ambient singleton for registering persistent workspace callbacks plus a read-only management UI. One account-scoped ScheduleDriver Durable Object stores enabled schedules and delivers them from a shared alarm; hook enablement remains in the Workshop Connections UI.
  • packages/gatekeeper-cloudflare: Cloudflare OAuth, serving three unrelated purposes from one connected account — sign-in (AUTH_GATEKEEPERS), AI Gateway billing, and Workers Observability read-only telemetry resources. Two resource granularities (whole account, or one Worker) both map to the single indivisible workers-observability.read scope, so the capability boundary is the binding, not the grant.
    • Layering: observability-api.ts is the only place that talks HTTP; observability-parse.ts validates every response (no blind casts); observability-session.ts is the agent-facing session, where every read funnels through one #observe so no path returns data unaudited; observability-discovery.ts derives field names/values from a sampled events query.
    • Scopes fail closed to BILLING_SCOPES, and an omitted resourceUrlPatterns ("all resource types") is distinct from [] ("none"): a billing-only connection must not silently acquire telemetry access, and recording a wider grant than was made would make ensureResources short-circuit into a binding that 403s with no way to re-prompt.
    • Three provider behaviours return wrong-but-plausible data with no error, so none is trusted (all verified against a live account): telemetry/keys/values ignore the filters they accept, so a constrained discovery call is answered from a filtered telemetry/query sample or it would disclose the whole account; an unknown filter key matches nothing, which bites because a log's own fields are returned nested under source but indexed under their bare name ({event: "x"}source.event, queried as event, hence observabilityFieldKey accepting the source. alias wherever a caller names a field); and /accounts pages at 20 by default, so it must be walked to the end.
    • Worker-scoped bindings prepend an immutable $metadata.service filter and re-filter the response, since a filter the provider silently ignored would leak another Worker's telemetry. A dropped event proves the filter was not applied, so the provider's own count is then withheld rather than reported (it would count the whole account) and the drop is logged at error; statistics is kept, since it describes what the query cost rather than how much matched. Pagination cursors come from the provider's raw events rather than the surviving ones, so a fully-foreign page can't stall pagination and hide the caller's own older data. Trace summaries are account-only (their shape describes the whole cross-service trace); a Worker binding can still fetch its own events for a known trace id. calculate() is the one read with no second line of defence — an aggregate can't be un-mixed — so it rests solely on the injected filter; that is accepted and documented on the method, with the group-by fix left as a follow-up.
    • A provider error message can quote a caller-supplied filter value back, so only its numeric codes are logged — filter values stay out of the audit trail (summarizeFilter), and the message travels to the caller who caused it.
    • Tests are two vitest projects: vitest.config.ts (Node, pure logic) and vitest.worker.config.ts (workerd, for RpcTarget/RpcStub/Durable Objects). The workerd suite reaches the gatekeeper through a TestHooks Durable Object because a DurableObjectClass carrying ctx.props is only reachable via ctx.facets — the way the overseer instantiates it.
    • src/configurator/*.tsx duplicate the resource-URL grammar from resources.ts and must: build-gatekeeper-configurator.mjs transpiles each per-file, stripping only @gadgets/configurator-ui and type-only imports, so they cannot import runtime helpers. __tests__/configurator-url.test.ts keeps the copies in step, and configurator-fields.test.ts drives render against a mocked runtime — the runtime's clearFields only drops an autocomplete's typed query, so a dependent field must also be nulled through setValues or the stale value silently survives into the resource URL.
  • packages/router: The public origin of a deployed gadgets instance. Serves the workshop-frontend assets and routes by path prefix: /api/* and /blueprint-screenshot/* to the workshop backend, /gatekeeper/<name>/* to whichever gatekeepers are bound (discovered by scanning its own GATEKEEPER_* service bindings, so installing a gatekeeper is purely a binding change). The same worker doubles as the dev router (pnpm dev-server): with no ASSETS binding it proxies frontend requests to the Vite dev server instead.

Deployment admin settings (the /admin panel) follow a few conventions worth knowing when extending them:

  • packages/workshop-backend/src/admin-config.ts defines AdminConfig — the deployment's "soft" customizations: agent instructions, banners/theme, and which gatekeeper connectors/resources are offered (plus the three-state mode for auto-provisioning gatekeepers, see provisioning-policy.ts). Connectors/resources default to enabled and the admin UI opts them out; auto-provisioning gatekeepers default to optional. Authentication/authorization config (sign-in providers via AUTH_GATEKEEPERS, password login via DISABLE_PASSWORD_AUTH) is deliberately NOT here — it stays env-var driven (auth/config.ts) so it can't be changed by a compromised admin session.
  • The AdminSettings durable object owns the authoritative AdminConfig and mirrors it to a single reserved KV key (.adminConfig, see isReservedBlueprintKey()), so hot-path code (connect/agent) reads it with one cheap KV get via readAdminConfig(env). The DO is the only writer (updateAdminConfig(patch)).
  • Admin operations are exposed as an AdminApi capability obtained via AuthenticatedApi.getAdminApi() (returns null for non-admins). The #isAdmin() check happens once when the capability is minted, so the individual methods don't re-check.
  • user.ts:getGatekeeperClassFor() is the single core chokepoint where disabled gatekeepers/resources are enforced before a capability is minted (gadget/agent code can't reach it directly).

Release pipeline (scripts/release/) — how customer instances get deployed:

  • build-release.ts bundles every deployable worker byte-identically (wrangler dry-run with the pinned wrangler), builds the Access-mode frontend asset build, and generates the release manifest — the contract between this repo's CI and the deploy service, produced by manifest-lib.ts from each package's wrangler.jsonc with account-specific values replaced by placeholders ($ACCOUNT_ID, $WORKER_NAME(...), $SECRET(...), $PUBLIC_BASE_URL, ...).
  • upload-release.ts mirrors the release to R2 content-addressed, manifest last; with --candidate the manifest lands under candidates/<id>/ (invisible to the deploy service) so e2e can verify it, and promote-release.ts then copies it to releases/<id>/ — publishing is that single all-or-nothing manifest copy. The copy is not isolated against concurrent promotions, so CI serializes promote runs (a GitLab resource group) and the script's newer-release guard skips candidates that a later release has already superseded.
  • The manifest is covered by a golden-file test; after an intentional manifest change, regenerate with UPDATE_GOLDEN=1 node --test scripts/release/manifest-lib.test.ts and review the golden diff.
  • Running the flow by hand (upload and promote need R2_ENDPOINT, R2_BUCKET, R2_ACCESS_KEY_ID, R2_SECRET_ACCESS_KEY):
    • node scripts/release/build-release.ts --out release-out — build everything into release-out/ (id defaults to r<CI_PIPELINE_IID>-<sha7> in CI, dev-<timestamp> locally; override with --release-id <id>).
    • node scripts/release/upload-release.ts --release release-out --candidate — mirror to R2; omit --candidate to publish directly (bypasses the gate — CI never does this).
    • node scripts/release/promote-release.ts --release-id <id> — copy the verified candidate's manifest into releases/<id>/.
  • Deploy-wizard configuration: an installable gatekeeper's user-supplied inputs default to OAuth CLIENT_ID/CLIENT_SECRET secrets; a per-package deploy-inputs.json overrides them, and NO_DEFAULT_CRED_INPUTS in manifest-lib.ts opts out gatekeepers that take no third-party OAuth app credentials (the wizard blocks Install on unfilled secret inputs, so a spurious default makes a gatekeeper uninstallable). Backend instance-state vars (ADMINS, DEPLOY_URL, ...) are injected by the deploy service at PUT time, never manifest-templated.

To test changes:

  • Run pnpm build to type-check, or vp run -F <package> build for one package — most packages declare build as a task rather than a script, and pnpm --filter cannot see a task. It is a type check and codegen pass, not a compile: every package but typed-storage is noEmit, because nothing imports the others' dist — wrangler and vite bundle from source. typed-storage emits because its exports resolves to dist/index.js. A re-run with nothing changed replays from the task cache — see below.

  • Run pnpm test to run unit tests, though as of this writing most packages don't have tests yet. It runs every package's test task, including @gadgets/scripts, whose suites run under node --test rather than vitest.

  • The cached per-package test run is a Vite+ test task in each package's vite.config.ts rather than a test script, so its input can exclude the scratch paths vitest writes and reads back (scripts/vitest-task-vite-config.ts, shared by all of them). Gatekeepers with a configurator UI re-export withTests from gatekeeper-configurator-vite-config.ts to get both tasks at once; the ones with no test files re-export its default instead, because vitest run exits 1 when it finds none.

    • integration-tests is the exception to the shared exclusions, via vitestTaskWithExclusions. Everywhere else .wrangler/validate is regenerated by the task that reads it, so it is dropped from the fingerprint; here the two trees it loads (workshop-backend's, from build:integration-worker, and the fixture gatekeeper's, from build:test-gatekeeper) are prebuilt by its dependencies and are the only path by which backend source reaches this suite at all — so they are tracked, and only .wrangler/{tmp,state} (wrangler's per-run scratch) are excluded. That works because capnweb-validate's output is byte-deterministic and neither worker rebuilds during the run: harness.ts deletes the Workshop's config.build when WORKSHOP_INTEGRATION_PREBUILT is set, and the fixture's wrangler.jsonc declares no build. build:integration-worker is correspondingly cached now, with {auto: true} minus **/.wrangler/** as input and an explicit output: ['.wrangler/validate/**'] — a cache hit has to leave the tree on disk, because its consumer reads it rather than rebuilding it. Its codegen prerequisites stay uncached, which is what keeps the generated modules current when its fingerprint is taken.
    • The same package's watch mode needs the Worker's inputs named explicitly, since wrangler bundles them outside Vitest's module graph. src/worker-inputs.ts is the single table behind the watcher roots, forceRerunTriggers and the delete handler in src/global-setup.ts — vitest's onFileDelete never consults forceRerunTriggers, so a deletion is rerun by hand there.
  • Every command in that shared test task runs under scripts/with-timeout.ts, so a wedged run fails fast instead of stalling the whole vp run: it kills the command's entire process tree and exits 124 (GNU timeout's code) after 60s with no output, or 600s in total. TESTS_WITH_TIMEOUT_DISABLE=1 turns it off.

  • Two ways to run one package's tests: pnpm --filter <package> test:run goes straight to vitest, vp run -F <package> test goes through the cache. The cached path replays instantly when the package is untouched, but its fingerprinting and archiving lose to plain vitest on a package you just edited — by more than the whole suite costs on a small one. Use test:run while iterating and pnpm test to verify. The direct script is test:run rather than test because a task may not share a name with a script.

  • pnpm build and pnpm clean are vp run --filter=!cloudflare-os <task> (through scripts/vp/run.ts, see the concurrency bullet below), not pnpm run --recursive <task>. Vite+ runs the same per-package scripts and tasks, in dependency order, but caches each one against its inputs, so an unchanged package replays its previous output instead of re-running. Commands joined with && — or given as an array in a task — are cached as separate entries, so a package whose codegen is fresh can still re-run its tsc. vp run --last-details explains every hit and miss, which is the thing to read when a build is slower than expected. Don't reintroduce a root script that calls pnpm run --recursive.

  • A cached vp run strips the environment. Each task and script sees only a built-in set (PATH, HOME, CI, NODE_OPTIONS, …). Anything else is invisible to the command and absent from the fingerprint, so a build that depends on an env var silently ignores it and no warning says so. A var can only be declared on a task — env/untrackedEnv don't exist on a package.json script — so any build that reads one has to be a task. workshop-frontend's build declares env: ['VITE_*'], which both forwards the flags and folds them into the fingerprint, so a changed value is a reported cache miss rather than a stale bundle replayed. Prefer env over untrackedEnv for anything that changes the output. scripts/env-passthrough.test.ts fails on any build-time env read that isn't accounted for.

  • Declaring env on one task does not help a sibling script that does the same work. A script duplicating a task's command takes the stripped path and the declaration buys nothing. So when you find a task with env, check what actually invokes that command. The configurator gatekeepers get it right by having no build script at all — build is a task that is only tsc and dependsOn: ['build:configurator']. gatekeeper-context and gatekeeper-scheduler take the other legitimate route, nesting vp run --cache build:app inside their build script, which vp inlines so the declaration still applies.

  • env fingerprints the value, not what it points at. workshop-backend's build is therefore cache: false, not env: ['FORMAT_BLUEPRINTS_DIR']: the variable names a blueprint directory outside the workspace, so with the path held fixed, edits inside it are invisible and a stale format-blueprints.ts replays. An uncached task runs with the full ambient environment, so it needs no env declaration. Same caution for any var naming a path outside the workspace.

  • pnpm build, pnpm test and pnpm clean use --filter=!cloudflare-os rather than -r because each root script is the vp run invocation: -r selects the root too, so it would re-enter itself and race a nested whole-workspace run against the outer one. Vite+ folds a nested vp run … back into its parent, but a node … script is opaque to that, so with these going through scripts/vp/run.ts the filter is the only thing preventing the recursion

  • "singleThreaded": true is set in the root tsconfig.json (and mirrored in the two standalone tsconfig.app.jsons that don't extend it), so every tsc run is single-threaded without per-script flags. tsgo's default mode splits the program across parallel checker instances with separate type caches, and since every file here touches capnweb's instantiation-heavy recursive generics, each checker re-derives the same huge type graphs: measured on workshop-backend, 6.9x the types and 4.2x the instantiations of a single checker, 7.3s/1.9GB vs 2.0s/0.7GB single-threaded (tsc 5.9 was 4.1s/0.9GB). Single-threaded is both the fastest and the smallest configuration, and it makes build tasks cheap enough for the computed concurrency (next bullet). Tests are the separate risk: the workerd fleets are the memory hogs there, and an OOM-killed (exit 137) workerd child wedges its vitest parent forever instead of failing. The watchdog above now turns that into an exit 124 after 60s of silence rather than a hang, so suspect memory first when you see one and drop vp's concurrency.

  • Caching is off for tasks that read a path they also write, which is why workshop-frontend's build excludes its own dist/ from input — without that it never cached (the gatekeeper SPA bundles are the same shape; see the build:app bullet below). The test tasks needed the same for the scratch paths vitest and wrangler write under node_modules/.vite, node_modules/.vite-temp and .wrangler; scripts/vitest-task-vite-config.ts covers which and why, and that list is unlikely to be closed — when a test task stops caching, --last-details names the path it read and wrote. This is also why no tsconfig sets incremental: tsc reads its own .tsbuildinfo and writes it back, taking the whole type check out of the task cache to save less than the cache does (measured: 2% hits and 21.1s on a clean tree, against 65% and 13.4s without it). Don't add it back without re-measuring vp run --last-details.

  • pnpm dev-server builds the gatekeeper UIs before starting wrangler dev, through two concurrent vp run -r --cache calls — build:configurator and build:app:dev (vp run takes one task each). The two runs share a single machine-derived concurrency limit, computed once and divided between them, floored at vp's default of 4 (scripts/vp/concurrency.ts), so together they stay inside the one budget instead of each claiming all of it. vp selects packages by which ones declare the task or script, so a new gatekeeper needs one of those to be built here at all, the same requirement pnpm build has. Watchers spawn only after the pre-flight finishes: every watch mode builds before it watches, so an earlier spawn would put two processes on the same src/generated files. Two dev-only details keep startup fast:

    • The generated dev configs spawn each worker's build.command binary directly (node <resolved entry>, resolved through the package's own node_modules) instead of through pnpm exec, which costs ~0.33s of process startup per call — paid for every worker, and again on each rebuild, all of it on the startup critical path. Commands that don't resolve are left as written. wrangler dev is reached the same way.
    • The app watchers are deferred until Wrangler is listening (TCP poll, 60s backstop): vite build --watch can't skip its initial build and these are the largest builds in the repo. Hence spawn rather than execFileSync at the tail of run-dev-server.ts — but shutdown is still driven by Wrangler's exit, because Ctrl-C reaches the whole process group and exiting out from under Wrangler would orphan its workerd children.
  • build:app is a Vite+ task in each gatekeeper's vite.config.ts rather than a package.json script so its input can be stated explicitly: {auto: true} minus **/dist-app/**, **/src/generated/** and **/.wrangler/** at base: "workspace", plus an explicit output. Automatic tracking alone never cached it, because the build writes into the same package tracking hashes as its input. Two traps: the exclusions have to be workspace-wide or the gatekeepers invalidate each other, and only directory contents can be excluded, not the directories themselves, so pnpm clean still costs one cold build.

  • build:app:dev is the same build with minify: false, run by the pnpm dev-server pre-flight so its app.txt matches what the watcher's un-skippable initial build will write — otherwise emitAppText rewrites the file and Wrangler restarts the worker mid-startup. It captures only app.txt, since dist-app/ has no reader outside vite.app.config.ts. build and deploy still use build:app, so nothing unminified ships, and build-app.mjs always sets GATEKEEPER_APP_UNMINIFIED explicitly — an inherited value would otherwise make a production build unminified and get it cached that way.

    Two structural constraints explain the file layout. Vite+ reads per-package settings only from vite.config.*, which the SPA's own build config occupied, so that moved to vite.app.config.ts (referenced by build-app.mjs -c, tsconfig.vite.json and gatekeeper-context's __tests__/vite-config.test.ts). And a task may not share a name with a package.json script, so the build:app script is gone and build calls vp run --cache build:app instead, deploy the same with --no-cache. Don't define the task in the workspace-root config: it gets created for every package, including the root, which then fails.

  • The packages whose tests run in workerd (router, typed-storage, backend-utils, workshop-backend, gatekeeper-scheduler, gatekeeper-cloudflare, gatekeeper-kit) load scripts/assert-workerd.ts as a setupFiles entry. It throws unless navigator.userAgent is Cloudflare-Workers, so a @cloudflare/vitest-pool-workers pool that fails to start fails the suite instead of silently falling back to Node — which otherwise looks like a pass in the packages that import no cloudflare:* module. Don't remove it to make a suite green.

Linting (oxlint, via Vite+):

  • pnpm lint runs what CI enforces: lint:check (oxlint), types:scripts and types:check. Run this before pushing.
  • Individual scripts:
    • pnpm lint:check / pnpm lint:fixvp lint, i.e. oxlint driven by Vite+ (rules in the lint block of the root vite.config.ts; correctness + suspicious as errors). Vite+ pins the oxlint it runs (1.76.0), so there is no separate oxlint dependency to drift from it and no .oxlintrc.json beside the config — one toolchain config, one version. Diagnostics are identical to what running that oxlint directly would emit.
    • pnpm types:check — an alias for pnpm build. They were separate scripts running the same recursive tsc twice; one name is kept for habit and the other because the codegen prerequisites hang off it. vp lint is not part of vp run, so it has no task cache; it takes about a second regardless.
  • Unused function parameters and caught errors are not lint-enforced; unused imports and local variables are still errors.
  • Some rules are kept as warnings (e.g. no-shadow) for incremental cleanup; warnings don't block CI.
  • Type-aware oxlint rules are intentionally not enabled yet. The type-aware engine is tsgo (TypeScript 7), which is now also the workspace tsc, and every package type-checks under it (capnweb 0.11's shared RpcPromise alias fixed the TS2321/TS2589 instantiation-depth errors that used to block three packages). But nobody has run the type-aware rules themselves: expect a triage pass, and note no-floating-promises conflicts with RPC promise pipelining (below), which intentionally leaves promises unawaited. Type safety is still enforced by tsc through pnpm types:check and pnpm build.
  • The typescript catalog entry is 7.0.2 (tsgo), but TS 7's main export is ./lib/version.cjs — the compiler API is gone from it — so everything that still needs that API gets its own JS-based compiler. scripts/build-gatekeeper-configurator.ts (and the mcp-shared schema test) import the root typescript6 alias (npm:typescript@6.0.3); capnweb-validate (0.2.4+) ships its own capped typescript dependency for the @validateRpc transform. typed-storage, the only package emitting declarations, sets the "rootDir": "./src" TS 7 requires (TS5011).
  • No tsconfig sets baseUrl, and none should. Every paths entry here is an explicit relative path, which tsc resolves against the tsconfig's own directory, so baseUrl bought nothing — and TypeScript 7 removed the option outright (TS5102).

IMPORTANT: This repository uses pnpm, not npm. Always use pnpm.

IMPORTANT: Remember when using RPC to use promise pipelining whenever possible. Cap'n Web implements promise pipelining (similar to Cap'n Proto). This means that if an RPC returns a stub, it's not necessary to await the RPC -- the promise itself can be used in place of the stub. Also, Cap'n Web lets you use the promise for a future result (even if it isn't a stub) in the arguments for another call; the promise will be replaced with its resolution on the server side before delivering the arguments. See the Cap'n Web README.md for more details.

IMPORTANT: When using React's useState(), the state value cannot be an RPC stub. At runtime, all stubs appear to be callable (because the system doesn't actually know if the stub points to a function on the server side or not). But the setter returned by useState() has different behavior if passed a function (including any callable object): it calls the function in order to get the state. In order to avoid this problem, whenever a useState() state will contain an RpcStub, it's important to wrap the stub in an object, and set the state to that object instead.

IMPORTANT: RPC stubs must be disposed to prevent resource leaks on the server side. Call stub[Symbol.dispose]() when the stub is no longer needed (or use a using declaration where possible). In particular, when a React component obtains a stub in a useEffect, the cleanup function should dispose the stub.

IMPORTANT: All RPC interface implementations should use the annotation @validateRpc() to apply capnweb-validate, which installs auto-generated runtime type validation matching the interface's TypeScript signatures. Do not write redundant validation code that duplicates the checks capnweb-validate already covers.

IMPORTANT: Server-side logging uses @gadgets/backend-utils/logger (frontend browser console.* is out of scope):

  • Define a package-owned field type and module-scoped logger with a stable dot-separated component and, for gatekeepers, vendorId: const logger = createLogger<GitHubLogFields>({ component: "gatekeeper.github", vendorId: VENDOR_ID });.
  • Emit concrete event names and relevant typed fields, for example: logger.warn("failed to notify credential expiry", { event: "credentials.expiry.notify.failed", error: err });. Each call emits one indexed object; module/child fields such as vendorId are inherited.
  • Use immutable logger.with(fields) for object-owned or nearby context. Prefer module/object loggers over logger parameters, and do not replace a shallow child logger with ambient context just to remove a local variable.
  • For bounded operation context needed by deep helpers, independent loggers, or other observability consumers, use createObservabilityContext from @gadgets/backend-utils/observability-context. Re-establish it per operation; it does not cross RPC, hibernation, or restart, and requires nodejs_als or nodejs_compat.
  • Pass caught values as error. The helper stringifies Error instances and primitives, uses an own string message for plain objects, omits undefined, and adds stacks to all Error logs. Keep this normalization deliberately small; do not traverse causes or copy arbitrary properties.
  • Extend field vocabularies locally. Levels: error needs attention, warn continues best-effort, info is notable lifecycle, and debug is noisy breadcrumbs. Never log secrets, prompts, headers, tokens, or request/response bodies.
  • To also dispatch a failure to the optional external issue Reporter (in addition to logging it), call reportIssue(failureSite, caught, options?) from @gadgets/backend-utils/error-reporting. Attach ambient fields from the package's observability context and augment them with capture-site fields: reportIssue("overseer.catalog-fallback", err, { handled: true, attributes: { ...obsContext.get(), gatekeeperId } });. It is a no-op when the ERROR_REPORTER binding is absent (local dev / deployments without an issue destination). Only bounded scalars are retained as attributes; reported context obeys the same no-secrets rules as log fields.

IMPORTANT: Frontend error reporting is a separate, opt-in path:

  • @gadgets/error-reporting owns the vendor-neutral browser/Worker event contract and tolerant, bounded normalization. VITE_FRONTEND_ERROR_REPORTING=true enables trusted frontend producers and their hidden source maps at build time; deployments without reporting should leave it unset.
  • The Workshop browser sends best-effort reports to the same-origin POST /api/client-errors endpoint. The backend dispatches only when both FRONTEND_ERROR_REPORTER and FRONTEND_ERROR_RATE_LIMITER are bound; otherwise the endpoint is an intentional no-op.
  • Gatekeeper management/configurator UIs run as Workshop-owned opaque-origin srcDoc frames. They send bounded reports with postMessage; the host accepts them only from the known frame window with origin null, adds host-owned surface/vendor context, and performs the same-origin POST. Do not add direct cross-origin reporting from a gatekeeper Worker domain.
  • Frontend reports never convey authority. reportedUserId is supplied by the client and unverified — the name records that it is a report, not a finding — so it is a diagnostic label only and must never be read to make a decision. pageLocation is origin and pathname only, rebuilt by normalizePageLocation rather than trusted from producers, because a share link's fragment is a bearer capability and an href also retains credentials; non-http(s) URLs are dropped entirely.
  • Install automatic capture only in trusted first-party surfaces, never gadget/user-authored code. Exception messages and stacks reach the external Reporter, so never intentionally put secrets, prompts, tokens, headers, or request/response bodies in thrown errors or report metadata.