Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ The project structure is:
* The trust boundary is `tools.ts`, and nothing outside it reads a tool's annotations: a tool the server declares `readOnlyHint: true` runs as an observation, everything else is queued for approval, and auto-*applying* a write additionally requires a `vetted` endpoint — which only the portal can produce, via `MCP_PORTAL_TRUST_ANNOTATIONS`.
* OAuth uses the official `@modelcontextprotocol/client`; always give SDK OAuth operations `sdkFetch(...)` so every request and redirect retains endpoint and SSRF checks.
* packages/gatekeeper-context: The Context Library — a gatekeeper whose account provides a singleton read session + a management UI, for authoring collections of context documents that agents read as observations. Collections have one of two visibilities: **private** (owned by a single account, readable/writable only by that account) and **public** (created/edited only by deployment admins, readable by everyone and auto-enabled for all users). It owns its state in three Durable Objects (`ContextCollectionDurableObject` for content, `UserLibraryDurableObject` for each account's own private collections, `LibraryRegistryDurableObject` for the domain's public set) plus a KV namespace. All data is namespaced by a `sharingDomain` (from the binding's props, see `domain.ts`) so multiple workshops sharing one gatekeeper instance stay isolated.
* Its `GatekeeperVendor` entrypoint (bound as `GATEKEEPER_CONTEXT`) declares `autoProvisionsAccount` and mints a `ContextAccount` via `createAccount()` (no user identity is passed in; the account keys its private data by its own generated `accountId`). The account exposes the agent read session (`getSession()`), collection discovery metadata (`getAgentCatalog()`), and a management UI (`startAppUi({ isAdmin })`). The UI is a single-file React SPA in `app/` (Vite + Tailwind + Kumo) bundled by `build-app.mjs` into `src/generated/app.txt`.
* Its `GatekeeperVendor` entrypoint (bound as `GATEKEEPER_CONTEXT`) declares `autoProvisionsAccount` and mints a `ContextAccount` via `createAccount()` (no user identity is passed in; the account keys its private data by its own generated `accountId`). The account exposes the agent read session (`getSession()`), collection discovery metadata (`getAgentCatalog()`), and a management UI (`startAppUi({ isAdmin })`). The UI is a single-file React SPA in `app/` (Vite + Tailwind + Kumo) bundled by `build-app.ts` into `src/generated/app.txt`.
* packages/gatekeeper-scheduler: Scheduled Tasks — an auto-provisioned gatekeeper whose account provides an ambient singleton for registering persistent workspace callbacks plus a read-only management UI. One account-scoped `ScheduleDriver` Durable Object stores enabled schedules and delivers them from a shared alarm; hook enablement remains in the Workshop Connections UI.
* packages/gatekeeper-cloudflare: Cloudflare OAuth, serving three unrelated purposes from one connected account — sign-in (`AUTH_GATEKEEPERS`), AI Gateway billing, and **Workers Observability** read-only telemetry resources. Two resource granularities (whole account, or one Worker) both map to the single indivisible `workers-observability.read` scope, so the capability boundary is the *binding*, not the grant.
* Layering: `observability-api.ts` is the only place that talks HTTP; `observability-parse.ts` validates every response (no blind casts); `observability-session.ts` is the agent-facing session, where every read funnels through one `#observe` so no path returns data unaudited; `observability-discovery.ts` derives field names/values from a sampled events query.
Expand All @@ -40,7 +40,7 @@ The project structure is:
* Worker-scoped bindings prepend an immutable `$metadata.service` filter *and* re-filter the response, since a filter the provider silently ignored would leak another Worker's telemetry. A dropped event proves the filter was not applied, so the provider's own `count` is then withheld rather than reported (it would count the whole account) and the drop is logged at `error`; `statistics` is kept, since it describes what the query cost rather than how much matched. Pagination cursors come from the provider's raw events rather than the surviving ones, so a fully-foreign page can't stall pagination and hide the caller's own older data. Trace *summaries* are account-only (their shape describes the whole cross-service trace); a Worker binding can still fetch its own events for a known trace id. `calculate()` is the one read with no second line of defence — an aggregate can't be un-mixed — so it rests solely on the injected filter; that is accepted and documented on the method, with the group-by fix left as a follow-up.
* A provider error message can quote a caller-supplied filter value back, so only its numeric `codes` are logged — filter *values* stay out of the audit trail (`summarizeFilter`), and the message travels to the caller who caused it.
* Tests are two vitest projects: `vitest.config.ts` (Node, pure logic) and `vitest.worker.config.ts` (workerd, for `RpcTarget`/`RpcStub`/Durable Objects). The workerd suite reaches the gatekeeper through a `TestHooks` Durable Object because a `DurableObjectClass` carrying `ctx.props` is only reachable via `ctx.facets` — the way the overseer instantiates it.
* `src/configurator/*.tsx` duplicate the resource-URL grammar from `resources.ts` and **must**: `build-gatekeeper-configurator.mjs` transpiles each per-file, stripping only `@gadgets/configurator-ui` and type-only imports, so they cannot import runtime helpers. `__tests__/configurator-url.test.ts` keeps the copies in step, and `configurator-fields.test.ts` drives `render` against a mocked runtime — the runtime's `clearFields` only drops an autocomplete's typed query, so a dependent field must *also* be nulled through `setValues` or the stale value silently survives into the resource URL.
* `src/configurator/*.tsx` duplicate the resource-URL grammar from `resources.ts` and **must**: `build-gatekeeper-configurator.ts` transpiles each per-file, stripping only `@gadgets/configurator-ui` and type-only imports, so they cannot import runtime helpers. `__tests__/configurator-url.test.ts` keeps the copies in step, and `configurator-fields.test.ts` drives `render` against a mocked runtime — the runtime's `clearFields` only drops an autocomplete's typed query, so a dependent field must *also* be nulled through `setValues` or the stale value silently survives into the resource URL.
* packages/router: The public origin of a deployed gadgets instance. Serves the workshop-frontend assets and routes by path prefix: `/api/*` and `/blueprint-screenshot/*` to the workshop backend, `/gatekeeper/<name>/*` to whichever gatekeepers are bound (discovered by scanning its own `GATEKEEPER_*` service bindings, so installing a gatekeeper is purely a binding change). The same worker doubles as the dev router (`pnpm dev-server`): with no `ASSETS` binding it proxies frontend requests to the Vite dev server instead.

Frontend conventions (Workshop, gatekeeper management apps, and shared UI):
Expand Down Expand Up @@ -98,9 +98,9 @@ To test changes:
- The generated dev configs spawn each worker's `build.command` binary directly (`node <resolved entry>`, resolved through the package's own `node_modules`) instead of through `pnpm exec`, which costs ~0.33s of process startup per call — paid for every worker, and again on each rebuild, all of it on the startup critical path. Commands that don't resolve are left as written. `wrangler dev` is reached the same way.
- The app watchers are deferred until Wrangler is listening (TCP poll, 60s backstop): `vite build --watch` can't skip its initial build and these are the largest builds in the repo. Hence `spawn` rather than `execFileSync` at the tail of `run-dev-server.ts` — but shutdown is still driven by Wrangler's exit, because Ctrl-C reaches the whole process group and exiting out from under Wrangler would orphan its workerd children.
- `build:app` is a Vite+ task in each gatekeeper's `vite.config.ts` rather than a package.json script so its `input` can be stated explicitly: `{auto: true}` minus `**/dist-app/**`, `**/src/generated/**` and `**/.wrangler/**` at `base: "workspace"`, plus an explicit `output`. Automatic tracking alone never cached it, because the build writes into the same package tracking hashes as its input. Two traps: the exclusions have to be workspace-wide or the gatekeepers invalidate each other, and only directory *contents* can be excluded, not the directories themselves, so `pnpm clean` still costs one cold build.
- `build:app:dev` is the same build with `minify: false`, run by the `pnpm dev-server` pre-flight so its `app.txt` matches what the watcher's un-skippable initial build will write — otherwise `emitAppText` rewrites the file and Wrangler restarts the worker mid-startup. It captures only `app.txt`, since `dist-app/` has no reader outside `vite.app.config.ts`. `build` and `deploy` still use `build:app`, so nothing unminified ships, and `build-app.mjs` always sets `GATEKEEPER_APP_UNMINIFIED` explicitly — an inherited value would otherwise make a production build unminified and get it cached that way.
- `build:app:dev` is the same build with `minify: false`, run by the `pnpm dev-server` pre-flight so its `app.txt` matches what the watcher's un-skippable initial build will write — otherwise `emitAppText` rewrites the file and Wrangler restarts the worker mid-startup. It captures only `app.txt`, since `dist-app/` has no reader outside `vite.app.config.ts`. `build` and `deploy` still use `build:app`, so nothing unminified ships, and `build-app.ts` always sets `GATEKEEPER_APP_UNMINIFIED` explicitly — an inherited value would otherwise make a production build unminified and get it cached that way.

Two structural constraints explain the file layout. Vite+ reads per-package settings only from `vite.config.*`, which the SPA's own build config occupied, so that moved to `vite.app.config.ts` (referenced by `build-app.mjs -c`, `tsconfig.vite.json` and gatekeeper-context's `__tests__/vite-config.test.ts`). And a task may not share a name with a package.json script, so the `build:app` script is gone and `build` calls `vp run --cache build:app` instead, `deploy` the same with `--no-cache`. Don't define the task in the workspace-root config: it gets created for *every* package, including the root, which then fails.
Two structural constraints explain the file layout. Vite+ reads per-package settings only from `vite.config.*`, which the SPA's own build config occupied, so that moved to `vite.app.config.ts` (referenced by `build-app.ts -c`, `tsconfig.vite.json` and gatekeeper-context's `__tests__/vite-config.test.ts`). And a task may not share a name with a package.json script, so the `build:app` script is gone and `build` calls `vp run --cache build:app` instead, `deploy` the same with `--no-cache`. Don't define the task in the workspace-root config: it gets created for *every* package, including the root, which then fails.
- The packages whose tests run in workerd (`router`, `typed-storage`, `backend-utils`, `workshop-backend`, `gatekeeper-scheduler`, `gatekeeper-cloudflare`, `gatekeeper-kit`) load `scripts/assert-workerd.ts` as a `setupFiles` entry. It throws unless `navigator.userAgent` is `Cloudflare-Workers`, so a `@cloudflare/vitest-pool-workers` pool that fails to start fails the suite instead of silently falling back to Node — which otherwise looks like a pass in the packages that import no `cloudflare:*` module. Don't remove it to make a suite green.

Linting (oxlint, via Vite+):
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
// `clearFields` in the sandbox runtime only deletes the autocomplete's typed query
// (`build-gatekeeper-configurator.mjs`: `delete queryByName[name]`) -- it does not touch `values`.
// (`build-gatekeeper-configurator.ts`: `delete queryByName[name]`) -- it does not touch `values`.
// So a dependent field has to be nulled through `setValues` as well, and forgetting the second half
// is invisible: `isReady` still passes and `resourceUrl` still builds, just pairing the new account
// with the old account's Worker. This test drives the real `render` against a recording JSX runtime
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
// The two configurator UIs build the resource URL that `parseObservabilityResourceUrl` then has to
// accept, but they cannot share the builders in `resources.ts`: each configurator module is
// transpiled on its own by `scripts/build-gatekeeper-configurator.mjs`, which only strips
// transpiled on its own by `scripts/build-gatekeeper-configurator.ts`, which only strips
// `@gadgets/configurator-ui` and type-only imports, so a runtime import would not resolve inside the
// sandboxed frame. The duplication is therefore deliberate, and this test is what keeps the copies
// honest: it runs each configurator's real `resourceUrl` and requires the result to round-trip through
Expand Down
2 changes: 1 addition & 1 deletion packages/gatekeeper-context/src/library-gatekeeper.ts
Original file line number Diff line number Diff line change
Expand Up @@ -149,7 +149,7 @@ export class ContextAccount
let ui = new RpcStub(new ContextApiImpl(
this.env, this.ctx.props.sharingDomain, this.ctx.props.accountId, context.isAdmin,
this.#collections(), this.#userLibraries(), this.#registries()));
// Bundled file-manager SPA (generated by build-app.mjs).
// Bundled file-manager SPA (generated by build-app.ts).
return { iframeHtml: APP_HTML, ui };
}

Expand Down
4 changes: 2 additions & 2 deletions packages/gatekeeper-context/vite.app.config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ import { viteSingleFile } from 'vite-plugin-singlefile'

const pkgDir = dirname(fileURLToPath(import.meta.url))

// `--watch` drives the `pnpm dev-server` hot loop (via build-app.mjs); one-shot build otherwise.
// `--watch` drives the `pnpm dev-server` hot loop (via build-app.ts); one-shot build otherwise.
const isWatch = process.argv.includes('--watch')

// Minification is the only thing that differs between the watch build and the one-shot build, so
Expand Down Expand Up @@ -39,7 +39,7 @@ function emitAppText(frontendErrorReporting: boolean): Plugin {
}
const outFile = resolve(pkgDir, 'src', 'generated', 'app.txt')
const contents =
`<!-- Generated from packages/gatekeeper-context/app by build-app.mjs. Do not edit. -->\n` + html
`<!-- Generated from packages/gatekeeper-context/app by build-app.ts. Do not edit. -->\n` + html
if (existsSync(outFile) && readFileSync(outFile, 'utf8') === contents) {
console.log(`app.txt unchanged (${(html.length / 1024).toFixed(0)} KiB), skipping write`)
return
Expand Down
4 changes: 2 additions & 2 deletions packages/gatekeeper-context/vite.config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ export default defineConfig({
// the gatekeepers would invalidate each other. Only directory *contents* can be excluded, not
// the directories, so a deleted output tree still costs one cold build.
'build:app': {
command: 'node build-app.mjs',
command: 'node build-app.ts',
dependsOn: ['clean:error-reporting-artifacts'],
input: [
{ auto: true },
Expand All @@ -52,7 +52,7 @@ export default defineConfig({
// rebuilds it moments later (`emptyOutDir`), so restoring it would be megabytes of writes on
// the startup critical path.
'build:app:dev': {
command: 'node build-app.mjs --dev',
command: 'node build-app.ts --dev',
dependsOn: ['clean:error-reporting-artifacts'],
input: [
{ auto: true },
Expand Down
4 changes: 2 additions & 2 deletions packages/gatekeeper-scheduler/vite.config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ export default defineConfig({
// the gatekeepers would invalidate each other. Only directory *contents* can be excluded, not
// the directories, so a deleted output tree still costs one cold build.
"build:app": {
command: "node build-app.mjs",
command: "node build-app.ts",
dependsOn: ["clean:error-reporting-artifacts"],
input: [
{ auto: true },
Expand All @@ -53,7 +53,7 @@ export default defineConfig({
// rebuilds it moments later (`emptyOutDir`), so restoring it would be megabytes of writes on
// the startup critical path.
"build:app:dev": {
command: "node build-app.mjs --dev",
command: "node build-app.ts --dev",
dependsOn: ["clean:error-reporting-artifacts"],
input: [
{ auto: true },
Expand Down
8 changes: 4 additions & 4 deletions packages/workshop-backend/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,11 @@
"scripts": {
"dev": "echo \"run 'pnpm dev-server' in the root directory instead\" >&2 && exit 1",
"deploy": "wrangler deploy",
"build:worker": "node build-browser-runtime.mjs && pnpm exec capnweb-validate build --out .wrangler/validate",
"build:worker": "node scripts/build-browser-runtime.ts && pnpm exec capnweb-validate build --out .wrangler/validate",
"clean": "rm -rf dist",
"test:integration": "node build-browser-runtime.mjs && node scripts/build-bundled-blueprints.ts && vitest run --config vitest.integration.config.ts",
"test:run": "node build-browser-runtime.mjs && node scripts/build-bundled-blueprints.ts && vitest run && vitest run --config vitest.integration.config.ts",
"test:watch": "node build-browser-runtime.mjs && node scripts/build-bundled-blueprints.ts && vitest",
"test:integration": "node scripts/build-browser-runtime.ts && node scripts/build-bundled-blueprints.ts && vitest run --config vitest.integration.config.ts",
"test:run": "node scripts/build-browser-runtime.ts && node scripts/build-bundled-blueprints.ts && vitest run && vitest run --config vitest.integration.config.ts",
"test:watch": "node scripts/build-browser-runtime.ts && node scripts/build-bundled-blueprints.ts && vitest",
"import:bundled-blueprint": "node scripts/import-bundled-blueprint.ts"
},
"dependencies": {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ import { dirname, resolve } from "node:path";
import { fileURLToPath } from "node:url";
import { build } from "esbuild";

const packageDir = dirname(fileURLToPath(import.meta.url));
const packageDir = resolve(dirname(fileURLToPath(import.meta.url)), "..");
const runtimeOutputFile = resolve(packageDir, "src/generated/browser-export-runtime.txt");
const sanitizerOutputFile = resolve(packageDir, "src/generated/html-sanitizer-runtime.txt");
const pageOutputFile = resolve(packageDir, "src/generated/browser-export-page.js");
Expand Down Expand Up @@ -39,7 +39,7 @@ writeIfChanged(runtimeOutputFile, runtimeResult.outputFiles[0].contents);
writeIfChanged(sanitizerOutputFile, sanitizerResult.outputFiles[0].contents);
writeIfChanged(pageOutputFile, pageResult.outputFiles[0].contents);

function writeIfChanged(outputFile, bytes) {
function writeIfChanged(outputFile: string, bytes: Uint8Array) {
const contents = new TextDecoder().decode(bytes);
if (!existsSync(outputFile) || readFileSync(outputFile, "utf8") !== contents) {
mkdirSync(dirname(outputFile), { recursive: true });
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ const { text: generated, count, totalBytes } = await generateBundledBlueprintsMo

// Skip the write when nothing changed. This script runs as a prerequisite of `build` and `test`,
// and rewriting an identical module would give it a fresh mtime, invalidating tsc's incremental
// cache for the whole package on every invocation. Same reason build-browser-runtime.mjs and the
// cache for the whole package on every invocation. Same reason build-browser-runtime.ts and the
// two SPA builds compare before writing.
let unchanged = false;
try {
Expand Down
2 changes: 1 addition & 1 deletion packages/workshop-backend/vite.config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ export default {
cache: false,
},
'build:browser-runtime': {
command: withTestTimeout('node build-browser-runtime.mjs'),
command: withTestTimeout('node scripts/build-browser-runtime.ts'),
cache: false,
},
/**
Expand Down
3 changes: 3 additions & 0 deletions pnpm-lock.yaml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading