diff --git a/docs/architecture.md b/docs/architecture.md index 067f1b0f7..8b48871d8 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -6,7 +6,7 @@ - [Declarative Schema](#declarative-schema) - [Layers](#layers) -- [User Layer (CLI / PR Comments / API)](#user-layer-cli-pr-comments-api) +- [User Layer (CLI / PR Comments / API)](#user-layer-cli--pr-comments--api) - [Status Checks and Branch Protection](#status-checks-and-branch-protection) - [Apply Options](#apply-options) - [Unsafe Changes](#unsafe-changes) @@ -110,9 +110,44 @@ schemabot skip-revert -e staging # Finalize (Vitess) Users can also run `schemabot plan` manually in a PR comment to re-plan without waiting for auto-plan. +The same flow as a timeline: + +``` + Developer GitHub PR SchemaBot Database + │ │ │ │ + │ push schema change │ │ │ + │───────────────────────▶│ webhook │ │ + │ │──────────────────────▶│ auto-plan │ + │ │ │──────────────────────▶│ + │ │ plan comment (DDL) │ live schema │ + │ review the diff │◀──────────────────────│◀──────────────────────│ + │ │ │ │ + │ "schemabot apply │ │ │ + │ -e staging" │ webhook │ │ + │───────────────────────▶│──────────────────────▶│ re-plan, lock, │ + │ │ │ review gate, │ + │ │ │ start execution │ + │ │ │──────────────────────▶│ + │ │ progress comment │ execute DDL │ + │ │◀───── updates ────────│◀───── progress ───────│ + │ │ │ (row copy → cutover) │ + │ │ check run → green │ │ + │ │◀──────────────────────│ │ + │ merge the PR │ │ │ + │───────────────────────▶│ webhook │ │ + │ │──────────────────────▶│ clean up PR state, │ + │ │ │ release locks │ + │ │ │ │ +``` + +The staging apply above repeats for each environment in the database's +[environment order](configuration.md#environment-order); production applies are +additionally gated on the staging check and the +[review gate](configuration.md#review-gate). + **Check Runs** — SchemaBot publishes aggregate GitHub checks that block merge until managed schema changes are applied. See [Status Checks and Branch Protection](#status-checks-and-branch-protection) below. -**API** — HTTP endpoints that both CLI and webhook use internally. The SchemaBot server exposes `/v1/plan`, `/v1/apply`, `/v1/progress`, `/v1/cutover`, etc. +**API** — HTTP endpoints that both CLI and webhook use internally. The SchemaBot server exposes `/api/plan`, `/api/apply`, `/api/progress`, `/api/cutover`, etc. ### Status Checks and Branch Protection diff --git a/docs/auth.md b/docs/auth.md new file mode 100644 index 000000000..aabb1cfca --- /dev/null +++ b/docs/auth.md @@ -0,0 +1,678 @@ +# Authentication and Authorization + + + +## Table of Contents + +- [The adoption path](#the-adoption-path) +- [The trust surfaces](#the-trust-surfaces) +- [The two-tier API model](#the-two-tier-api-model) +- [The decision flow](#the-decision-flow) +- [API authenticators](#api-authenticators) + - [`none`: allow-all, but observable](#none-allow-all-but-observable) + - [`oidc`: Bearer tokens](#oidc-bearer-tokens) + - [`forward_auth`: authenticating proxy](#forward_auth-authenticating-proxy) +- [Calling SchemaBot as a service](#calling-schemabot-as-a-service) +- [Per-database operator scoping](#per-database-operator-scoping) +- [GitHub-side authorization](#github-side-authorization) +- [Group and team matching](#group-and-team-matching) +- [Fail-closed principles](#fail-closed-principles) +- [Observability](#observability) +- [💡 For AI agents](#-for-ai-agents) + + + +This document explains how SchemaBot decides who a caller is and what they may +do. Read it when you are standing up a SchemaBot server of your own: auth is +usually the first real decision an adopter hits, because a fresh server starts +with API authentication off and you have to choose how to turn it on for your +infrastructure. + +Callers reach SchemaBot through two front doors, and they authenticate +independently. The GitHub PR workflow authenticates against GitHub: webhooks +sign themselves, and PR commands are authorized against the commenting user's +GitHub identity. The CLI and the direct API authenticate against the server +itself, through the `auth` configuration most of this document covers. GitHub +is also optional: SchemaBot runs fine as a CLI-only deployment with no GitHub +App at all, and in that case the API authenticator is your entire auth story +and the GitHub sections of this document +([GitHub-side authorization](#github-side-authorization), the webhook surface) +do not apply to you. + +``` + the GitHub PR workflow (optional) the CLI and direct API + ┌──────────────────────────────┐ ┌──────────────────────────────┐ + │ GitHub ───▶ /webhook │ │ user / service ───▶ /api │ + │ │ │ │ + │ authenticated by the HMAC │ │ authenticated by auth.type │ + │ webhook signature; commands │ │ (none, oidc, forward_auth); │ + │ authorized against the │ │ authorized by the two-tier │ + │ commenter's GitHub identity │ │ model and operator scoping │ + └──────────────┬───────────────┘ └──────────────┬───────────────┘ + │ │ + └──────────────────┬──────────────────┘ + ▼ + SchemaBot server +``` + +You do not have to choose all at once. Auth is designed as a path: start with +nothing configured and a working deployment, and add exactly as much as your +infrastructure needs, when it needs it. + +## The adoption path + +**Step 0: run with auth off.** Leave `auth.type` unset. If you use the GitHub +PR workflow, it is already secure at this step: webhooks authenticate +themselves with HMAC signatures, and PR commands are authorized against GitHub +identity. What is +open is the direct API, so this step is right as long as the network around the +server is the boundary you intend, which is exactly true of local development +and often true of an internal cluster. You are not flying blind in the +meantime: every unauthenticated write is logged and counted, so even this step +leaves an audit trail. + +**Step 1: authenticate the API.** The trigger for this step is your +infrastructure, not SchemaBot. Integrating into real company infrastructure +means isolated networking boundaries and security requirements that expect +callers to carry identity, not just to be inside the right network. When the +API or CLI needs to be reachable across one of those boundaries, set +`auth.type`. Pick whichever matches what you already have: `oidc` if your +users can get a JWT from an identity provider, `forward_auth` if the server +sits behind an authenticating proxy or service mesh. At this step every authenticated user can read, and writes are limited +to the admin groups you name. For many deployments this is the destination: +humans read, admins operate, and changes flow through the PR workflow. + +```yaml +auth: + type: oidc + issuer: "https://issuer.example.com" + audience: "schemabot" +``` + +(Each authenticator's full configuration, including the `forward_auth` +equivalent, is shown in [its section below](#api-authenticators).) + +**Step 2: narrow reads, if you need to.** Reads are deployment-wide +visibility, not access to data, so most deployments leave them open to any +authenticated caller. Set `read_groups` only when even visibility needs a +boundary. + +**Step 3: grant teams their own databases.** When a database's owning team +wants to run changes directly (CLI instead of PR comments), grant their group +on that database with `operator_groups`, scoped to the environments you list +in `operator_environments`. Start with staging-style environments. The grant +is per-database and per-environment, so handing one team direct access to +their staging database changes nothing about anyone else's. + +**Step 4: decide how open production is.** Direct writes to production are a +bigger trade than they look, because scoped operators bypass the PR source +policy and with it the GitOps audit trail. How much of that trade to take is a +choice, and it depends on your workflow, not on a rule. A single-developer +setup can reasonably keep CLI write access everywhere: you are the audit +trail. (The PR workflow works well solo too: the review gate is optional, so +you get plans, checks, and history without needing a second person.) The two +doors also mix: a common rhythm is iterating quickly with the CLI against +local or staging databases, while GitHub stays the GitOps source of record for +what ships. A multi-person team is where PR review truly shines, so production +stays on the PR workflow where every change gets a second pair of eyes, while +staging-style environments stay open to direct access. And as a deployment +matures, best practice converges on GitHub-only production: production leaves +`operator_environments` entirely, every production change lands as a PR, and +the admin `write_groups` held by the platform team that operates SchemaBot +remain as the break-glass path. Direct write access is use-at-your-own-risk; +pick the point on that spectrum that matches how you work today. + +The rest of this document is the model behind those steps. Every configuration +field named below is specified in +[configuration.md](configuration.md#authentication), which is the YAML +reference. + +## The trust surfaces + +Requests reach a SchemaBot server on four distinct surfaces, and each surface +has its own authentication mechanism. Nothing is shared between them. A caller +trusted on one surface holds no standing on another. + +| Surface | Who calls it | Authenticated by | Authorized by | +|---|---|---|---| +| `/webhook` | GitHub | HMAC signature per configured GitHub App ([webhook secrets](configuration.md#multi-environment-deployment)) | PR command actor authorization and the [review gate](configuration.md#review-gate), both against GitHub identity | +| `/api` | Users and services (CLI, direct API) | `auth.type`: `none`, `oidc`, or `forward_auth` | The two-tier model below, plus per-database operator scoping | +| `/livez`, `/health`, `/tern-health/*` | Kubernetes probes, load balancers | Unauthenticated by design: probes carry no credentials | — | +| Metrics | Prometheus scrapers | Served on a dedicated listener, never on the API port | — | + +What the credential looks like on the wire, per surface: + +``` +POST /webhook X-Hub-Signature-256: sha256=… HMAC of the body +GET /api/status Authorization: Bearer eyJ… oidc +GET /api/status X-Forwarded-User: jane forward_auth, set by the proxy +GET /livez (no credentials) +``` + +One channel is deliberately absent from the table: the gRPC connection between +a control plane and its data planes. SchemaBot dials the endpoints listed in +`tern_deployments` and trusts what answers, so securing that channel (mTLS, +network policy, a service mesh) is your job as the platform operator. It is +infrastructure between your own components, not a caller to authenticate. + +The rest of this document covers the `/api` surface, and then the GitHub-side +authorization that gates PR comment commands. + +## The two-tier API model + +Every API request is classified into one of two tiers before any handler runs. +The **read tier** is visibility: `status`, `progress`, `logs`, listing locks, +history, database discovery, and `pull`, which exports a live schema. The +**write tier** is anything that stages or makes a change: `plan`, `apply`, the +control operations (`stop`, `start`, `cutover`, `volume`, `revert`, +`skip-revert`, `rollback`), lock acquire and release, and settings mutation. + +``` +read: GET /api/status GET /api/locks POST /api/pull +write: POST /api/plan POST /api/apply POST /api/locks/acquire +``` + +Two classification choices are deliberate, and both err toward write. + +**`plan` is a write.** Running a plan stages a change against a specific +database and reads its live schema, so it belongs to the change workflow, not +to open visibility. Viewing the results of an existing plan stays in the read +tier. + +**A route nobody classified is a write.** Classification is by method +(`GET` and `HEAD` are reads) plus a short, explicit list of non-GET endpoints +that are read-only anyway. Anything else requires write authorization, +including a brand-new endpoint someone adds next month and forgets to think +about. A route-sweep test drives every write-tier route against the same rule +the middleware uses, so an endpoint cannot ship without an authorization +decision behind it. + +The read tier is intentionally broad. Anyone admitted to it sees every +database's status, history, and schema, and there is no per-database scoping on +reads: visibility across the deployment is the point. Scoping exists only on +the write path, where what it protects is data rather than information. + +## The decision flow + +One API request, end to end. The right-hand column under "write" is the +two-phase decision that [per-database operator +scoping](#per-database-operator-scoping) explains in detail. + +``` + API request + │ + ┌──────────────┴──────────────┐ + │ exempt path? │──▶ bypass: probes are open, + │ /livez /health /webhook … │ webhooks verify HMAC + └──────────────┬──────────────┘ + │ /api/… + ▼ + ┌─────────────────────────────┐ + │ authenticate (auth.type) │──▶ 401 unauthenticated + │ none / oidc / forward_auth │ + └──────────────┬──────────────┘ + │ subject + groups + ▼ + ┌─────────────────────────────┐ + │ classify tier │ + │ GET/HEAD + read list → read │ + │ everything else → write │ + └──────┬───────────────┬──────┘ + read │ │ write + ▼ ▼ + ┌──────────────────┐ ┌────────────────────────┐ + │ in read_groups, │ │ in write_groups? │─ yes ▶ allow + │ write_groups, or │ │ (admin: every database)│ (admin) + │ operator_groups? │ └───────────┬────────────┘ + │ (empty = open) │ │ no + └────────┬─────────┘ ▼ + yes │ no ▶ 403 ┌────────────────────────┐ + ▼ │ in any database's │─ no ─▶ 403 + allow │ operator_groups? │ + └───────────┬────────────┘ + │ yes: admitted, target + │ not yet known (middleware) + ▼ + ┌────────────────────────┐ + │ handler resolves the │─ lookup ─▶ 500 + │ target database │ fails + └───────────┬────────────┘ + ▼ + ┌────────────────────────┐ + │ groups grant THIS │─ no ─▶ 403 + │ database, in an allowed│ + │ environment? │ + └───────────┬────────────┘ + │ yes + ▼ + allow (scoped) +``` + +The group checks shown are `forward_auth`'s, the richest of the three +authenticators. Under `oidc` a valid token clears the read tier, only the admin +groups clear the write tier, and there is no operator branch. Under `none` +everything is allowed, counted, and logged. + +A worked example: a caller whose forwarded groups include +`myorg/payments-team` sends `POST /api/apply` for the `payments` database in +`staging`. The middleware admits the request, because `myorg/payments-team` is +listed in some database's `operator_groups`. The handler then resolves the +target from the stored plan, sees that the caller's group grants `payments` +and that `staging` is in `operator_environments`, and allows. The same caller +targeting `production`, or any other database, gets a `403` at the handler. + +A few operations take the admin exit only. Settings mutation, checks +maintenance, and webhook redrive have no single target database to scope to, so +the operator branch never applies to them: `write_groups` or nothing. + +## API authenticators + +`auth.type` selects one of three authenticators. All three share the tier +classification above, and all three record every decision in the same metric. + +### `none`: allow-all, but observable + +The default. Every request is allowed and attributed to a synthetic anonymous +user. This suits local development, and deployments where the network is +genuinely the only boundary. + +```yaml +auth: + type: none # or leave auth unset entirely +``` + +Allow-all does not mean invisible. Every request is still recorded in the +auth-decision metric with reason `auth_disabled`, and every write operation is +logged with its method, path, and remote address. If someone port-forwards to +the pod and starts mutating things, a deployment running without authentication +has an alertable signal for it instead of silence: + +``` +schemabot.auth_decisions.total{tier="write", decision="allow", reason="auth_disabled"} +``` + +### `oidc`: Bearer tokens + +The server validates a JWT on each request against the issuer's public keys +(JWKS), fetched and cached via OIDC discovery. It never calls the provider at +request time, so any spec-compliant OIDC provider works. + +```yaml +auth: + type: oidc + issuer: "https://issuer.example.com" # required + audience: "schemabot" # required: the expected aud claim + groups_claim: "groups" # optional (default "groups") +``` + +Callers pass the token as a Bearer header; the CLI takes it via `--token` or +`SCHEMABOT_TOKEN`: + +``` +GET /api/status +Authorization: Bearer eyJhbGciOiJSUzI1NiIs… +``` + +The `aud` claim is required and always checked. Skipping it would accept a +token minted for any other application that happens to share the issuer. + +A valid token clears the read tier. The write tier additionally requires the +token's groups claim to include an admin group; the direct write path under +OIDC is intentionally for a small privileged set. Per-database operator groups +are not consulted on this authenticator. They belong to `forward_auth`. + +### `forward_auth`: authenticating proxy + +Use this when a reverse proxy has already authenticated the caller and forwards +the identity as HTTP headers. This is the pattern used by the Kubernetes API +server's authenticating proxy, Grafana's auth proxy, and oauth2-proxy. + +```yaml +auth: + type: forward_auth + forward_auth: + user_header: X-Forwarded-User # optional (default) + groups_header: X-Forwarded-Groups # optional (default) + # Trust anchor: configure at least one. + trusted_proxy_spiffe: + - spiffe://example.org/ns/ingress/sa/proxy + trusted_proxy_cidrs: + - 10.0.0.0/8 + read_groups: [] # empty = any authenticated caller reads + write_groups: [myorg/schema-admins] +``` + +A request as the server sees it, after the proxy has authenticated the caller: + +``` +POST /api/plan +X-Forwarded-User: jane +X-Forwarded-Groups: myorg/schema-admins,myorg/payments-team +X-Forwarded-Client-Cert: …;URI=spiffe://example.org/ns/ingress/sa/proxy +``` + +Forwarded headers are only as trustworthy as the path they arrived by, so the +server's job splits in two: first prove the request actually came from the +trusted proxy, then read the forwarded identity and apply the tiers. + +Proxy proof has three modes, selected by which trust anchors you configure: + +- **CIDR only.** Trust any request whose source IP is in a trusted network. +- **SPIFFE only.** Trust any request whose mTLS-verified client certificate, + read from the Envoy `X-Forwarded-Client-Cert` (XFCC) header, carries a + trusted SPIFFE ID. XFCC is itself a spoofable HTTP header, so this mode is + safe only when the surrounding mesh sanitizes inbound XFCC and the server is + not directly reachable. The server logs a startup warning in this mode to + make that dependence visible. +- **Both.** Require the trusted network and the trusted SPIFFE ID together, + for defense in depth. + +Configuring neither is a startup error, not an open door. + +Identity is read only from the canonical configured headers. A smuggled +underscore variant such as `X_Forwarded_User` is never consulted. Reads are +open to any authenticated caller unless `read_groups` narrows them; writes +require membership in `write_groups` or in a database's `operator_groups`, +covered in the next section. Membership in any of the three group lists +satisfies the read tier, so a write grant never needs a parallel read grant to +be usable. + +An optional second lane serves services calling as themselves rather than on +behalf of a user. It has its own section: +[calling SchemaBot as a service](#calling-schemabot-as-a-service). + +## Calling SchemaBot as a service + +Not every caller is a person. A schema inventory that pulls live schemas, a +dashboard that polls apply status, an agentic tool that queries schema +information across your fleet of databases, a replication control plane that +needs to know which tables exist and when they changed: these are services +calling SchemaBot as themselves, with their own identity rather than a +forwarded user's. Both authenticators support this, and on both the shape is +the same: the service gets the read tier, and every log line and metric +attributes the request to the service's own identity. + +The read tier is a complete consumer surface. SchemaBot is the system that +knows what is in your databases, and a service consumer typically builds on +four endpoints: + +| Endpoint | Returns | +|---|---| +| `GET /api/databases` | Every registered database, with type and environments | +| `POST /api/pull` | A live schema snapshot of one database | +| `GET /api/history/{database}` | Every apply for a database, with states and timestamps | +| `GET /api/progress/apply/{apply_id}` | Per-table detail for one apply: the DDL, change type, and timestamps | + +The expected call pattern is periodic sync: recurring `pull` calls across the +databases the service cares about, with `history` and `progress` answering +what changed and when. + +**Service callers are read-only by design.** The read tier already gives a +service everything it needs to observe the deployment. The write tier stays +with identities that carry accountability, a GitHub user on the PR workflow or +a person behind the CLI, so no configuration grants a service the write tier. +If a service ever needs to mutate something, that is a design conversation, +not a config change. + +Under `oidc`, a service is just another token holder. It presents a JWT minted +for SchemaBot's audience, typically through the client-credentials grant, and +a valid token clears the read tier. Writes additionally require an admin group +in the token's groups claim, which service tokens usually do not carry. + +Under `forward_auth`, forwarding a user identity does not fit a service, so a +dedicated lane exists. A caller-forwarding gateway terminates the service's +mTLS, verifies its client certificate, and forwards the verified SPIFFE ID in +a dedicated header (`caller_spiffe_header`). The server honors that header +only when everything lines up: the request provably arrived through a gateway +listed in `trusted_gateway_spiffe`, the forwarded caller is in the +`read_service_spiffe` allowlist, and the request is read-tier. The user and +groups headers are never read on this lane, so a gateway can vouch for a +service but can never inject a user identity. + +```yaml +auth: + type: forward_auth + forward_auth: + trusted_proxy_spiffe: # the lane requires SPIFFE-anchored trust + - spiffe://example.org/ns/ingress/sa/proxy + trusted_gateway_spiffe: # gateways allowed to vouch for a caller + - spiffe://example.org/ns/service-ingress/sa/gateway + read_service_spiffe: # services granted read-tier access + - spiffe://example.org/ns/reporting/sa/reporting + caller_spiffe_header: X-Forwarded-Caller-Spiffe-Id # optional (default) +``` + +A service request as the server sees it, arriving through the gateway: + +``` +GET /api/status +X-Forwarded-Client-Cert: …;URI=spiffe://example.org/ns/service-ingress/sa/gateway +X-Forwarded-Caller-Spiffe-Id: spiffe://example.org/ns/reporting/sa/reporting +``` + +There are no API keys, tokens, or auth headers anywhere in this lane. Identity +is the service's mesh certificate, end to end: + +``` + service pod gateway SchemaBot ++---------------------+ +---------------------+ +----------------------+ +| app sends plain | mTLS | verifies the pod's | mTLS | trusts the caller | +| HTTP to its sidecar |----->| certificate, strips |----->| header only from a | +| | | inbound copies of | | listed gateway, | +| sidecar presents | | the caller header, | | checks the caller | +| the pod's identity | | then sets it from | | against the | +| | | the verified cert | | allowlist, grants | ++---------------------+ +---------------------+ | the read tier | + +----------------------+ +``` + +Deny is the default at every step, which also makes onboarding concrete: you +do not have to guess a service's SPIFFE ID. Have the service make one call +before it is allowlisted; the denial log records the caller identity that +actually arrived, and that exact string is what you add to +`read_service_spiffe`. + +The lane fails closed like everything else. A gateway list without callers, +callers without a gateway, or the lane without SPIFFE-anchored proxy trust +each refuse to start; the pairing rules are in +[the configuration reference](configuration.md#forward-auth-authenticating-proxy). +The allowlist is exact, and every denial is logged and counted with its own +reason (an unlisted caller, a missing forwarded identity, a write attempt), so +a new service caller's rollout is verifiable from the auth-decision metric. + +## Per-database operator scoping + +`write_groups` grants every database in the deployment. That is the admin lane, +and for many deployments it is enough. Operator scoping exists for the next +step: letting a database's owning team mutate their own database directly, +without handing them the rest of the deployment. The database carries +`operator_groups`, and the server carries `operator_environments`, the +instance-wide list of environments where scoped writes are allowed at all. + +```yaml +auth: + type: forward_auth + forward_auth: + operator_environments: [staging] # scoped writes allowed here, instance-wide +databases: + payments: + type: mysql + operator_groups: [myorg/payments-team] + environments: + staging: + dsn: "env:STAGING_PAYMENTS_DSN" + production: + dsn: "env:PROD_PAYMENTS_DSN" +``` + +With this config, `myorg/payments-team` can plan and apply against +`payments` in `staging` directly, and nothing else: not `production`, and not +any other database. + +The decision happens in two phases because of an ordering problem: the +middleware runs before the request body is parsed, so at admission time the +server cannot know which database the request is about. + +1. **The middleware admits.** Any caller in `write_groups`, or in any + database's `operator_groups`, may reach write-tier endpoints. All this + establishes is "this caller may write to *something*". +2. **The handler enforces.** Each mutating handler resolves the target + database, from the request itself, the stored plan, or the stored apply, and + checks that the caller's groups grant that database in that environment. + +Locks are the one deliberate wrinkle. A lock has no environment dimension, so +an operator's lock grant applies to their database across every environment, +including ones outside `operator_environments`. That reach is fail-safe: a lock +only ever prevents changes, so the worst a staging-scoped operator can do with +it is hold production applies of their own database still. The reverse +direction is not fail-safe, and it is closed: force release bypasses the lock +ownership check and could undo another holder's safety brake, an admin's +incident lock for example, so force release stays admin-only. + +Know what you are trading before you grant this. Scoped operators bypass PR +source policy (`allowed_repos` and `allowed_dirs` are not evaluated for direct +API plans), so the grant exchanges the GitOps audit trail for direct access. +That is a reasonable trade for staging-style environments. Before granting a +production environment, add a trusted-source or stored-plan gate. + +## GitHub-side authorization + +PR comment commands (`schemabot apply`, `stop`, `cutover`, and the rest) carry +a GitHub identity, the comment author, and are authorized entirely within the +GitHub domain. API auth plays no part in them. + +```yaml +pr_command_authorization: + enabled: true + admin_teams: [myorg/db-admins] # may command any database +databases: + payments: + operator_teams: [myorg/payments-operators] # may command this database +``` + +Two independent checks apply: + +- **Actor authorization** (`pr_command_authorization`) checks the commenting + user against the admin teams and users, and against the target database's + `operator_teams` and `operator_users`, at command time via live GitHub + team-membership lookups. When the evaluation itself fails, a GitHub API + error for example, the command is blocked and a PR comment says so. A + failure to evaluate is never treated as a denial on the merits, and never + as an allow. +- **The [review gate](configuration.md#review-gate)** independently requires a + satisfying PR approval before apply. The author's own approval never counts. + +The field suffix tells you which identity domain you are granting. `*_teams` +and `*_users` are GitHub identities, verified by GitHub. `*_groups` are +forwarded identity groups, verified by the forward-auth proxy. The namespaces +do not overlap (a GitHub team slug has no meaning in a groups header), so each +lane is granted explicitly and a grant in one lane never leaks into the other. + +All of these are server-instance values owned by the platform operator. Keep +them in the server's own configuration, never in repo-editable config, where a +database team could grant themselves access with a self-merged PR. + +## Group and team matching + +A caller group matches a configured group on an exact string, or by final path +segment when at least one side is a bare slug with no `/` in it. The bare-slug +condition is what bounds the bridge: it lets an identity provider that emits +`schema-admins` match a configured `org/schema-admins`, without letting +`org-a/admins` match `org-b/admins` across organization boundaries. + +``` +caller "myorg/admins" config "myorg/admins" match (exact) +caller "schema-admins" config "myorg/schema-admins" match (bare-slug bridge) +caller "org-a/admins" config "org-b/admins" no match (both qualified) +``` + +If your proxy forwards groups from more than one organization, org-qualify the +entries of any list that decides a boundary, `operator_groups` especially. A +bare `admins` in config would match every organization's `admins`. + +When a decision is attributed to a group, the configured name is what gets +reported, not the caller's raw group string. Logs and metrics should name +something you can find in your own config. + +## Fail-closed principles + +Every layer resolves uncertainty toward denial, and misconfiguration toward +refusing to start. + +**Unclassified routes require write authorization.** Covered under +[the two-tier model](#the-two-tier-api-model): a new endpoint is guarded before +anyone has decided anything about it. + +**A denial and a failure are different answers, and they stay different.** A +caller who is simply not granted gets a `403` naming the groups that would +grant access. A target that cannot be resolved at decision time, because a +stored plan or apply lookup failed, surfaces as the operation's own `500` and +never as an authorization answer. GitHub-side evaluation failures block the +command without ruling on it. Collapsing these would turn every storage blip +into a confusing "permission denied". + +**Misconfiguration is a startup error, not a silent no-op.** A forward-auth +config with no trust anchor, an `operator_groups` grant without +`operator_environments` or the reverse, a grant on a database none of whose +environments are allowed, a service-caller lane missing its counterpart list: +each one refuses to start, because each one is a grant or a gate that could +never do what its author intended. A server that starts anyway would just +defer the surprise to the first request. The errors name the exact pairing: + +``` +forward_auth read_service_spiffe requires at least one trusted_gateway_spiffe gateway +``` + +**Unauthenticated mode is loud.** `auth.type: none` logs every write and counts +every request, so "we forgot to turn auth on" shows up on a dashboard rather +than in an incident review. + +## Observability + +Three metrics cover the authorization decision points end to end. Their full +attribute vocabularies are in [pkg/metrics/README.md](../pkg/metrics/README.md). + +| Metric | Layer | Decision it records | +|---|---|---| +| `schemabot.auth_decisions.total` | API middleware | Every API request: tier, allow or deny, and the reason (`auth_disabled`, `unauthenticated`, `not_admin`, …) | +| `schemabot.direct_write_authorization.total` | API handlers | Per-database direct-write decisions once the target is known: scoped allows, admin allows, and every denial reason | +| `schemabot.pr_command_actor_authorization.total` | Webhook | PR comment command actor decisions: which principal granted access, or why the command was blocked | + +A spike in denials on any of them means one of two things: someone is probing, +or a grant that should exist does not. Watch for deny counts by reason, for +example: + +``` +schemabot.auth_decisions.total{tier="write", decision="deny", reason="not_admin"} +``` + +Every deny is also logged with the subject, target, operation, and reason, so +a single denied request is triageable from logs alone. + +## 💡 For AI agents + +If you are an AI agent working against a SchemaBot deployment (or the person +wiring one up), the model above compresses to a few rules: + +- **An agent can safely use the CLI for reads, but writes require guardrails.** It's safe for an agent working + under supervision to call the CLI with a person's credentials + for read-tier operations. However for writes, identity is indistinguishable from the person acting, so keep write + operations in the person's own hands or in the PR workflow. If an agent is permitted to perform write actions, + it's strongly recommended to set ground rules for what can and cannot be done, and what should require + approval (use at your own risk). It's always recommended to use the GitHub PR workflow for production writes + with proper auth gates to prevent agents from performing unintended schema changes +- **The read tier is your surface, and it is complete.** Database discovery, + live schema pulls, apply history, and per-table progress cover what an agent + needs to reason about schemas across a fleet. See + [calling SchemaBot as a service](#calling-schemabot-as-a-service). +- **A 403 is an answer, not an obstacle.** Denials are the configuration + working as intended. The fix is an operator adding a grant (a group, an + allowlist entry), never retrying, credential switching, or proposing + `auth.type: none` to clear the error. +- **Onboarding is one denied call away.** Make a read call before you are + allowlisted; the denial log records the exact identity string an operator + needs to grant. +- **Schema changes go through the PR workflow.** If a task calls for changing + a schema, the path is a pull request editing the declarative schema files, + where plans, checks, and review gates apply. That is the same door humans + use, and it is why no configuration grants a service the write tier. diff --git a/docs/configuration.md b/docs/configuration.md index 0b7fdef8a..924309062 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -776,6 +776,8 @@ Approval is checked at the time of `schemabot apply` and `schemabot apply-confir ## Authentication +This section is the YAML reference. For the model behind it (the trust surfaces, why the tiers are drawn where they are, the fail-closed principles, and what to monitor), read [auth.md](auth.md). + By default (`auth.type: none` or unset) the SchemaBot API is unauthenticated — every request is allowed, which suits local development and deployments where the network is the only boundary. Setting `auth.type` turns on per-request authentication and a two-tier authorization model: - **Read tier** — visibility: `status`, `progress`, `logs`, `locks` (list), history, database discovery, and `pull` (read a live schema). diff --git a/docs/lint-and-safety-levels.md b/docs/lint-and-safety-levels.md index 98f1f35d6..60165c2b0 100644 --- a/docs/lint-and-safety-levels.md +++ b/docs/lint-and-safety-levels.md @@ -8,7 +8,7 @@ - [Lint severity levels](#lint-severity-levels) - [What "unsafe" means](#what-unsafe-means) - [Issues versus the unsafe-changes rejection](#issues-versus-the-unsafe-changes-rejection) -- [Blocked changes (⛔ Cannot apply)](#blocked-changes-cannot-apply) +- [Blocked changes (⛔ Cannot apply)](#blocked-changes--cannot-apply) - [Iconography reference](#iconography-reference) diff --git a/docs/namespaces.md b/docs/namespaces.md index db41ed23a..d498265d0 100644 --- a/docs/namespaces.md +++ b/docs/namespaces.md @@ -5,11 +5,11 @@ ## Table of Contents - [Schema Directory Structure](#schema-directory-structure) - - [MySQL — Single schema name](#mysql-single-schema-name) - - [MySQL — Multiple schema names on the same database](#mysql-multiple-schema-names-on-the-same-database) - - [MySQL — Different databases entirely](#mysql-different-databases-entirely) - - [Vitess — Multiple keyspaces](#vitess-multiple-keyspaces) - - [Vitess — VSchema changes](#vitess-vschema-changes) + - [MySQL — Single schema name](#mysql--single-schema-name) + - [MySQL — Multiple schema names on the same database](#mysql--multiple-schema-names-on-the-same-database) + - [MySQL — Different databases entirely](#mysql--different-databases-entirely) + - [Vitess — Multiple keyspaces](#vitess--multiple-keyspaces) + - [Vitess — VSchema changes](#vitess--vschema-changes) - [Where to Put the Schema Directory](#where-to-put-the-schema-directory) - [`$ENV` Substitution in Namespace Names](#env-substitution-in-namespace-names) - [Example](#example) diff --git a/docs/spirit_progress.md b/docs/spirit_progress.md index 437e51a26..7fc44437f 100644 --- a/docs/spirit_progress.md +++ b/docs/spirit_progress.md @@ -9,7 +9,7 @@ - [Spirit runner lifecycle](#spirit-runner-lifecycle) - [Engine layer](#engine-layer) - [How Spirit phases surface as task states](#how-spirit-phases-surface-as-task-states) -- [Tern layer — drive writes, readers read stored](#tern-layer-drive-writes-readers-read-stored) +- [Tern layer — drive writes, readers read stored](#tern-layer--drive-writes-readers-read-stored) - [What gets persisted in storage](#what-gets-persisted-in-storage) - [Polling modes (atomic vs sequential)](#polling-modes-atomic-vs-sequential) - [Atomic mode (`--defer-cutover`)](#atomic-mode---defer-cutover) diff --git a/scripts/gen-doc-toc.py b/scripts/gen-doc-toc.py index 1d899bcc5..3aaf7805a 100755 --- a/scripts/gen-doc-toc.py +++ b/scripts/gen-doc-toc.py @@ -22,11 +22,13 @@ def slugify(text: str) -> str: # GitHub's slugger: lowercase, drop punctuation except `-` and `_`, - # collapse whitespace to single `-`. Underscores are preserved because - # GitHub treats them as word characters in heading anchors. + # replace each whitespace character with `-`. Runs are not collapsed: + # "CLI / PR" leaves two spaces once the slash is dropped, and GitHub + # anchors that as "cli--pr". Underscores are preserved because GitHub + # treats them as word characters in heading anchors. text = text.strip().lower() text = re.sub(r"[^\w\s\-]", "", text, flags=re.UNICODE) - text = re.sub(r"\s+", "-", text) + text = re.sub(r"\s", "-", text) return text