This records the threat model, design decisions, and accepted residual risks for the Tau Ceti automated review system. It came out of a nine-angle adversarial audit (2026-06) and the hardening that followed (issues #7–#15, #22).
Production review generation currently runs from a trusted operator-run worker plus ad hoc trusted
contributors' command lines. The reusable CI workflow (.github/workflows/review.yml) can perform
the same metered-API review after a green build or an authorized /review, but Tau Ceti currently
disables that generation path to conserve budget. Either path runs each rubric through a read-only
reviewer CLI and posts a scoreboard plus per-rubric findings. The independent auto-merge workflows
consume the posted scoreboard.
The PR diff, description, comments, and file contents are attacker-controlled. Every reviewer is treated as potentially prompt-injected. The design does not try to prevent injection (it can't, see R1); instead it ensures an injected reviewer has nothing worth stealing and no power to merge on its own.
A secret reaches a public artifact (the public reviews branch or a PR comment) only if all
four links hold:
- the secret is present in the job — provider keys, the GitHub App token;
- the reviewer can reach it — via its environment or the filesystem;
- the attacker induces it into the model's output — prompt injection in the diff;
- we persist/publish that output — committed to the public branch, posted as a comment.
We deliberately keep transcripts public with no redaction gate (link 4 is open by choice), so safety rests entirely on breaking links 1–2: removing the reviewer's access to any secret.
- I1 — never evaluate PR-controlled Lake configuration in the privileged job; Mathlib source is cloned at the rev pinned in the base manifest. Dependency bumps may change only the manifest and toolchain pins, which a trusted validator reads as data and checks for forward movement before they are overlaid inside the secretless landrun sandbox. Lakefiles never enter the automated merge scope. Closes the pre-auth RCE.
- I2 — reviewers run in a clean workspace (PR source without
.git, roadmap, Mathlib, diff) with a minimal per-provider env: only that provider's key, never the other key and never a GitHub token. Keys are staged to files, read into memory, and unlinked before any reviewer runs.persist-credentials: falseon every checkout. codex usesshell_environment_policy.inherit=none. - I3 — the App token is split into two narrowly-scoped tokens (TauCeti
pull-requests:writeto comment; TauCetiReviewcontents:writeto persist), minted only after reviewers finish. - I4 —
/reviewrequires an exact command line (not a substring); per-PR daily round cap. - I5 — reserve-before-spend: skip a rubric if spend-so-far plus a per-call ceiling would breach the daily budget; count every attempt; persist spend incrementally.
- I6 — the verdict is parsed only from after a one-time random marker the runner injects per
review. Attacker content (including a forged
{"verdict":"approve"}) sits before the marker and is ignored; parsing fails closed on a missing marker, bad JSON, or an out-of-set verdict. - I7 — an approval is bound to the
head_sha+ a fingerprint of the rubric text. A new commit or a rubric edit re-runs all rubrics instead of carrying forward stale approvals. - I8 — reviewers are told to assume an adversarial, possibly same-model author and to verify substance rather than defer to fluent output (we accept self-declared provenance and treat any mismatch as the reviewer's problem to catch — no provider-exclusion machinery).
Merging is performed by the tauceti-review-bot GitHub App, which is a review-bypass actor
on main (no machine user account is used; a GitHub App cannot be a CODEOWNER). A PR is merged
only when all of:
- the newest marked scoreboard comment names the current commit and says every rubric approves;
- TauCeti CI's trusted
scopestatus is green on the current commit, and every changed path is underTauCeti/or an explicitly allowed root pin; lakefiles are never allowed; - any PR touching a Lake pin has a green trusted
bump-guard; - CI's
buildcheck is green (status checks are not bypassed, only the review requirement).
In Tau Ceti production, review generation and auto-merge are independent. The Review workflow's
enabled state plus the CI_REVIEW_ENABLED repository variable on TauCetiProject/TauCeti control
metered review generation; disabling it does not pause scoreboard-driven merging. The reusable
review workflow's enable_automerge input is not the production merge path. To pause automated
merging, disable both the Auto-merge and Merge sweep workflows in TauCetiProject/TauCeti; both
must be re-enabled to resume normal operation.
- R1 — prompt injection is not eliminable. No wording makes an LLM immune. Contained by zero-privilege reviewers and no unilateral merge authority; treat every reviewer as injected.
- R2 — two-provider independence ceiling. With only claude + codex, excluding the author would leave a single model deciding everything, so the gate is one model with no cross-check. Removing this needs a third provider or a human second-signal.
- R3 — fork auto-review depends on the sandbox staying airtight. GitHub-hosted runners are not a hard boundary against a determined attacker with network egress; we mitigate to "no secret is reachable to exfiltrate," which is the achievable bar.
- R7 — scoreboard trust is social. The merge workflows deliberately apply no
author-association gate to scoreboard comments. A PR author without repository write access can
post their own all-green board; because the newest board by
updated_atwins, posting a new board or editing an old one also supersedes a genuine changes-requested board. This can satisfy the review signal, although it cannot bypass the trustedbuild(which carries the axiom audit),scope, path-policy, orbump-guardchecks. Restricting who may supply the review signal would be a separate policy change. - R6 — a reviewer can read its own provider key via
/proc/self/environ(the CLI needs it to function). Blast radius is one key, only on model compliance with injection. Tracked in issue #22; the real fix is uid-separation or a local auth proxy. - Admin bypass.
enforce_adminsis off (human break-glass); the bot's token has no admin and cannot bypass status checks.
TauCeti CI compiles untrusted PR Lean, which runs arbitrary code at build time (elaboration,
initialize, macros, a PR lakefile). pull_request runs the PR's own workflow file, so a
sandbox written into ci.yml would be defeatable (the PR strips it). The build therefore lives
in a trusted, base-defined pull_request_target workflow (TauCeti/.github/workflows/pr-build.yml):
- Trusted definition + validated config. The PR cannot change what runs. The build/audit use
the base's
lakefileand scripts, overlaying only validated sources plus forward-only manifest and toolchain pins. Any Lake configuration or other infrastructure path is routed to a human, so a PR cannot redefine what "build"/"axioms" mean to fake a green check. - Sandbox. PR
TauCeti/code compiles only under landrun (pinnedv0.1.14+ SHA256), offline (no network), writes confined tobase/.lake, with/devnarrowed to specific nodes. A fail-closed self-test proves enforcement (out-of-tree write,/dev/shmwrite, and network all denied) before any PR code runs. Verified live: an adversarial build-time#evalhad its out-of-tree write and network egress denied. - No secrets, no token in reach.
permissions: contents:read, statuses:write, nosecrets.*,persist-credentials:false; the token is never in a landrun step's env. Mathlib is fetched via the trusted base checkout (no PR Lake on the network). Thebuildcommit status posted to the PR head SHA is the required merge check. - Fork auto-review.
pull_request_targetruns automatically for fork PRs (no maintainer- approval gate), so fork PRs get sandboxed CI + downstream review without a human gate.
CI residuals (accepted): the elan installer is fetched live (trusted source, runs before PR
code, no secrets); review's PR resolution from workflow_run could pick the wrong PR if one head
SHA maps to multiple PRs (low); the fork auto-run path relies on documented pull_request_target
semantics (same-repo path verified live; a true fork build not yet exercised). The post-merge
ci.yml build on main is trusted (already-merged code).
Public transcripts mean a leak is durable in git history. Response order: revoke the key first (rotate the provider key / regenerate the App key), then scrub history if needed. The mitigations above are designed so that there is no reachable secret to leak in the first place.