Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion GOLD-STANDARD.html
Original file line number Diff line number Diff line change
Expand Up @@ -1234,7 +1234,8 @@ <h2>6. The harness standard</h2>
│ ├── correctness.yml # fresh agent (≠ grader, ≠ author) hunts changed lines for logic
│ │ # defects; blocks on a high-confidence defect, named override on record
│ ├── security.yml # security-reviewer on gated paths or risk:high label; blocks on HIGH
│ ├── deploy-dev.yml # merge to main -&gt; client dev environment
│ ├── deploy-dev.yml # merge to main -&gt; client dev environment (automatic, unattended)
│ ├── deploy-promote.yml # dev -&gt; test -&gt; prod; manual only, named approver every time
│ └── eval-*.yml # eval-regression + eval-suite (§11 agentic work)
├── scripts/rails/ # diff-anchors.sh, apply-branch-protection.sh
├── eval-datasets/ + prompts/ # golden-set template + versioned judge prompts (§11 work only)
Expand Down Expand Up @@ -1509,6 +1510,9 @@ <h2>10. The kit (what's in this repo)</h2>
│ ├── CLAUDE.md.template
│ ├── spec-template.md
│ ├── spike-template.md # the written finding a spike leaves behind (§5.3a)
│ ├── rollback-template.md # Phase 8: the written "roll back if X" + rehearsal record
│ ├── alert-definitions-template.md # Phase 9: baselines, thresholds, who is woken
│ ├── incident-playbook-template.md # Phase 9: detect/diagnose/escalate/communicate
│ ├── settings.json
│ ├── mcp.json # team MCP server set; packs merge additions
│ ├── HARNESS.md # developer-facing tour; installs to docs/harness.md
Expand All @@ -1518,6 +1522,8 @@ <h2>10. The kit (what's in this repo)</h2>
│ ├── hooks/ # stop-gate, review-gate, save-review-receipt (.ps1 + .sh each)
│ │ # + sensitive-edit-nudge (advisory example, unregistered)
│ ├── workflows/ # ci.yml, grader.yml, correctness.yml, security.yml, deploy-dev.yml
│ │ # + deploy-promote.yml (the deploy rail's second half: dev→test→prod,
│ │ # manual only, human go/no-go every time — §7)
│ │ # (+ eval-regression.yml, eval-suite.yml for agentic specs — §11)
│ ├── packs/ # composable additions: stacks/dotnet, cicd/github, cicd/azure-devops,
│ │ # frontend/generic, frontend/react, tools/gitnexus
Expand Down
17 changes: 17 additions & 0 deletions GOLD-STANDARD.md
Original file line number Diff line number Diff line change
Expand Up @@ -443,6 +443,18 @@ change.
never in specs. The Anthropic API key is client-procured (section 8).
- **Environments:** merge -> dev (automatic), dev -> test (on demand, smoke-tested), test -> prod
(Phase 8 ceremony and thereafter on the client's release cadence, human go/no-go every time).
Two workflows, deliberately separate: `deploy-dev` is automatic and unattended; `deploy-promote`
is manual-trigger only and cannot run without a named approver, because promotion beyond dev is
the standard's most protected stop. The go/no-go is the target environment's own approval
mechanism (GitHub required reviewers / Azure DevOps environment checks) rather than anything
hand-rolled — the client's security team can already audit it, and `deploy-promote` refuses to
run against an environment that has no approver configured. Neither workflow rebuilds: both ship
the exact artifact a named CI run produced, and a promotion is rejected unless the source
environment has already run those same bytes.
- **Rollback:** every deploy captures the last known-good version and restores it on failure. The
human path — the deploy that succeeded and went wrong an hour later — is written down in advance
in `ROLLBACK.md` (`kit/rollback-template.md`), including what a rollback does **not** undo, and
is proven by the client's own operators rehearsing deploy -> roll back -> redeploy in test.

---

Expand Down Expand Up @@ -541,6 +553,9 @@ intent-driven-development/ # cloned locally as delivery-standard/ on some ma
│ ├── CLAUDE.md.template
│ ├── spec-template.md
│ ├── spike-template.md # the written finding a spike leaves behind (§5.3a)
│ ├── rollback-template.md # Phase 8: the written "roll back if X" + rehearsal record
│ ├── alert-definitions-template.md # Phase 9: baselines, thresholds, who is woken
│ ├── incident-playbook-template.md # Phase 9: detect/diagnose/escalate/communicate
│ ├── settings.json
│ ├── mcp.json # team MCP server set; packs merge additions
│ ├── HARNESS.md # developer-facing tour; installs to docs/harness.md
Expand All @@ -550,6 +565,8 @@ intent-driven-development/ # cloned locally as delivery-standard/ on some ma
│ ├── hooks/ # stop-gate, review-gate, save-review-receipt (.ps1 + .sh each)
│ │ # + sensitive-edit-nudge (advisory example, unregistered)
│ ├── workflows/ # ci.yml, grader.yml, correctness.yml, security.yml, deploy-dev.yml
│ │ # + deploy-promote.yml (the deploy rail's second half: dev→test→prod,
│ │ # manual only, human go/no-go every time — §7)
│ │ # (+ eval-regression.yml, eval-suite.yml for agentic specs — §11)
│ ├── packs/ # composable additions: stacks/dotnet, cicd/github, cicd/azure-devops,
│ │ # frontend/generic, frontend/react, tools/gitnexus
Expand Down
6 changes: 4 additions & 2 deletions docs/companion/the-rails.html
Original file line number Diff line number Diff line change
Expand Up @@ -367,6 +367,7 @@ <h2>Five checks &mdash; and knowing which ones can say "no"</h2>
<tr><td><code>correctness</code></td><td>every PR that changes source</td><td><strong>Blocks</strong> on a high-confidence defect</td><td>A fresh AI agent &mdash; separate from the grader &mdash; hunts the changed lines for plain logic defects: the bug class ci can't see (it compiles, the tests pass) and security doesn't look for (it's not exploitable, just wrong). Passes trivially when no source changed; a named human can override on the record.</td></tr>
<tr><td><code>security</code></td><td>the <code>risk:high</code> label <strong>or</strong> any PR touching a registered gated path</td><td><strong>Blocks</strong> on HIGH; advises otherwise</td><td>Runs the security-reviewer agent. Path-triggered: fires on any PR touching a guarded path (auth, migrations, the pipeline, infra), independent of the spec's tier.</td></tr>
<tr><td><code>deploy-dev</code></td><td>merge to main</td><td>n/a (it ships)</td><td>Deploys the merged artifact to the client's dev environment, and restores the last good version when a deploy fails.</td></tr>
<tr><td><code>deploy-promote</code></td><td>manual only &mdash; never a trigger</td><td>n/a (it ships)</td><td>Promotes that same artifact onward to test and production. Holds until a named approver signs, and refuses a target environment that has no approver configured. Restores the last good version when a promotion fails.</td></tr>
</tbody>
</table>
<p><strong>The grader advises; it never blocks.</strong> It is tempting to let a confident AI
Expand Down Expand Up @@ -1690,6 +1691,7 @@ <h2>The five workflows</h2>
<tr><td><code>correctness</code></td><td>every PR that changes source</td><td><strong>Blocks</strong> on a high-confidence defect</td><td>A fresh AI agent, separate from the grader, hunts the changed lines for plain logic defects &mdash; the bug class ci can't see (it compiles, the tests pass) and security doesn't look for (just wrong, not exploitable). Trivial pass when no source changed; named-human override on the record.</td></tr>
<tr><td><code>security</code></td><td>the <code>risk:high</code> label <strong>or</strong> any PR touching a registered gated path</td><td><strong>Blocks</strong> on HIGH; advises otherwise</td><td>Runs the security-reviewer agent. Path-triggered: fires on any PR touching a guarded path (auth, migrations, the pipeline, infra), independent of the spec's tier.</td></tr>
<tr><td><code>deploy-dev</code></td><td>merge to main</td><td>n/a (it ships)</td><td>Deploys the merged artifact to the client's dev environment, and restores the last good version when a deploy fails.</td></tr>
<tr><td><code>deploy-promote</code></td><td>manual only &mdash; never a trigger</td><td>n/a (it ships)</td><td>Promotes that same artifact onward to test and production. Holds until a named approver signs, and refuses a target environment that has no approver configured. Restores the last good version when a promotion fails.</td></tr>
</tbody>
</table>
<p class="note">The grader <strong>advises; it never blocks</strong> &mdash; a polished, plausible
Expand Down Expand Up @@ -1740,8 +1742,8 @@ <h2>Deploy and promotion</h2>
<thead><tr><th>Step</th><th>How</th><th>Gate</th></tr></thead>
<tbody>
<tr><td>Merge &rarr; dev</td><td>The <code>deploy-dev</code> workflow ships every merged change to dev</td><td>Automatic &mdash; the merge already cleared the bar</td></tr>
<tr><td>Dev &rarr; test</td><td>Promoted deliberately, smoke-tested on arrival</td><td>On demand; the test environment is added at the first hardening pass</td></tr>
<tr><td>Test &rarr; prod</td><td>The first promotion is the Phase 8 go/no-go; every promotion after rides the client's release cadence</td><td>A human go/no-go <strong>every</strong> time &mdash; the single most protected stop in the standard</td></tr>
<tr><td>Dev &rarr; test</td><td>The <code>deploy-promote</code> workflow, run by hand, smoke-tested on arrival</td><td>On demand &mdash; and it will not start until a named approver signs</td></tr>
<tr><td>Test &rarr; prod</td><td>The same <code>deploy-promote</code> workflow. The first promotion is the Phase 8 go/no-go; every one after rides the client's release cadence</td><td>A human go/no-go <strong>every</strong> time &mdash; the single most protected stop in the standard, and enforced by the environment's own approval rule rather than by convention</td></tr>
</tbody>
</table>
<p>Two rules govern every promotion:</p>
Expand Down
2 changes: 1 addition & 1 deletion docs/phase-8-deployment.md
Original file line number Diff line number Diff line change
Expand Up @@ -218,7 +218,7 @@ change-control board adds latency.
| ------------------------------ | -------------------------------------------- | ---------------- | ------------------------------------------------------------------------------------------------------------------------ |
| Release notes | Claude (from the merged specs) | Pod Lead | Readable by people who never saw a spec: features, fixes, limitations, what changes for users |
| Deployment checklist | Claude (from the proven RUNBOOK) | Setup Owner | Every step ordered, observable, and verified in rehearsal; sign-off lines for the ceremony roles |
| Rollback procedure + evidence | Claude (drafts), client ops (executes) | Setup Owner | Executed in test by the client's own operators — deploy, roll back, redeploy — with the trigger condition written down |
| Rollback procedure + evidence | Claude (drafts from `kit/rollback-template.md`), client ops (executes) | Setup Owner | Executed in test by the client's own operators — deploy, roll back, redeploy — with the trigger condition written down, and what a rollback does **not** undo answered for this release |
| Rollout-shape decision | Product owner + operations | Product Owner | Cutover/pilot/parallel chosen; in-flight work answered; fallback and trigger conditions recorded |
| Smoke results (test + prod) | Claude (runs), Quality Engineer (owns) | Quality Engineer | One non-destructive journey per top-priority story, green in both environments, results recorded per test |
| The go/no-go record | Pod Lead | Pod Lead | Every named role asked and answered, decision and rationale recorded — the durable proof a human said go |
Expand Down
4 changes: 2 additions & 2 deletions docs/phase-9-monitoring.md
Original file line number Diff line number Diff line change
Expand Up @@ -242,8 +242,8 @@ gate falls at hypercare's end, closing both together.
| ------------------------- | -------------------------------------------- | --------------------- | -------------------------------------------------------------------------------------------------------------------- |
| Monitoring configuration | Claude (drafts), Setup Owner (wires) | Setup Owner → client | Dashboards live in the client's stack, each with a named owner; every top-priority feature observable |
| Production baseline | Claude (measures) | Quality Engineer | Normal recorded per key metric with its measurement period; modeled values flagged with revisit dates |
| Alert definitions | Claude (drafts), ops (confirm) | Client operations | Every critical failure mode covered; every threshold derived from baseline and confirmed by the people being paged |
| Incident playbook | Claude (drafts), ops (correct) | Client operations | Detect-diagnose-escalate-communicate per alert; templates included; cross-referenced to the RUNBOOK |
| Alert definitions | Claude (drafts from `kit/alert-definitions-template.md`), ops (confirm) | Client operations | Every critical failure mode covered; every threshold derived from baseline and confirmed by the people being paged |
| Incident playbook | Claude (drafts from `kit/incident-playbook-template.md`), ops (correct) | Client operations | Detect-diagnose-escalate-communicate per alert; templates included; cross-referenced to the RUNBOOK |
| The drill record | Quality Engineer | Quality Engineer | Every critical alert fired and answered by the client's on-call from the playbook; failures fixed and re-drilled |
| Engagement retrospective | Claude (evidence base), humans (the candor) | Pod Lead | Product and process findings with receipts; debt log; harvest list — concrete items, not platitudes |
| Close & Transfer handoff | Claude (drafts) | Pod Lead | Monitoring inventory, drill record, debt log, open items with owners |
Expand Down
41 changes: 41 additions & 0 deletions docs/the-rails.md
Original file line number Diff line number Diff line change
Expand Up @@ -314,6 +314,36 @@ if X") written down in advance, not invented mid-incident. The rails are not pro
being present; they are proven by a deploy failing and the rollback catching it. A rail that has
never failed safely has not been proven (section 9).

The trigger belongs in `ROLLBACK.md` (from `kit/rollback-template.md`), written while nobody is
under pressure, because the person deciding at 2 a.m. should be *executing* a decision rather than
making one. That file also forces the question teams skip: **what a rollback does not undo.** Code
reverts cleanly; state does not. A release carrying a destructive schema migration, a one-way data
transform, or a published message cannot simply be reversed, and the restored version may not
understand the data it now finds. If that question has no answer at the Phase 8 go/no-go, the
release is not ready to promote — that is a finding, not a footnote.

### The two halves of the deploy rail

Deploy is **two** workflows, and the split is the point:

- `deploy-dev` is automatic and unattended. It fires on a successful CI run on the protected
branch and needs no human, because the merge bar has already been cleared.
- `deploy-promote` is manual-trigger only. It cannot fire on its own, and it holds until a named
person approves.

The go/no-go is the target environment's **own** approval mechanism — GitHub Environment required
reviewers, Azure DevOps environment checks — not logic invented in a workflow file. Two reasons:
the client's security team can already audit it, and it cannot be quietly edited away without
branch protection noticing. `deploy-promote` refuses to run against a target environment that has
no approver configured, so "we forgot to set that up" fails loudly instead of silently promoting
to production unattended.

Both rules above are enforced mechanically, not by convention. Promotion is rejected unless the
named CI run succeeded, ran on the protected branch, *and* the source environment has already run
those exact bytes — which is what stops someone promoting a green build straight to production
having skipped test entirely. The pipeline would otherwise happily oblige, and nothing would say
so.

---

## 6. The infrastructure pipeline
Expand Down Expand Up @@ -458,6 +488,10 @@ deliberately and caught:
exact line, record the override label, watch it go green, close it unmerged. A blocking gate is
only proven when both its block and its escape have been seen to work.
- A known-bad deploy proves the **pipeline restores** the last good version.
- An attempted promotion proves the **go/no-go actually holds**: the run must pause for a named
approver, and a build that has only reached dev must be *refused* a promotion straight to prod.
A promotion path that sails through unapproved, or that lets you skip an environment, is not a
gate — and both failures look exactly like success until someone tries them.
- A probe PR touching a guarded path proves the **security gate** fires — a throwaway change opened
solely to confirm the gate triggers, then closed unmerged.

Expand Down Expand Up @@ -495,6 +529,13 @@ the provenance trail is what makes the rails auditable rather than merely automa
- **The rollback that was only ever written.** Documented, reviewed, never run — then executed for
the first time during an incident, where every surprise costs downtime. Rehearse it in test,
before it is needed.
- **The promotion gate nobody configured.** The promote pipeline exists, the environment exists,
and the environment has no approver on it — so every promotion sails straight through to
production with a green tick and no human in the loop. It looks identical to a working gate
right up until it matters, which is why the pipeline refuses to run rather than assuming.
- **The skipped environment.** A green build promoted from dev directly to prod because the
operator picked the wrong target and nothing checked. "The same build that passed test" is only
true if something enforces that it actually passed test.
- **A secret in the repo.** The one unrecoverable foundation mistake. The client's vault from day
one — never in code, never in CLAUDE.md, never in a spec.
- **The unattended destructive apply.** An agent runs an infrastructure `apply` that deletes or
Expand Down
13 changes: 12 additions & 1 deletion docs/whats-installed.md
Original file line number Diff line number Diff line change
Expand Up @@ -131,8 +131,19 @@ override. Branch protection tops it off: blocking checks are mandatory and a per
didn't write the change must approve it. In one sentence: *machines verify the facts; a human
makes the call.*

Getting a change into an environment is two workflows, deliberately not one. Merging ships to
**dev** automatically, with no human in the loop, because the change already cleared the bar
above. Going any further — to test, then to production — is a separate workflow that cannot
start on its own and stops until a **named person approves**. Neither one rebuilds anything: both
ship the exact package the build produced, so what reaches production is the thing that was
tested rather than a fresh copy nobody has seen. And a promotion is refused unless the previous
environment has already run those same bytes, so nobody can skip test by accident. If a deploy
goes wrong, the previous good version is restored automatically; the decision to undo a deploy
that went wrong *later* is written down in advance, in `ROLLBACK.md`, rather than improvised at
3 a.m.

Before trusting any gate, run the drills in `RAILS.md`: break the build on purpose, plant a
defect, commit a fake secret — and watch each gate catch it.
defect, commit a fake secret, attempt a promotion nobody approved — and watch each gate catch it.

## Where the pieces come from (custody chain)

Expand Down
Loading