Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 13 additions & 2 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,18 +5,29 @@ on:
branches: ["**"]
pull_request:

# CI needs to read the repo, nothing more.
permissions:
contents: read

jobs:
build:
name: typecheck · test · build
name: typecheck · lint · test · build (node ${{ matrix.node }})
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
# engines says >=20 and the release workflow builds on 24 — test what
# users actually run, not only the floor.
node: ["20", "22", "24"]
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: "20"
node-version: ${{ matrix.node }}
cache: npm
- run: npm ci
- run: npm run typecheck
- run: npm run lint
- run: npm test
- run: npm run build

Expand Down
11 changes: 10 additions & 1 deletion .github/workflows/dogfood.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,10 @@ on:
branches: ["**"]
pull_request:

# CI needs to read the repo, nothing more.
permissions:
contents: read

jobs:
coverage:
name: self-coverage (proofkeeper verifies proofkeeper)
Expand All @@ -29,11 +33,16 @@ jobs:
# install it as an external CLI from rac-core (its distribution is
# "requirements-as-code"; the console script is `rac`). Proofkeeper consumes
# only its published `rac export --graph` JSON output.
#
# Pinned to a commit: an unpinned default-branch install would execute
# whatever lands on rac-core main in this CI on every push — and make the
# gate non-reproducible. Bump the SHA deliberately when adopting a newer
# contract.
- uses: actions/setup-python@v5
with:
python-version: "3.11"
- name: Install the rac engine (provides the `rac` CLI)
run: pip install "git+https://github.com/itsthelore/rac-core.git"
run: pip install "git+https://github.com/itsthelore/rac-core.git@76e8feff9ae508cfbd4bc9a861c408901920da1d"

- name: Proofkeeper coverage over its own corpus (fails on any unverified)
run: node dist/cli.js coverage --corpus lore-proofkeeper/
10 changes: 10 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,16 @@ match exactly, or the workflow fails the publish.
summary), and `--verbose` streams a per-turn audit trail (tool calls,
errors, model latency) to stderr.

- **Hardened delivery.** The Dogfood gate installs the rac engine pinned to a
commit (reproducible CI, no default-branch supply-chain exposure); CI runs
at least privilege on Node 20/22/24 with a new lint gate (Biome). Config
typos now fail at parse time with the offender named (duplicate capability
ids, unknown environments/personas/defaultTarget) instead of silently
driving the wrong URL. Write-back re-runs are idempotent: an existing head
branch is re-pointed, an already-open PR is returned instead of a 422, and
the marked PR comment is found past one hundred comments. `## Verified By`
paths are always written POSIX-style.

## 2026.07.1 — the "any model" release

The release that makes **bring-your-own-model** mean *any* model — and proves Proofkeeper on itself. Everything since the first cut:
Expand Down
42 changes: 42 additions & 0 deletions biome.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
{
"$schema": "https://biomejs.dev/schemas/2.5.2/schema.json",
"vcs": {
"enabled": true,
"clientKind": "git",
"useIgnoreFile": true
},
"files": {
"includes": ["src/**", "tests/**", "!tests/fixtures/**"]
},
"formatter": {
"enabled": false
},
"linter": {
"enabled": true,
"rules": {
"preset": "recommended",
"complexity": {
"useLiteralKeys": "off"
},
"style": {
"noNonNullAssertion": "off",
"useTemplate": "off"
}
}
},
"assist": {
"enabled": false
},
"overrides": [
{
"includes": ["tests/compiler-emit.test.ts"],
"linter": {
"rules": {
"suspicious": {
"noTemplateCurlyInString": "off"
}
}
}
}
]
}
105 changes: 105 additions & 0 deletions lore-proofkeeper/designs/design-delivery-hygiene.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,105 @@
---
schema_version: 1
id: PK-KWFX3KXWBRKD
type: design
---
# Delivery Hygiene — Pinning, Permissions, Lint, and Idempotent Re-Runs

## Context

The final batch of a four-part review response: the first three hardened the
agent loop and its verdicts; this one hardens how Proofkeeper itself is built,
checked, and re-run. Everything here is process and edge hardening — no drive
or verdict semantics change.

## User Need

A maintainer must be able to trust that CI results are reproducible (not a
function of another repository's default branch), that the published package
works on the Node versions users actually run, that re-running a write-back
cannot fail or spam, and that a config typo is caught before a drive targets
the wrong URL.

## Design

- **Supply chain.** `dogfood.yml` installs rac-core at a commit SHA with a
comment stating the bump policy; both CI workflows gain
`permissions: contents: read`.
- **Matrix + lint.** The build job fans out over Node 20/22/24 and gains a
`npm run lint` step. Lint is Biome (single pinned dev dependency,
lint-only — the formatter stays off). Two recommended rules are disabled as
deliberate house style: bracket access on `Record<string, unknown>`
(`useLiteralKeys`) and post-validation non-null assertions
(`noNonNullAssertion`); `useTemplate` is off to avoid churn. The remaining
recommended set is enforced at zero diagnostics.
- **Config cross-validation.** `parseConfig` ends with a `validateReferences`
pass: duplicate capability ids, unknown `defaultTarget`, unknown capability
`environment`, and unknown `persona` references all raise `ConfigParseError`
naming the offender. `resolveTarget`'s fallback remains for
programmatically built configs, but a parsed config can no longer reach it
with a typo.
- **Idempotent GitHub layer.** `createBranch` catches the 422 "already
exists" and force-updates the existing head ref to the fresh base;
`openPullRequest` catches the 422 and returns the already-open pull request
for that head; `listComments` pages until a short page. All three keep
their error behavior for every other failure.
- **Portable references.** `linksFromResults` POSIX-normalizes spec and trace
paths before they become corpus `## Verified By` content.
- `parseCoverageArgs` uses the same `requireValue` guard as every other
parser.

## Constraints

- The lint baseline must hold at zero diagnostics — rules are configured off
explicitly rather than tolerated as warnings.
- Idempotency handling matches on GitHub's documented 422 semantics only;
unknown errors still throw.
- No behavior change for configs and write-backs that were already
well-formed.

## Rationale

Pinning by SHA (not tag) makes the dogfood gate exactly reproducible and
makes adopting a new contract an explicit, reviewable diff. Biome over
ESLint: one pinned binary, no plugin graph — a smaller supply-chain surface in
the same PR that pins the other one. Parse-time reference validation follows
the engine's own philosophy: refuse with a named offender rather than degrade
silently.

## Alternatives

- **Renovate/Dependabot-style automated pin bumps.** Deferred: process
machinery beyond this change's scope; the manual bump comment suffices at
current cadence.
- **ESLint + typescript-eslint.** Rejected for now: larger dependency
surface; the strict tsconfig already covers most type-adjacent rules.
- **Deleting stale head branches instead of re-pointing.** Rejected:
re-pointing preserves an open pull request and its review thread.

## Accessibility

Not applicable — CI configuration and internal validation; user surface is
error text that names the fix.

## Style Guidance

Validation errors name the artifact and the offending reference
("capability 'A' references undefined environment 'prod'") so the fix needs no
searching.

## Open Questions

- Whether the dogfood pin should move to a rac-core release tag once CalVer
releases are cut there regularly.

## Related Requirements

- req-delivery-hygiene

## Related Roadmaps

- autonomous-qa-enhancements

## Status

Accepted
67 changes: 67 additions & 0 deletions lore-proofkeeper/requirements/req-delivery-hygiene.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
---
schema_version: 1
id: PK-KWFX3K4WPAXN
type: requirement
---
# Delivery Hygiene — Supply Chain, Lint, Matrix, Idempotent Write-Back

## Problem

The delivery pipeline had hardening gaps a review surfaced. The dogfood gate
installed rac-core from its default branch unpinned, executing whatever landed
there in this repository's CI on every push — non-reproducible and a supply
chain exposure. CI workflows ran with the default token scope instead of
least privilege, on a single Node version while the release built on a newer
one. No lint tooling guarded style regressions the type checker cannot see.
Config typos degraded silently at run time (an unknown environment fell back
to the default URL; a duplicate capability id was accepted). And a re-run of
the write-back was fatal rather than idempotent: an existing head branch made
branch creation throw 422, a busy pull request's marked comment beyond the
first hundred was missed and duplicated, and Windows path separators could be
written into corpus `## Verified By` references.

## Requirements

- [REQ-001] The dogfood gate installs rac-core pinned to a commit SHA, bumped deliberately; CI workflows carry an explicit least-privilege `permissions` block.
- [REQ-002] The build job runs on every Node major the package supports (20, 22, 24), not only the floor.
- [REQ-003] A lint gate (`npm run lint`) runs in CI alongside typecheck/test/build.
- [REQ-004] Config cross-references fail at parse time with the offender named: duplicate capability ids, a `defaultTarget` or capability `environment` naming no defined environment, and a capability persona naming no defined persona.
- [REQ-005] Write-back re-runs are idempotent at the GitHub layer: an existing head branch is re-pointed, an already-open pull request for the head is returned rather than failed, and comment listing paginates past one hundred.
- [REQ-006] Corpus verifier references are written with POSIX separators regardless of the authoring platform.

## Success Metrics

- The dogfood job's install line names a SHA; changing rac-core main does not
change this repository's CI behavior until the pin is bumped.
- `npm run lint` exits 0 locally and in CI; a lint regression fails the build.
- A config with a typo'd environment name is rejected at parse with the
capability and name in the error.
- Running the write-back twice for the same capability yields one branch and
one pull request, updated in place.

## Risks

- A pinned rac can drift behind the contract. Mitigation: the pin is visible
in the workflow with a comment saying when to bump; contract changes arrive
as deliberate updates.
- Parse-time reference validation rejects configs that previously "worked" by
fallback. Mitigation: the errors name the fix, and silent wrong-target runs
were the worse failure.

## Assumptions

- GitHub's 422 messages for existing refs and pull requests keep containing
"already exists" (the strings matched for idempotent handling).
- Biome's recommended rule set, with the two house-style exceptions recorded
in `biome.json`, is a stable lint baseline.

## Related Roadmaps

- autonomous-qa-enhancements

## Verified By

- `tests/scope-config.test.ts`
- `tests/github-rest-gateway.test.ts`
- `tests/writeback-proposer.test.ts`
- `tests/qa-command.test.ts`
7 changes: 7 additions & 0 deletions lore-proofkeeper/roadmaps/autonomous-qa-enhancements.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,13 @@ failure from its siblings, cap observation growth, and make token spend and
per-turn conduct visible. Serves the fast-reliable-scoped-QA outcome under
real-world failure.

### Delivery hygiene

Pin the dogfood engine install to a commit, run CI at least privilege across
every supported Node major with a lint gate, fail config typos at parse time
with the offender named, and make write-back re-runs idempotent at the GitHub
layer. Serves the trust outcome for the pipeline itself.

## Success Measures

- A pull request shows exactly one Proofkeeper QA comment regardless of how many
Expand Down
Loading
Loading