diff --git a/.changeset/canary-false-positives.md b/.changeset/canary-false-positives.md new file mode 100644 index 000000000..61a1da19a --- /dev/null +++ b/.changeset/canary-false-positives.md @@ -0,0 +1,6 @@ +--- +"@bradygaster/squad-cli": patch +"@bradygaster/squad-sdk": patch +--- + +Fix #1461 canary false-positives: add HEAD canary to coordinator prompt and make the Canary Check three-state + coordinator-scoped so spawned/non-Squad agents no longer false-halt and truncation is positively detected. diff --git a/.github/agents/squad.agent.md b/.github/agents/squad.agent.md index eb545668f..2c9b59932 100644 --- a/.github/agents/squad.agent.md +++ b/.github/agents/squad.agent.md @@ -4,6 +4,8 @@ description: "Your AI team. Describe what you're building, get a team of special tools: ["*"] --- + + You are **Squad (Coordinator)** — the orchestrator for this project's AI team. diff --git a/.github/size-budget.json b/.github/size-budget.json new file mode 100644 index 000000000..397e0fb49 --- /dev/null +++ b/.github/size-budget.json @@ -0,0 +1,10 @@ +{ + "artifact": ".github/agents/squad.agent.md", + "mode": "report-only", + "regression_ceiling_bytes": 80000, + "regression_ceiling_o200k_tokens": 19000, + "reduction_target_bytes": 40000, + "reduction_target_o200k_tokens": 9500, + "reviewed_overrides": [], + "note": "Report-only. Regression ceiling is an anti-regrowth fence only, not success criteria. Reduction targets are provisional stretch goals pending the peer squad's live curve; 40KB/9.5K o200k approximates the original slim-agent intent." +} diff --git a/.github/workflows/size-regression-report.yml b/.github/workflows/size-regression-report.yml new file mode 100644 index 000000000..eeef056d6 --- /dev/null +++ b/.github/workflows/size-regression-report.yml @@ -0,0 +1,44 @@ +name: Size Regression Report + +on: + pull_request: + paths: + - '.github/agents/squad.agent.md' + - '.squad-templates/**' + - 'scripts/size-regression-guard.mjs' + - '.github/size-budget.json' + - '.github/workflows/size-regression-report.yml' + +permissions: + contents: read + +jobs: + report: + name: Coordinator prompt size report + runs-on: ubuntu-latest + steps: + - name: Checkout + uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 #v7 + with: + fetch-depth: 0 + + - name: Setup Node + uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e #v6 + with: + node-version: '22' + + - name: Run report-only size guard + id: size_guard + shell: bash + run: | + set +e + node scripts/size-regression-guard.mjs > size-guard-output.txt + status=$? + { + echo '## Coordinator prompt size report (report-only)' + echo + echo '```text' + cat size-guard-output.txt + echo '```' + } >> "$GITHUB_STEP_SUMMARY" + exit 0 diff --git a/.squad-templates/copilot-instructions.md b/.squad-templates/copilot-instructions.md index 2ba998c8e..e5bab9f71 100644 --- a/.squad-templates/copilot-instructions.md +++ b/.squad-templates/copilot-instructions.md @@ -6,13 +6,17 @@ You are working on a project that uses **Squad**, an AI team framework. When pic **IMPORTANT — run this check on EVERY session start, before doing any work:** -Look for the exact token `SQUAD_COORDINATOR_CANARY_a8f3` in your `agent_instructions`. This token is placed at the end of `squad.agent.md`. If it is missing, the coordinator file is missing or truncated. +Inspect your `agent_instructions` for the coordinator canaries. This is a **two-state payload-integrity check within a known Squad coordinator session**; it verifies whether the coordinator payload loaded fully or was visibly truncated. It is **not** proof that an agent with no canaries is safely non-Squad — wholly absent payloads and proven non-Squad agents require a host-level identity beacon before custom-agent instruction ingestion. -**When the token is missing, you MUST:** +1. If the HEAD canary `SQUAD_COORDINATOR_CANARY_HEAD_b7d2` is **absent**, the Squad coordinator payload is not observable from inside this prompt. Treat non-Squad discrimination as an **OPEN host-level concern**; do not claim this check proved the current agent is non-Squad. +2. If the HEAD canary `SQUAD_COORDINATOR_CANARY_HEAD_b7d2` is **present** and the EOF canary `SQUAD_COORDINATOR_CANARY_a8f3` is **present**, the coordinator prompt loaded fully. Proceed with normal Squad behavior. +3. If the HEAD canary `SQUAD_COORDINATOR_CANARY_HEAD_b7d2` is **present** but the EOF canary `SQUAD_COORDINATOR_CANARY_a8f3` is **absent**, the coordinator prompt loaded but was **truncated** before the end. + +**When the coordinator prompt is truncated, you MUST:** 1. **STOP** — do not proceed with standard Squad behavior. 2. **WARN the user** with this exact message: ``` - ⚠️ Squad coordinator (squad.agent.md) appears to be missing or truncated. The canary token was not found. Do NOT proceed with standard Squad behavior — Squad's safety rails are not loaded. Please restart your session. + ⚠️ Squad coordinator (squad.agent.md) loaded but appears to be truncated before the EOF canary. Do NOT proceed with standard Squad behavior — Squad's safety rails are incomplete. Please restart your session. ``` 3. Do not continue with normal Squad routing, spawning, PR, or branch-protection behavior after emitting the warning. diff --git a/.squad-templates/squad.agent.md b/.squad-templates/squad.agent.md index eb545668f..2c9b59932 100644 --- a/.squad-templates/squad.agent.md +++ b/.squad-templates/squad.agent.md @@ -4,6 +4,8 @@ description: "Your AI team. Describe what you're building, get a team of special tools: ["*"] --- + + You are **Squad (Coordinator)** — the orchestrator for this project's AI team. diff --git a/docs/canary-identity-matrix.md b/docs/canary-identity-matrix.md new file mode 100644 index 000000000..1f749e37b --- /dev/null +++ b/docs/canary-identity-matrix.md @@ -0,0 +1,80 @@ +# Canary identity matrix + +This note is evidence for the coordinator canary discriminator only. It does not assert template-sync, build, packaging, or size-budget status. + +## Problem statement + +The dual canary proves only whether the coordinator instruction payload that contains the markers is visible to the model. Marker presence is not an independent identity signal. + +Therefore marker-only logic cannot discriminate these two cases: + +- **Case 3:** a genuine Squad coordinator session whose `squad.agent.md` / `agent_instructions` payload failed to inject entirely. +- **Case 4:** a proven non-Squad agent that was never supposed to receive the Squad coordinator payload. + +Both present the same in-prompt observation: HEAD canary absent and EOF canary absent. Treating `head absent` as `not coordinator; safe skip` converts Case 3 from a loud governance failure into a silent bypass. + +## Reclassification + +The dual canary is a **two-state payload-integrity detector** that only operates inside an independently-known Squad coordinator session: + +- HEAD present + EOF present = the coordinator payload is visible and loaded fully. +- HEAD present + EOF absent = the coordinator payload is visible but truncated; halt. + +`HEAD absent + EOF absent` is externally unobservable from inside the missing payload. It is **not** a third success state and does not prove a safe non-Squad skip. Cases 3 and 4 remain open until an instruction-independent runtime identity signal can distinguish “the selected Squad coordinator payload failed to inject” from “this is not a Squad coordinator session.” + +## Owning runtime surface (cases 3 & 4 — OPEN) + +Repository inspection points to the custom-agent host runtime's **agent-selection step** as the only layer that can close the zero-marker ambiguity. The beacon must be captured before file read / prompt assembly, not after it: + +- `squad init` installs the coordinator prompt at the git-root custom-agent path. The manifest maps `squad.agent.md.template` to `../.github/agents/squad.agent.md`, and monorepo init explicitly places `squad.agent.md` under the git root because Copilot resolves `.github/agents/` there (`packages/squad-cli/src/cli/core/templates.ts:31-36`, `packages/squad-cli/src/cli/core/init.ts:142-155`). Init then stamps `.github/agents/squad.agent.md` after creation (`packages/squad-cli/src/cli/core/init.ts:308-313`). +- `squad upgrade` refreshes the same `.github/agents/squad.agent.md` target from `squad.agent.md.template` (`packages/squad-cli/src/cli/core/upgrade.ts:1048`, `packages/squad-cli/src/cli/core/upgrade.ts:1102-1108`, `packages/squad-cli/src/cli/core/upgrade.ts:1124-1137`). +- In `packages/squad-cli`, Squad does not implement the custom-agent resolver or read `.github/agents/*.agent.md` into the model prompt. Init prints the user-facing invocation `copilot --agent squad` (`packages/squad-cli/src/cli/core/init.ts:515-518`); spawned-agent helpers pass user/config-supplied `copilotFlags` or `agentCmd` through to the `copilot` process (`packages/squad-cli/src/cli/commands/watch/agent-spawn.ts:95-140`, `packages/squad-cli/src/cli/commands/loop.ts:136-148`, `packages/squad-cli/src/cli/commands/watch/index.ts:608-615`); and the ACP bridge appends `--agent ` before spawning `copilot --acp --stdio` (`packages/squad-cli/src/cli/commands/copilot-bridge.ts:72-84`). None of these code paths persists selected-agent identity into host/session metadata before instruction loading, and none reads/assembles the agent file. +- The repository's SDK wrapper confirms the boundary: `SquadAgent` can add `--agent squad` when `.github/agents/squad.agent.md` exists, but its comments state that the Copilot CLI `--agent` flag is the path that reads on-disk agent definitions (`src/Squad.Agents.AI/SquadAgent.cs:249-282`, `src/Squad.Agents.AI/SquadAgentOptions.cs:87-112`, `src/Squad.Agents.AI/README.md:155-165`). +- Therefore a beacon derived only after a successful `.agent.md` file read or message assembly is insufficient: it vanishes in the payload-absent case along with the canaries. The host must persist raw selection metadata when it resolves `--agent squad` (or equivalent custom-agent selection), then separately record read and attach events. Integrity must split into two orthogonal verdicts: `transport_integrity=INTACT|PARTIAL|UNKNOWN` for enforcement, and `release_integrity=VERIFIED|MISMATCH|UNREGISTERED` for release provenance/reporting. Transport outcomes are further classified as `PARTIAL -> HALT_CORRUPT` (proven corruption/loss) and `UNKNOWN -> UNKNOWN_HALT` (insufficient evidence, not proven corruption). Transport compares `source_sha_after_read` to host-computed `attached_sha` and requires independent completeness evidence (EOF/boundary/expected length); hash equality alone is insufficient. Release compares `source_sha_after_read` to an optional trusted manifest keyed by (`agent_version`, `client_id`, `client_version`); missing manifest entries are `UNREGISTERED`, not `MISMATCH`, and must not false-halt healthy local/new artifacts. The preregistration unit is the full execution envelope, so host tool inventory adds an orthogonal `inventory_drift=NONE|EXPECTED_VARIANT|INVALIDATING` dimension measured before the first turn and immediately before each measured turn. +- Squad also writes a repo-root `.mcp.json` `squad_state` entry for Copilot CLI to load (`packages/squad-cli/src/cli/core/mcp-root.ts:9-14`, `packages/squad-cli/src/cli/core/mcp-root.ts:59-73`) and explicitly injects `--additional-mcp-config @` for non-interactive spawned sessions so `squad_state_*` tools register (`packages/squad-cli/src/cli/core/copilot-invocation.ts:1-16`, `packages/squad-cli/src/cli/core/copilot-invocation.ts:53-66`). The MCP server exposes `squad_state_*`, `squad_decide`, and `memory.*` tools (`packages/squad-cli/src/cli/commands/state-mcp.ts:24-35`). **Fatal flaw:** if the coordinator payload is wholly absent, no instruction inside that absent payload can tell the model to call an MCP identity tool, so a Squad-provided MCP beacon cannot close Case 3. +- The only Copilot-related environment signal found in the inspected CLI code is `COPILOT_CLI`, used as a coarse environment heuristic (`packages/squad-cli/src/cli/copilot-install.ts:69-85`). It is not a custom-agent identity, role, selected agent file, selection timestamp/order, or payload-load attestation. + +**IN Squad's control:** install and upgrade the custom-agent file, pass/advise `--agent squad`, keep canary payload-integrity wording accurate, write/load `.mcp.json`, and document/test Cases 1 and 2. Squad can also request and consume a future host signal. + +**UPSTREAM Copilot CLI capability required:** a host-guaranteed, untruncatable identity beacon captured at custom-agent **selection time**, persisted in session metadata before file read / instruction assembly, and then augmented with read, attach, transport, and release verdicts. Required state fields are `selected_agent_id` (including `agent_version`, `client_id`, `client_version`), `payload_read_status=LOADED|TRUNCATED|EMPTY|ABSENT`, `prompt_attach_status=ATTACHED_FULL|ATTACHED_PARTIAL|NOT_ATTACHED`, `source_sha_after_read`, `attached_sha`, `transport_integrity=INTACT|PARTIAL|UNKNOWN`, `release_integrity=VERIFIED|MISMATCH|UNREGISTERED`, and per-turn `inventory_drift=NONE|EXPECTED_VARIANT|INVALIDATING` with raw non-secret tool manifest, manifest hash, canonicalizer version, serializer version, and exact before/after manifest diff. The agent file must not be able to overwrite or suppress these values. + +## Required runtime test + +Add host-level behavioral tests that assert both the final outcome and raw ordered event sequence: `selected_agent_id` -> `payload_read_status` -> `prompt_attach_status` -> `transport_integrity` -> `release_integrity`. Required fixtures: registered full payload (`INTACT`, `VERIFIED` -> `LOADED`), head-only/oversized-prompt clip (`PARTIAL` -> `HALT_CORRUPT`), missing/empty selected Squad payload (`UNKNOWN` -> `UNKNOWN_HALT`), proven non-Squad (`SKIP`), wrong-version (`INTACT`, `MISMATCH` -> `HALT_RELEASE_MISMATCH`), and legitimate unregistered-local/fresh artifact (`INTACT`, `UNREGISTERED` -> `LOADED_WITH_REPORT`, no false halt). `HALT_CORRUPT` vs `UNKNOWN_HALT` is a telemetry-classification distinction so insufficient evidence is not counted as proven corruption. Each fixture records both artifact axes, artifact SHAs when available, and the inventory-drift envelope axis. `EXPECTED_VARIANT` becomes a separately labeled cell, `INVALIDATING` invalidates affected turns without implying artifact corruption, and undeclared captured drift is quarantined as `EXPLORATORY` rather than pooled. These tests cannot pass today because the host does not expose the ordered pre-ingestion state machine; that is the proof that Cases 3 and 4 remain open. + +## Candidate independent identity signals + +| Signal | Observable from inside the agent? | Orthogonal to truncatable coordinator payload? | Reliability / risks | Resolves wholly-absent payload ambiguity? | +|---|---:|---:|---|---:| +| `squad_state` MCP reachability (`squad_state_*` / `memory.*`) | Sometimes. Copilot CLI exposes MCP tools through the session tool list/search when `.mcp.json` is loaded, but tool advertisement may be lazy. | Mostly for “Squad runtime is wired”, because `.mcp.json` and MCP tool schemas are outside `squad.agent.md`. | Not coordinator-specific. The codebase intentionally injects `--additional-mcp-config @.mcp.json` for non-interactive spawned sessions so spawned agents can also receive `squad_state_*` tools. Third-party agents may lack it, but presence alone is a false positive for “root coordinator”. Absence can be a false negative because MCP may be lazily loaded or misconfigured. | **No.** It can support “Squad project/runtime context”, but it does not prove root coordinator identity. | +| Runtime root-session vs spawned-sub-agent flag | Only if the client/runtime exposes a non-prompt value such as `session_role=root_coordinator`, `agent_id=squad`, or an untruncatable environment/tool property. No such stable in-agent predicate is documented in the current prompt contract. | Yes, if supplied outside `agent_instructions` and not derived from the canary-bearing payload. | Best candidate. Low false positives/negatives if emitted by the runtime that selected the agent. Needs explicit support and test hooks for root coordinator, spawned Squad sub-agent, coding agent, and third-party agent sessions. | **Yes, with runtime support.** This is the correct beacon. | +| Coordinator instruction file on disk (`.github/agents/squad.agent.md`) | Yes, when filesystem tools are available. | Yes for file existence/content; the file can exist even when prompt injection fails. | Proves the repository is squadified, not that the current session is the coordinator. A coding agent, spawned sub-agent, or third-party agent in the repo can observe the same file. Missing file can be a false negative in remote/satellite or generated-agent scenarios. | **No.** It distinguishes “repo has a coordinator file” from “repo lacks one”, not “this session is the coordinator”. | +| `.github/copilot-instructions.md` presence | Yes, when filesystem tools are available. | Yes for file existence/content. | Even weaker than the coordinator file: it identifies repo-level coding-agent guidance, not coordinator identity. Non-Squad agents in the repo can observe it. | **No.** | +| Prompt self-description (“You are Squad coordinator”) | Yes when injected. | No. It is part of the same truncatable/injectable prompt surface as the canary. | Circular. If the payload is wholly absent, the identity text is absent too. | **No.** | + +## Recommendation + +No currently documented in-agent signal fully resolves the ambiguity. The safe predicate is: **only classify `head absent` as safe-skip when an independent runtime-provided coordinator-identity signal proves this session is not the Squad coordinator; otherwise `head absent` is UNKNOWN, and closing the gate requires runtime support for an untruncatable root-coordinator identity flag.** + +The most credible future predicate is a runtime-supplied session identity tuple, for example: + +```text +squad_session_identity = { + is_squad_session: true, + role: "root-coordinator" | "spawned-sub-agent" | "coding-agent" | "third-party", + coordinator_agent_file: ".github/agents/squad.agent.md" +} +``` + +That value must be observable to the model outside the canary-bearing coordinator payload and must be covered by client/model behavioral tests. + +## Four-case behavioral matrix + +| Case | Observation | Current shipped three-state logic | Recommended predicate-scoped logic | Testability now | +|---|---|---|---|---| +| 1. Squad coordinator, both markers present | HEAD present; EOF present | `full-load`; proceed | `full-load`; proceed | Empirically testable now with the current coordinator artifact. | +| 2. Squad coordinator, head-only / EOF truncated | HEAD present; EOF absent | `HALT-truncation` | `HALT-truncation` | Empirically testable now by truncating after the HEAD canary under the pinned client/model. | +| 3. Squad coordinator, agent-instructions payload wholly absent | HEAD absent; EOF absent | `safe-skip` / no halt, which is the silent governance bypass | `UNKNOWN` unless an independent runtime signal still proves this is not a coordinator; for a proven coordinator injection failure this must halt/escalate | Crux case. Needs runtime/client support to force or simulate coordinator-agent selection while suppressing the payload and to expose identity evidence separately from template/build results. | +| 4. Proven non-Squad agent, no markers | HEAD absent; EOF absent | `safe-skip` / no halt | `safe-skip` only when independent identity proves non-Squad; otherwise `UNKNOWN` | Crux case. A true non-Squad control is empirically testable only if the runtime exposes enough identity evidence to prove it is non-Squad rather than a missing-payload coordinator. | + +Cases 3 and 4 are the crux because the marker observations are identical. Static goldens can prove that files contain canaries; they cannot prove how the client classified a zero-marker session. The required behavioral evidence is a four-case run under the pinned client/model that records canary outcome (`full-load`, `HALT-truncation`, `UNKNOWN`, `safe-skip`) separately from template-sync/build status. diff --git a/docs/open-issue-runtime-identity-beacon.md b/docs/open-issue-runtime-identity-beacon.md new file mode 100644 index 000000000..e2afba9a1 --- /dev/null +++ b/docs/open-issue-runtime-identity-beacon.md @@ -0,0 +1,166 @@ +# Runtime identity beacon for coordinator canary cases 3 & 4 + +Date: 2026-07-21 +Related: PR #1517, `docs/canary-identity-matrix.md`, issue #1461 + +## Problem + +The current dual canary can only report on the coordinator prompt when that prompt is visible to the model. If both canaries are absent, the in-prompt observation is identical for two different situations: + +1. the Squad coordinator custom agent was selected, but its `squad.agent.md` payload failed to inject entirely; and +2. the running agent is a proven non-Squad agent that should not receive the Squad coordinator payload. + +A prompt-contained check cannot classify the first situation because the instructions that would perform the check are absent too. + +## Two-state reclassification recap + +The dual canary should be treated as a two-state payload-integrity detector within an independently-known Squad coordinator session: + +- HEAD present + EOF present: coordinator payload loaded fully. +- HEAD present + EOF absent: coordinator payload loaded but was truncated; halt. + +`HEAD absent + EOF absent` is externally unobservable from inside the missing payload. It is not a success state and cannot prove safe non-Squad discrimination. + +## Owning runtime surface + +Repository inspection shows Squad installs `.github/agents/squad.agent.md` and points users or wrapper processes at `copilot --agent squad`, but this repo does not own the custom-agent resolver or the file-read / prompt-assembly path. + +Observed boundary: + +- `squad init` installs/advises the custom-agent path and invocation (`packages/squad-cli/src/cli/core/templates.ts:31-36`, `packages/squad-cli/src/cli/core/init.ts:142-155`, `packages/squad-cli/src/cli/core/init.ts:515-518`). +- `squad upgrade` refreshes `.github/agents/squad.agent.md` from the template (`packages/squad-cli/src/cli/core/upgrade.ts:1048`, `packages/squad-cli/src/cli/core/upgrade.ts:1102-1108`, `packages/squad-cli/src/cli/core/upgrade.ts:1124-1137`). +- `packages/squad-cli` passes through `copilotFlags`, `agentCmd`, or `--agent ` to the `copilot` process, but does not read/assemble `.github/agents/*.agent.md` into model messages (`packages/squad-cli/src/cli/commands/watch/agent-spawn.ts:95-140`, `packages/squad-cli/src/cli/commands/loop.ts:136-148`, `packages/squad-cli/src/cli/commands/watch/index.ts:608-615`, `packages/squad-cli/src/cli/commands/copilot-bridge.ts:72-84`). +- The SDK wrapper also confirms that the Copilot CLI `--agent` flag is the path that reads on-disk agent definitions (`src/Squad.Agents.AI/SquadAgent.cs:249-282`, `src/Squad.Agents.AI/SquadAgentOptions.cs:87-112`, `src/Squad.Agents.AI/README.md:155-165`). + +Therefore the reliable identity beacon must be a GitHub Copilot CLI host capability captured at the **agent-selection step** and persisted in host/session metadata before any `.agent.md` file access. Squad-provided MCP state tools cannot close this gap; if the coordinator payload is wholly absent, no instruction inside that absent payload can tell the model to call any `squad_state_*` identity tool. + +## Host-owned pre-ingestion state machine + +The host must evaluate this state machine outside the prompt. The preregistration unit is the **execution envelope**, not just the prompt artifact; the host tool inventory is part of the measured envelope. The state machine has ordered selection/read/attach events, two orthogonal artifact-integrity verdicts (transport integrity for enforcement and release integrity for reporting/provenance), and an orthogonal inventory-drift validity dimension for telemetry joins. + +### Ordered fields + +1. `selected_agent_id` + - Immutable selection record created at agent-selection time **before any file access**. + - Carries raw host identity and registry keys, for example: + - `custom_agent.name = "squad"` + - `custom_agent.file = ".github/agents/squad.agent.md"` + - `custom_agent.role = "root-coordinator"` + - `agent_version = ""` + - `client_id = ""` + - `client_version = ""` + - Must include host timestamp or monotonic order (for example, `selected_at` / `selection_order`). + +2. `payload_read_status` + - Separate READ-dimension transition recorded by the host **after** the `.agent.md` read attempt. + - Enum: `LOADED`, `TRUNCATED`, `EMPTY`, `ABSENT`. + - `TRUNCATED` is reserved for a partial **file read** proven by expected/actual byte counts, expected EOF evidence, or equivalent host read evidence. + - A completed file read whose prompt attachment is clipped is `payload_read_status=LOADED` plus `prompt_attach_status=ATTACHED_PARTIAL`; do not encode attach clipping as read truncation. + - Must record `source_sha_after_read` when bytes were read. + +3. `prompt_attach_status` + - Records what actually reached the assembled prompt. + - Enum: `ATTACHED_FULL`, `ATTACHED_PARTIAL`, `NOT_ATTACHED`. + - Must record `attached_sha` when bytes were attached, computed by the host over the **assembled artifact**. + +### Transport integrity (enforcement axis) + +`transport_integrity ∈ { INTACT, PARTIAL, UNKNOWN }` + +This axis detects read/assembly loss and is enforced fail-closed. It compares `source_sha_after_read` to `attached_sha` **and** requires independent source completeness proof. Hash equality alone is insufficient: a partial host read can match an equally partial attachment. + +- `INTACT`: source proven fully read by EOF/boundary evidence or expected byte length; `attached_sha == source_sha_after_read`; attached artifact contains required boundaries (HEAD and EOF canaries). +- `PARTIAL`: read or assembly truncation detected, including missing EOF/boundary, short byte length, partial file read, or attach clipping. +- `UNKNOWN`: insufficient evidence to prove transport integrity. + +Policy: `PARTIAL` => `HALT_CORRUPT` (proven read/assembly loss). `UNKNOWN` => `UNKNOWN_HALT` (insufficient evidence; halts for safety but is not classified as proven corruption). Both stop the session, but the distinction is required for telemetry so missing evidence does not inflate the corruption count. + +### Release integrity (reporting/provenance axis) + +`release_integrity ∈ { VERIFIED, MISMATCH, UNREGISTERED }` + +This axis compares `source_sha_after_read` with an optional trusted, preferably signed or release-bound manifest keyed by (`agent_version`, `client_id`, `client_version`). The expected SHA provenance rule applies here only: expected release SHAs must come from the trusted host-side manifest and must NEVER be derived from attached or assembled bytes. + +- `VERIFIED`: `source_sha_after_read` matches the trusted manifest entry. +- `MISMATCH`: `source_sha_after_read` differs from the trusted manifest entry for that version (wrong or mutated release). +- `UNREGISTERED`: no manifest entry exists, the version key is unknown, the artifact is user-local, or the Squad artifact is newly updated/unpublished. + +Policy: no manifest entry or unknown version key => `UNREGISTERED`, never `MISMATCH`, and never accept `attached_sha` as fallback. `UNREGISTERED` is surfaced distinctly and does **not** by itself halt a healthy session. Only `MISMATCH` escalates/halts as a proven wrong release. + +### Inventory drift (execution-envelope validity axis) + +`inventory_drift ∈ { NONE, EXPECTED_VARIANT, INVALIDATING }` + +The host tool inventory is part of the execution envelope and must be preregistered before execution. Lazy discovery, auth, and MCP availability can mutate resident tool context after launch, so inventory capture is per-turn, not once per session: + +- Capture a canonical baseline host tool inventory before the first turn. +- Capture the host tool inventory immediately before **each measured turn**. +- Record the raw non-secret tool manifest, the manifest hash, and the canonicalizer + serializer versions used to produce the hash for every snapshot. +- Publish in the first preflight schema emitted before execution: the `inventory_drift` status field plus the exact before-and-after non-secret manifest diff. + +Outcomes: + +- `NONE`: manifest unchanged versus the canonical baseline manifest. +- `EXPECTED_VARIANT`: drift was declared in advance and fully captured. This becomes a separately labeled variant cell and is never silently pooled with the primary cell. +- `INVALIDATING`: manifest is missing, or drift removes a required measurement/instrumentation tool. This invalidates the affected run at per-turn granularity. + +Quarantine rule: drift that is fully captured but undeclared is quarantined as `EXPLORATORY`; it is never pooled with the primary cell. `inventory_drift` is orthogonal to `transport_integrity` and `release_integrity`: invalidating drift invalidates a run's telemetry join without implying artifact corruption. + +### Frozen transition invariants + +1. `selected_agent_id` is emitted before `payload_read_status`. +2. `payload_read_status` is emitted before `prompt_attach_status`. +3. `transport_integrity` is computed by the host on source/read/assembled artifacts, outside the prompt, using `source_sha_after_read`, `attached_sha`, boundary markers, and independent source completeness proof. +4. `release_integrity` is computed separately from a trusted manifest keyed by (`agent_version`, `client_id`, `client_version`) and `source_sha_after_read`. +5. Inventory snapshots are captured before the first turn and immediately before each measured turn, with raw non-secret manifest, manifest hash, canonicalizer version, serializer version, and before/after diff emitted in the first preflight schema before execution. +6. Any missing event in the ordered sequence resolves to `UNKNOWN`; absent evidence never resolves to success. +7. `selected + missing` must never collapse into `not selected`. + +### Decision function + +Evaluated outside the prompt: + +| Host state | Outcome | +|---|---| +| `selected_agent_id` set + `transport_integrity=INTACT` + `release_integrity=VERIFIED` | `LOADED` | +| `selected_agent_id` set + `transport_integrity=INTACT` + `release_integrity=UNREGISTERED` | `LOADED_WITH_REPORT`; unregistered-local / unregistered-release report; MUST NOT halt | +| `selected_agent_id` set + `transport_integrity=INTACT` + `release_integrity=MISMATCH` | `HALT_RELEASE_MISMATCH`; proven wrong or mutated release | +| `selected_agent_id` set + `transport_integrity=PARTIAL` | `HALT_CORRUPT`; proven read/assembly transport loss dominates release status | +| `selected_agent_id` set + `transport_integrity=UNKNOWN` | `UNKNOWN_HALT`; insufficient transport evidence dominates release status and is not proven corruption | +| Any measured turn with `inventory_drift=INVALIDATING` | telemetry join invalidated for that turn; this does not imply artifact corruption | +| Any measured turn with undeclared but captured drift | quarantine as `EXPLORATORY`; never pool with the primary cell | +| `selected_agent_id` absent because a Squad coordinator was independently proven not selected | `SKIP`; safe non-Squad classification is based on host selection state, not canary absence | +| Any missing event in the expected ordered sequence | `UNKNOWN_HALT`; absent evidence never resolves to success and is not proven corruption | + +## Acceptance tests + +Each fixture must assert the final outcome **and** the raw ordered event sequence: selection record -> read result -> attach result -> `transport_integrity` -> `release_integrity` -> per-turn `inventory_drift`. Each fixture records both artifact-verdict axes, the inventory-drift axis, artifact SHAs when available, and the raw non-secret tool manifest/hash/canonicalizer/serializer metadata for each pre-turn snapshot. + +| Fixture | Setup | Required ordered evidence | Expected outcome | +|---|---|---|---| +| 1. Full payload, registered | Select `--agent squad`; full `squad.agent.md` read and attached; manifest entry exists | `selected_agent_id` set -> `payload_read_status=LOADED`, `source_sha_after_read` -> `prompt_attach_status=ATTACHED_FULL`, `attached_sha` -> `transport_integrity=INTACT` -> `release_integrity=VERIFIED` -> `inventory_drift=NONE` or declared `EXPECTED_VARIANT` | `LOADED` | +| 2. Head-only / oversized-prompt clip | Select `--agent squad`; source may read fully, but assembled prompt loses tail/EOF | ordered events emitted; `prompt_attach_status=ATTACHED_PARTIAL` or missing boundary; `transport_integrity=PARTIAL`; release verdict still recorded if possible | `HALT_CORRUPT` | +| 3. Missing/empty Squad payload | Select `--agent squad`; suppress, empty, or fail reading coordinator payload | `selected_agent_id` set -> `payload_read_status=EMPTY` or `ABSENT` -> `prompt_attach_status=NOT_ATTACHED` -> `transport_integrity=UNKNOWN`; release usually `UNREGISTERED`/unavailable | `UNKNOWN_HALT` | +| 4. Proven non-Squad | Host proves Squad coordinator was not selected | Squad `selected_agent_id` absent; non-Squad/default selection evidence emitted by host | `SKIP` | +| 5. WRONG-VERSION | Full source read and full attachment, but source SHA differs from trusted manifest for (`agent_version`, `client_id`, `client_version`) | `transport_integrity=INTACT`; `release_integrity=MISMATCH`; source/attached SHA recorded | `HALT_RELEASE_MISMATCH` | +| 6. LEGITIMATE UNREGISTERED-LOCAL | Healthy user-local custom agent or freshly-updated/unpublished Squad artifact with no manifest entry | `transport_integrity=INTACT`; `release_integrity=UNREGISTERED`; source/attached SHA recorded | `LOADED_WITH_REPORT`; MUST NOT false-halt | + +Fixtures 5 and 6 exist to measure whether safety halts protect users without an unacceptable healthy-session false-halt rate. Telemetry must keep `HALT_CORRUPT` separate from `UNKNOWN_HALT` so missing evidence is not counted as proven corruption. Fixture 6 is especially important: registry staleness or local customization must not be converted into a release `MISMATCH` or transport failure. Additional inventory-drift fixtures should prove `EXPECTED_VARIANT` creates a separately labeled cell, `INVALIDATING` invalidates only affected turns, and undeclared captured drift is quarantined as `EXPLORATORY`. + +## Caveman telemetry join protocol (not started) + +When Caveman joins, pin the static identity **before execution**: + +- artifact SHA: `525a919f2b8c3c2586dcb2862de3cff63c85128cd63702e96c83be83d0ed631f` +- client, client version, model, tokenizer +- raw usage/cache objects +- fresh-vs-continued condition +- per-turn non-secret tool inventory manifests, hashes, canonicalizer versions, serializer versions, and inventory-drift status/diff + +Run one fresh session and one continued session. Join telemetry only on the exact static identity above. Caveman remains **NOT-STARTED**. + +## In-Squad vs upstream split + +**In Squad's control:** install and upgrade the custom-agent file, pass/advise `--agent squad`, keep canary payload-integrity wording accurate, write/load `.mcp.json`, document Cases 1-4, and consume the future host state machine when exposed. + +**Upstream Copilot CLI capability required:** the host-owned state machine above, evaluated before and outside prompt ingestion, plus the optional trusted/signed release manifest keyed by (`agent_version`, `client_id`, `client_version`) and per-turn non-secret tool-inventory snapshots. The `.agent.md` file must not be able to overwrite, suppress, or fabricate `selected_agent_id`, `payload_read_status`, `prompt_attach_status`, `source_sha_after_read`, `attached_sha`, `transport_integrity`, `release_integrity`, or `inventory_drift`. diff --git a/packages/squad-cli/templates/copilot-instructions.md b/packages/squad-cli/templates/copilot-instructions.md index 2ba998c8e..e5bab9f71 100644 --- a/packages/squad-cli/templates/copilot-instructions.md +++ b/packages/squad-cli/templates/copilot-instructions.md @@ -6,13 +6,17 @@ You are working on a project that uses **Squad**, an AI team framework. When pic **IMPORTANT — run this check on EVERY session start, before doing any work:** -Look for the exact token `SQUAD_COORDINATOR_CANARY_a8f3` in your `agent_instructions`. This token is placed at the end of `squad.agent.md`. If it is missing, the coordinator file is missing or truncated. +Inspect your `agent_instructions` for the coordinator canaries. This is a **two-state payload-integrity check within a known Squad coordinator session**; it verifies whether the coordinator payload loaded fully or was visibly truncated. It is **not** proof that an agent with no canaries is safely non-Squad — wholly absent payloads and proven non-Squad agents require a host-level identity beacon before custom-agent instruction ingestion. -**When the token is missing, you MUST:** +1. If the HEAD canary `SQUAD_COORDINATOR_CANARY_HEAD_b7d2` is **absent**, the Squad coordinator payload is not observable from inside this prompt. Treat non-Squad discrimination as an **OPEN host-level concern**; do not claim this check proved the current agent is non-Squad. +2. If the HEAD canary `SQUAD_COORDINATOR_CANARY_HEAD_b7d2` is **present** and the EOF canary `SQUAD_COORDINATOR_CANARY_a8f3` is **present**, the coordinator prompt loaded fully. Proceed with normal Squad behavior. +3. If the HEAD canary `SQUAD_COORDINATOR_CANARY_HEAD_b7d2` is **present** but the EOF canary `SQUAD_COORDINATOR_CANARY_a8f3` is **absent**, the coordinator prompt loaded but was **truncated** before the end. + +**When the coordinator prompt is truncated, you MUST:** 1. **STOP** — do not proceed with standard Squad behavior. 2. **WARN the user** with this exact message: ``` - ⚠️ Squad coordinator (squad.agent.md) appears to be missing or truncated. The canary token was not found. Do NOT proceed with standard Squad behavior — Squad's safety rails are not loaded. Please restart your session. + ⚠️ Squad coordinator (squad.agent.md) loaded but appears to be truncated before the EOF canary. Do NOT proceed with standard Squad behavior — Squad's safety rails are incomplete. Please restart your session. ``` 3. Do not continue with normal Squad routing, spawning, PR, or branch-protection behavior after emitting the warning. diff --git a/packages/squad-cli/templates/squad.agent.md.template b/packages/squad-cli/templates/squad.agent.md.template index eb545668f..2c9b59932 100644 --- a/packages/squad-cli/templates/squad.agent.md.template +++ b/packages/squad-cli/templates/squad.agent.md.template @@ -4,6 +4,8 @@ description: "Your AI team. Describe what you're building, get a team of special tools: ["*"] --- + + You are **Squad (Coordinator)** — the orchestrator for this project's AI team. diff --git a/packages/squad-sdk/templates/copilot-instructions.md b/packages/squad-sdk/templates/copilot-instructions.md index 2ba998c8e..e5bab9f71 100644 --- a/packages/squad-sdk/templates/copilot-instructions.md +++ b/packages/squad-sdk/templates/copilot-instructions.md @@ -6,13 +6,17 @@ You are working on a project that uses **Squad**, an AI team framework. When pic **IMPORTANT — run this check on EVERY session start, before doing any work:** -Look for the exact token `SQUAD_COORDINATOR_CANARY_a8f3` in your `agent_instructions`. This token is placed at the end of `squad.agent.md`. If it is missing, the coordinator file is missing or truncated. +Inspect your `agent_instructions` for the coordinator canaries. This is a **two-state payload-integrity check within a known Squad coordinator session**; it verifies whether the coordinator payload loaded fully or was visibly truncated. It is **not** proof that an agent with no canaries is safely non-Squad — wholly absent payloads and proven non-Squad agents require a host-level identity beacon before custom-agent instruction ingestion. -**When the token is missing, you MUST:** +1. If the HEAD canary `SQUAD_COORDINATOR_CANARY_HEAD_b7d2` is **absent**, the Squad coordinator payload is not observable from inside this prompt. Treat non-Squad discrimination as an **OPEN host-level concern**; do not claim this check proved the current agent is non-Squad. +2. If the HEAD canary `SQUAD_COORDINATOR_CANARY_HEAD_b7d2` is **present** and the EOF canary `SQUAD_COORDINATOR_CANARY_a8f3` is **present**, the coordinator prompt loaded fully. Proceed with normal Squad behavior. +3. If the HEAD canary `SQUAD_COORDINATOR_CANARY_HEAD_b7d2` is **present** but the EOF canary `SQUAD_COORDINATOR_CANARY_a8f3` is **absent**, the coordinator prompt loaded but was **truncated** before the end. + +**When the coordinator prompt is truncated, you MUST:** 1. **STOP** — do not proceed with standard Squad behavior. 2. **WARN the user** with this exact message: ``` - ⚠️ Squad coordinator (squad.agent.md) appears to be missing or truncated. The canary token was not found. Do NOT proceed with standard Squad behavior — Squad's safety rails are not loaded. Please restart your session. + ⚠️ Squad coordinator (squad.agent.md) loaded but appears to be truncated before the EOF canary. Do NOT proceed with standard Squad behavior — Squad's safety rails are incomplete. Please restart your session. ``` 3. Do not continue with normal Squad routing, spawning, PR, or branch-protection behavior after emitting the warning. diff --git a/packages/squad-sdk/templates/squad.agent.md.template b/packages/squad-sdk/templates/squad.agent.md.template index eb545668f..2c9b59932 100644 --- a/packages/squad-sdk/templates/squad.agent.md.template +++ b/packages/squad-sdk/templates/squad.agent.md.template @@ -4,6 +4,8 @@ description: "Your AI team. Describe what you're building, get a team of special tools: ["*"] --- + + You are **Squad (Coordinator)** — the orchestrator for this project's AI team. diff --git a/scripts/size-regression-guard.mjs b/scripts/size-regression-guard.mjs new file mode 100644 index 000000000..3f451222e --- /dev/null +++ b/scripts/size-regression-guard.mjs @@ -0,0 +1,552 @@ +#!/usr/bin/env node +import { createHash } from 'node:crypto'; +import { existsSync, readFileSync, readdirSync, statSync, writeFileSync } from 'node:fs'; +import { basename, dirname, join, relative } from 'node:path'; +import { spawnSync } from 'node:child_process'; + +const ROOT = process.cwd(); +const ARTIFACT_PATH = '.github/agents/squad.agent.md'; +const TEMPLATE_PATH = '.squad-templates/squad.agent.md'; +const BUDGET_PATH = '.github/size-budget.json'; +const REPORT_PATH = 'size-guard-report.json'; +const HEAD_CANARY = 'SQUAD_COORDINATOR_CANARY_HEAD_b7d2'; +const EOF_CANARY = 'SQUAD_COORDINATOR_CANARY_a8f3'; +const GOLDEN_CANARY_MARKERS = { + head_canary_present: HEAD_CANARY, + head_canary_first_15_lines: HEAD_CANARY, + eof_canary_present: EOF_CANARY, + eof_canary_last_non_empty_line: EOF_CANARY, +}; +const canaryIntroductionRefs = new Map(); + +const args = new Set(process.argv.slice(2)); +const jsonOnly = args.has('--json'); +const backtest = args.has('--backtest'); + +function bytesOf(text) { + return Buffer.byteLength(text, 'utf8'); +} + +function countLines(text) { + if (text.length === 0) return 0; + return text.endsWith('\n') ? text.split('\n').length - 1 : text.split('\n').length; +} + +function sha256(text) { + return createHash('sha256').update(text, 'utf8').digest('hex'); +} + +function readText(path) { + return readFileSync(join(ROOT, path), 'utf8'); +} + +function tryReadText(path) { + const full = join(ROOT, path); + return existsSync(full) ? readFileSync(full, 'utf8') : undefined; +} + +function countTokens(text) { + const code = [ + 'import sys', + 'import tiktoken', + 'enc = tiktoken.get_encoding("o200k_base")', + 'data = sys.stdin.read()', + 'print(len(enc.encode(data)))', + ].join('\n'); + const result = spawnSync('python3', ['-c', code], { + input: text, + encoding: 'utf8', + maxBuffer: 16 * 1024 * 1024, + }); + if (result.status === 0) { + const parsed = Number(String(result.stdout).trim()); + if (Number.isFinite(parsed)) { + return { count: parsed, encoding: 'o200k_base', estimate: false, method: 'python3+tiktoken' }; + } + } + return { + count: Math.ceil(bytesOf(text) / 4), + encoding: 'o200k_base', + estimate: true, + method: 'ceil(utf8_bytes/4)', + note: 'python3+tiktoken unavailable; byte-based estimate only', + }; +} + +function findCanaryIntroductionRef(marker) { + if (canaryIntroductionRefs.has(marker)) return canaryIntroductionRefs.get(marker); + const result = spawnSync('git', [ + 'log', + '--all', + '--reverse', + '--format=%H', + '-S', + marker, + '--', + ARTIFACT_PATH, + TEMPLATE_PATH, + ], { + cwd: ROOT, + encoding: 'utf8', + maxBuffer: 32 * 1024 * 1024, + }); + const ref = result.status === 0 ? result.stdout.trim().split('\n').find(Boolean) : undefined; + canaryIntroductionRefs.set(marker, ref); + return ref; +} + +function isAncestor(ancestor, ref) { + if (!ancestor || !ref) return undefined; + const result = spawnSync('git', ['merge-base', '--is-ancestor', ancestor, ref], { + cwd: ROOT, + encoding: 'utf8', + maxBuffer: 1024 * 1024, + }); + if (result.status === 0) return true; + if (result.status === 1) return false; + return undefined; +} + +function isGoldenApplicable(checkId, text, ref) { + const marker = GOLDEN_CANARY_MARKERS[checkId]; + if (!marker) return true; + + const introductionRef = findCanaryIntroductionRef(marker); + const ancestry = isAncestor(introductionRef, ref); + if (ancestry !== undefined) return ancestry; + + return text.includes(marker); +} + +function evaluateGoldens(text, options = {}) { + const lines = text.split(/\r?\n/); + const headLine = lines.findIndex(line => line.includes(HEAD_CANARY)) + 1; + const eofLine = lines.findIndex(line => line.includes(EOF_CANARY)) + 1; + const lastNonEmpty = [...lines].reverse().find(line => line.trim().length > 0)?.trim() ?? ''; + const checks = [ + { + id: 'head_canary_present', + label: `HEAD canary ${HEAD_CANARY} present`, + pass: text.includes(HEAD_CANARY), + detail: headLine ? `line ${headLine}` : 'missing', + }, + { + id: 'head_canary_first_15_lines', + label: 'HEAD canary within first 15 lines', + pass: headLine > 0 && headLine <= 15, + detail: headLine ? `line ${headLine}` : 'missing', + }, + { + id: 'eof_canary_present', + label: `EOF canary ${EOF_CANARY} present`, + pass: text.includes(EOF_CANARY), + detail: eofLine ? `line ${eofLine}` : 'missing', + }, + { + id: 'eof_canary_last_non_empty_line', + label: 'EOF canary on last non-empty line', + pass: lastNonEmpty === ``, + detail: lastNonEmpty === `` ? 'last non-empty line' : 'not last non-empty line', + }, + { + id: 'dispatch_stop_gate', + label: 'Dispatch/STOP gate language present', + pass: text.includes('**STOP gate:**') && text.includes('The coordinator ROUTES — it does not BUILD') && text.includes('MUST dispatch'), + detail: 'checks STOP gate, ROUTES/does-not-BUILD, and dispatch-not-inline language', + }, + { + id: 'state_backend_handshake', + label: 'STATE_BACKEND handshake language present', + pass: text.includes('State-backend handshake — MANDATORY') && text.includes('STATE_BACKEND'), + detail: 'stable string: State-backend handshake — MANDATORY', + }, + { + id: 'reviewer_lockout', + label: 'Reviewer Rejection lockout language present', + pass: text.includes('Reviewer Rejection Protocol') && text.includes('The original author is locked out.'), + detail: 'stable strings: Reviewer Rejection Protocol / The original author is locked out.', + }, + { + id: 'mention_guard_and_cast_name', + label: '@copilot auto-assign / cast-name `name` rule present', + pass: text.includes('copilot-auto-assign: true/false') && text.includes('The `name` parameter MUST be the agent\'s lowercase cast name'), + detail: 'stable strings: copilot-auto-assign: true/false / lowercase cast name', + }, + ]; + for (const check of checks) { + check.applicable = isGoldenApplicable(check.id, text, options.ref); + if (!check.applicable) { + const marker = GOLDEN_CANARY_MARKERS[check.id]; + check.detail = `N/A: ${marker} not introduced at ref ${options.ref ?? '(unknown ref)'}`; + } + } + const applicableChecks = checks.filter(check => check.applicable); + const passed = applicableChecks.filter(check => check.pass).length; + return { + passed, + failed: applicableChecks.length - passed, + applicable: applicableChecks.length, + not_applicable: checks.length - applicableChecks.length, + total: checks.length, + checks, + }; +} + +function measureText(text, sourcePath = ARTIFACT_PATH, options = {}) { + return { + path: sourcePath, + bytes: bytesOf(text), + chars: [...text].length, + lines: countLines(text), + tokens: countTokens(text), + artifact_sha256: sha256(text), + goldens: evaluateGoldens(text, options), + }; +} + +function findSkillFiles(dir) { + const files = []; + const full = join(ROOT, dir); + if (!existsSync(full)) return files; + function walk(abs) { + for (const entry of readdirSync(abs, { withFileTypes: true })) { + const child = join(abs, entry.name); + if (entry.isDirectory()) walk(child); + else if (entry.isFile() && entry.name === 'SKILL.md') files.push(relative(ROOT, child)); + } + } + walk(full); + return files; +} + +function extractFrontmatter(text) { + if (!text.startsWith('---\n') && !text.startsWith('---\r\n')) return undefined; + const match = text.match(/^---\r?\n([\s\S]*?)\r?\n---\r?\n/); + return match ? match[0] : undefined; +} + +function measureAlwaysLoadedSkillFrontmatter() { + const skillFiles = [...findSkillFiles('.copilot/skills'), ...findSkillFiles('.squad/skills')]; + const alwaysOn = []; + for (const path of skillFiles) { + const text = readText(path); + const frontmatter = extractFrontmatter(text); + if (!frontmatter) continue; + if (/always[-_ ]?(on|apply|loaded)\s*:\s*true/i.test(frontmatter)) { + alwaysOn.push({ path, bytes: bytesOf(frontmatter) }); + } + } + if (alwaysOn.length === 0) { + return { + status: 'unavailable', + bytes: 0, + files: [], + note: 'No always-on skill frontmatter marker was determinable locally; excluded from D2 rather than silently assuming zero.', + }; + } + return { + status: 'measured', + bytes: alwaysOn.reduce((sum, file) => sum + file.bytes, 0), + files: alwaysOn, + }; +} + +function measureSquadStateToolSchema() { + const candidates = [ + 'packages/squad-cli/src/cli/commands/state-mcp.ts', + 'packages/squad-cli/dist/cli/commands/state-mcp.js', + ]; + const located = candidates.find(path => existsSync(join(ROOT, path))); + return { + status: 'unavailable', + bytes: 0, + located_source: located ?? null, + note: located + ? 'squad_state MCP schemas are assembled from the runtime ToolRegistry; static schema JSON was not locally extractable without importing project packages.' + : 'No squad_state MCP source/schema file was located.', + }; +} + +function computeD2(artifactBytes) { + const skillFrontmatter = measureAlwaysLoadedSkillFrontmatter(); + const squadStateToolSchema = measureSquadStateToolSchema(); + const measuredAdditions = [skillFrontmatter, squadStateToolSchema] + .filter(component => component.status === 'measured') + .reduce((sum, component) => sum + component.bytes, 0); + return { + local_estimate_only: true, + label: 'LOCAL-ESTIMATE-ONLY; does not subtract from any provider aggregate', + total_bytes: artifactBytes + measuredAdditions, + components: { + artifact: { status: 'measured', bytes: artifactBytes, path: ARTIFACT_PATH }, + always_loaded_skill_frontmatter: skillFrontmatter, + squad_state_tool_schema: squadStateToolSchema, + }, + }; +} + +function readBudget() { + const text = tryReadText(BUDGET_PATH); + if (!text) return { status: 'missing', path: BUDGET_PATH }; + try { + return { status: 'loaded', path: BUDGET_PATH, data: JSON.parse(text) }; + } catch (error) { + return { status: 'invalid', path: BUDGET_PATH, error: error instanceof Error ? error.message : String(error) }; + } +} + +function percentOfCurrent(delta, current) { + return Number.isFinite(delta) && Number.isFinite(current) && current > 0 + ? (delta / current) * 100 + : null; +} + +function sizeDelta(current, limit) { + if (!Number.isFinite(current) || !Number.isFinite(limit)) { + return { value: null, percent_of_current: null }; + } + return { + value: limit - current, + percent_of_current: percentOfCurrent(limit - current, current), + }; +} + +function reductionGap(current, target) { + if (!Number.isFinite(current) || !Number.isFinite(target)) { + return { value: null, percent_of_current: null }; + } + return { + value: Math.max(0, current - target), + percent_of_current: percentOfCurrent(Math.max(0, current - target), current), + }; +} + +function evaluateBudget(metrics) { + const budget = readBudget(); + if (budget.status !== 'loaded') return budget; + const artifactBytes = metrics.bytes; + const artifactTokens = metrics.tokens.count; + const regressionCeilingBytes = Number(budget.data.regression_ceiling_bytes); + const regressionCeilingTokens = Number(budget.data.regression_ceiling_o200k_tokens); + const reductionTargetBytes = Number(budget.data.reduction_target_bytes); + const reductionTargetTokens = Number(budget.data.reduction_target_o200k_tokens); + const grewBeyondRegressionCeiling = Number.isFinite(regressionCeilingBytes) && artifactBytes > regressionCeilingBytes; + const overrides = Array.isArray(budget.data.reviewed_overrides) ? budget.data.reviewed_overrides : []; + const coveringOverride = overrides.find(entry => { + const approvedBytes = Number(entry?.approved_bytes); + return Number.isFinite(approvedBytes) && approvedBytes >= artifactBytes && entry?.reviewer && entry?.reason; + }); + return { + status: 'loaded', + path: BUDGET_PATH, + mode: budget.data.mode, + regression_ceiling_bytes: regressionCeilingBytes, + regression_ceiling_o200k_tokens: regressionCeilingTokens, + reduction_target_bytes: reductionTargetBytes, + reduction_target_o200k_tokens: reductionTargetTokens, + regression_headroom: { + bytes: sizeDelta(artifactBytes, regressionCeilingBytes), + o200k_tokens: sizeDelta(artifactTokens, regressionCeilingTokens), + }, + reduction_gap: { + bytes: reductionGap(artifactBytes, reductionTargetBytes), + o200k_tokens: reductionGap(artifactTokens, reductionTargetTokens), + }, + grew_beyond_regression_ceiling: grewBeyondRegressionCeiling, + reviewed_override_covers_current_size: Boolean(coveringOverride), + covering_override: coveringOverride ?? null, + note: grewBeyondRegressionCeiling + ? (coveringOverride ? 'grew beyond regression ceiling; reviewed override covers current size' : 'grew beyond regression ceiling; no reviewed override covers current size') + : 'within report-only anti-regrowth ceiling; reduction target remains separate', + }; +} + +function git(args, options = {}) { + const result = spawnSync('git', args, { + cwd: ROOT, + encoding: options.encoding ?? 'utf8', + maxBuffer: options.maxBuffer ?? 32 * 1024 * 1024, + }); + if (result.status !== 0) { + throw new Error(`git ${args.join(' ')} failed: ${result.stderr || result.stdout}`); + } + return result.stdout; +} + +function gitShowText(ref, paths) { + for (const path of paths) { + const result = spawnSync('git', ['show', `${ref}:${path}`], { + cwd: ROOT, + encoding: 'utf8', + maxBuffer: 32 * 1024 * 1024, + }); + if (result.status === 0) return { text: result.stdout, path }; + } + return undefined; +} + +function findSlimAnchor() { + try { + const log = git(['log', '--all', '--format=%H%x09%s']); + const match = log.split('\n').find(line => /1308|slim squad\.agent/i.test(line)); + if (match) { + const [sha, subject] = match.split('\t'); + return { ref: sha, label: '#1308 slim commit', note: subject }; + } + } catch (error) { + // Fall through to earliest reachable anchor. + } + const allCommits = git(['log', '--all', '--reverse', '--format=%H']).trim().split('\n').filter(Boolean); + for (const sha of allCommits) { + if (gitShowText(sha, [ARTIFACT_PATH, TEMPLATE_PATH])) { + return { ref: sha, label: 'slim-era anchor fallback', note: 'Could not locate #1308; earliest reachable squad.agent.md used.' }; + } + } + return undefined; +} + +function backtestReport() { + const slim = findSlimAnchor(); + const refs = []; + if (slim) refs.push(slim); + else refs.push({ ref: null, label: '#1308 slim commit', note: 'unavailable: no reachable squad.agent.md history found' }); + refs.push({ ref: 'origin/dev', label: 'origin/dev', note: 'current mainline baseline' }); + refs.push({ ref: 'HEAD', label: 'HEAD', note: 'current branch working tree for HEAD artifact' }); + + const rows = refs.map(entry => { + if (entry.ref === null) return { ...entry, available: false }; + const content = entry.ref === 'HEAD' + ? { text: readText(ARTIFACT_PATH), path: ARTIFACT_PATH } + : gitShowText(entry.ref, [ARTIFACT_PATH, TEMPLATE_PATH]); + if (!content) return { ...entry, available: false, note: `${entry.note}; artifact/template unavailable at ref` }; + const metrics = measureText(content.text, content.path, { ref: entry.ref }); + const budgetMetrics = evaluateBudget(metrics); + return { + ...entry, + available: true, + source_path: content.path, + short_ref: entry.ref === 'HEAD' || entry.ref === 'origin/dev' ? entry.ref : entry.ref.slice(0, 12), + bytes: metrics.bytes, + chars: metrics.chars, + lines: metrics.lines, + tokens: metrics.tokens, + golden_passed: metrics.goldens.passed, + golden_failed: metrics.goldens.failed, + golden_applicable: metrics.goldens.applicable, + golden_not_applicable: metrics.goldens.not_applicable, + golden_total: metrics.goldens.total, + budget: budgetMetrics, + }; + }); + + return { mode: 'backtest', report_only: true, refs: rows }; +} + +function currentReport() { + const artifact = readText(ARTIFACT_PATH); + const template = tryReadText(TEMPLATE_PATH); + const d1 = measureText(artifact, ARTIFACT_PATH, { ref: 'HEAD' }); + return { + mode: 'current', + report_only: true, + generated_at: new Date().toISOString(), + d1_artifact_ceiling: d1, + d2_net_resident_startup_estimate: computeD2(d1.bytes), + d3_governance_golden_regression: d1.goldens, + d4_reviewed_override: evaluateBudget(d1), + artifact_sha256: d1.artifact_sha256, + template_sha256: template ? sha256(template) : null, + template_path: TEMPLATE_PATH, + }; +} + +function pad(value, width) { + const string = String(value); + return string.length >= width ? string : string + ' '.repeat(width - string.length); +} + +function tokenText(tokens) { + return `${tokens.count}${tokens.estimate ? ' est' : ''}`; +} + +function formatInteger(value) { + return Number.isFinite(value) ? Math.round(value).toLocaleString('en-US') : 'n/a'; +} + +function formatPercent(value) { + return Number.isFinite(value) ? `${value.toFixed(2)}%` : 'n/a'; +} + +function formatBudgetDelta(label, delta) { + if (!delta) return 'n/a'; + const bytes = delta.bytes; + const tokens = delta.o200k_tokens; + return `${label}: ${formatInteger(bytes.value)} B / ${formatInteger(tokens.value)} tok (${formatPercent(bytes.percent_of_current)} B / ${formatPercent(tokens.percent_of_current)} tok)`; +} + +function renderBacktest(report) { + const header = ['Ref', 'Bytes', 'Chars', 'Lines', 'Tokens', 'Goldens', 'Ceiling headroom', 'Reduction gap', 'Source']; + const rows = report.refs.map(row => row.available + ? [ + row.label, + row.bytes, + row.chars, + row.lines, + tokenText(row.tokens), + `${row.golden_passed}/${row.golden_applicable} applicable pass + ${row.golden_not_applicable} N/A`, + formatBudgetDelta('headroom', row.budget?.regression_headroom), + formatBudgetDelta('gap', row.budget?.reduction_gap), + row.source_path, + ] + : [row.label, 'n/a', 'n/a', 'n/a', 'n/a', 'n/a', 'n/a', 'n/a', row.note ?? 'unavailable']); + const widths = header.map((title, index) => Math.max(title.length, ...rows.map(row => String(row[index]).length))); + const lines = [ + 'Size Regression Guard Backtest (REPORT-ONLY)', + '', + header.map((cell, index) => pad(cell, widths[index])).join(' | '), + widths.map(width => '-'.repeat(width)).join('-|-'), + ...rows.map(row => row.map((cell, index) => pad(cell, widths[index])).join(' | ')), + '', + 'Note: goldens introduced after a historical ref are N/A; FAIL is reserved for applicable regressions.', + ]; + return lines.join('\n'); +} + +function renderCurrent(report) { + const d1 = report.d1_artifact_ceiling; + const tokenSuffix = d1.tokens.estimate ? ' (estimate; python3+tiktoken unavailable)' : ' (o200k_base)'; + const d2 = report.d2_net_resident_startup_estimate; + const d4 = report.d4_reviewed_override; + const regressionHeadroom = formatBudgetDelta('regression headroom', d4.regression_headroom); + const reductionGap = formatBudgetDelta('reduction gap', d4.reduction_gap); + return [ + 'Size Regression Guard (REPORT-ONLY)', + '', + `D1 artifact: ${d1.bytes} bytes, ${d1.chars} chars, ${d1.lines} lines, ${d1.tokens.count} tokens${tokenSuffix}`, + `D3 goldens: ${d1.goldens.passed}/${d1.goldens.applicable} applicable pass (${d1.goldens.failed} fail, ${d1.goldens.not_applicable} N/A)`, + `D2 local estimate only: ${d2.total_bytes} bytes; skill frontmatter=${d2.components.always_loaded_skill_frontmatter.status}; squad_state schema=${d2.components.squad_state_tool_schema.status}`, + `D4 budget: ${d4.status}${d4.status === 'loaded' ? `; regression_ceiling_bytes=${d4.regression_ceiling_bytes}; regression_ceiling_o200k_tokens=${d4.regression_ceiling_o200k_tokens}; ${d4.note}` : ''}`, + `D4 ${regressionHeadroom}`, + `D4 ${reductionGap} beyond provisional target`, + `artifact_sha256: ${report.artifact_sha256}`, + `template_sha256: ${report.template_sha256 ?? 'unavailable'}`, + '', + 'Governance goldens:', + ...d1.goldens.checks.map(check => `- ${check.applicable ? (check.pass ? 'PASS' : 'FAIL') : 'N/A'} ${check.id}: ${check.detail}`), + '', + `Machine-readable JSON written to ${REPORT_PATH}`, + ].join('\n'); +} + +const report = backtest ? backtestReport() : currentReport(); +if (!backtest) { + writeFileSync(join(ROOT, REPORT_PATH), `${JSON.stringify(report, null, 2)}\n`); +} + +if (jsonOnly) { + process.stdout.write(`${JSON.stringify(report, null, 2)}\n`); +} else { + process.stdout.write(`${backtest ? renderBacktest(report) : renderCurrent(report)}\n`); +} + +process.exit(0); diff --git a/templates/copilot-instructions.md b/templates/copilot-instructions.md index 2ba998c8e..e5bab9f71 100644 --- a/templates/copilot-instructions.md +++ b/templates/copilot-instructions.md @@ -6,13 +6,17 @@ You are working on a project that uses **Squad**, an AI team framework. When pic **IMPORTANT — run this check on EVERY session start, before doing any work:** -Look for the exact token `SQUAD_COORDINATOR_CANARY_a8f3` in your `agent_instructions`. This token is placed at the end of `squad.agent.md`. If it is missing, the coordinator file is missing or truncated. +Inspect your `agent_instructions` for the coordinator canaries. This is a **two-state payload-integrity check within a known Squad coordinator session**; it verifies whether the coordinator payload loaded fully or was visibly truncated. It is **not** proof that an agent with no canaries is safely non-Squad — wholly absent payloads and proven non-Squad agents require a host-level identity beacon before custom-agent instruction ingestion. -**When the token is missing, you MUST:** +1. If the HEAD canary `SQUAD_COORDINATOR_CANARY_HEAD_b7d2` is **absent**, the Squad coordinator payload is not observable from inside this prompt. Treat non-Squad discrimination as an **OPEN host-level concern**; do not claim this check proved the current agent is non-Squad. +2. If the HEAD canary `SQUAD_COORDINATOR_CANARY_HEAD_b7d2` is **present** and the EOF canary `SQUAD_COORDINATOR_CANARY_a8f3` is **present**, the coordinator prompt loaded fully. Proceed with normal Squad behavior. +3. If the HEAD canary `SQUAD_COORDINATOR_CANARY_HEAD_b7d2` is **present** but the EOF canary `SQUAD_COORDINATOR_CANARY_a8f3` is **absent**, the coordinator prompt loaded but was **truncated** before the end. + +**When the coordinator prompt is truncated, you MUST:** 1. **STOP** — do not proceed with standard Squad behavior. 2. **WARN the user** with this exact message: ``` - ⚠️ Squad coordinator (squad.agent.md) appears to be missing or truncated. The canary token was not found. Do NOT proceed with standard Squad behavior — Squad's safety rails are not loaded. Please restart your session. + ⚠️ Squad coordinator (squad.agent.md) loaded but appears to be truncated before the EOF canary. Do NOT proceed with standard Squad behavior — Squad's safety rails are incomplete. Please restart your session. ``` 3. Do not continue with normal Squad routing, spawning, PR, or branch-protection behavior after emitting the warning. diff --git a/templates/squad.agent.md.template b/templates/squad.agent.md.template index eb545668f..2c9b59932 100644 --- a/templates/squad.agent.md.template +++ b/templates/squad.agent.md.template @@ -4,6 +4,8 @@ description: "Your AI team. Describe what you're building, get a team of special tools: ["*"] --- + + You are **Squad (Coordinator)** — the orchestrator for this project's AI team.