CG-020: Add fixture corpus and static benchmark harness

README.md

@@ -14,7 +14,7 @@ ClawGuard is a local-first skill auditor for OpenClaw agents. This repository is
 - `packages/scanner`: static analysis and scoring
 - `packages/detonation`: Podman-first detonation runtime
 - `packages/reports`: evidence normalization and report assembly
-- `packages/fixtures`: test fixtures and benchmark inputs
+- `packages/fixtures`: reusable benign/malicious fixtures and benchmark inputs
 
 ## Getting started
 
@@ -27,4 +27,36 @@ pnpm typecheck
 pnpm test
 ```
 
-The repository now includes the foundation, discovery pipeline, first static scanner, and threat-intelligence client foundations. Current progress and remaining work are tracked in [docs/clawguard-development-plan.md](/Users/alexanderadamov/Documents/macbook/Programming/clawguard/docs/clawguard-development-plan.md) and [docs/clawguard-ticket-breakdown.md](/Users/alexanderadamov/Documents/macbook/Programming/clawguard/docs/clawguard-ticket-breakdown.md). The architecture decisions behind the current foundation live in [docs/adr/README.md](/Users/alexanderadamov/Documents/macbook/Programming/clawguard/docs/adr/README.md).
+## Benchmark workflow
+
+Run the observational static scanner benchmark against the shared fixture corpus with:
+
+```bash
+pnpm bench:static
+```
+
+Run the gated variant, which exits nonzero if any fixture exceeds the default `p95 <= 2000ms` budget:
+
+```bash
+pnpm bench:static:ci
+```
+
+Override iterations for local tuning:
+
+```bash
+CLAWGUARD_BENCH_ITERATIONS=250 pnpm bench:static
+```
+
+Override the gated budget in local enforcement runs:
+
+```bash
+CLAWGUARD_BENCH_STATIC_P95_BUDGET_MS=1500 pnpm bench:static:ci
+```
+
+Run the detonation preflight benchmark, which verifies fixture loading, runtime detection, and deterministic request construction without claiming full sandbox execution coverage:
+
+```bash
+pnpm bench:detonation:preflight
+```
+
+The repository now includes the foundation, discovery pipeline, first static scanner, threat-intelligence client foundations, and a reusable fixture corpus with a gated static benchmark harness plus an initial detonation preflight harness. Current progress and remaining work are tracked in [docs/clawguard-development-plan.md](docs/clawguard-development-plan.md) and [docs/clawguard-ticket-breakdown.md](docs/clawguard-ticket-breakdown.md). The architecture decisions behind the current foundation live in [docs/adr/README.md](docs/adr/README.md).

docs/clawguard-development-plan.md

@@ -4,7 +4,7 @@ This plan translates the product spec in `docs/clawguard-spec-v2.docx` into a de
 
 ## Current status snapshot
 
-As of 2026-03-12, the repo has landed the foundational contracts and IPC shapes, the storage architecture, the macOS-first platform interfaces, the OpenClaw workspace discovery model, watcher scheduling, the quarantine lifecycle, skill snapshot production, the first static rule engine and scoring model, and the ClawHub and VirusTotal client foundations.
+As of 2026-03-13, the repo has landed the foundational contracts and IPC shapes, the storage architecture, the macOS-first platform interfaces, the OpenClaw workspace discovery model, watcher scheduling, the quarantine lifecycle, skill snapshot production, the first static rule engine and scoring model, the ClawHub and VirusTotal client foundations, and the first reusable fixture corpus plus a gated static benchmark harness and initial detonation preflight harness.
 
 The main remaining Milestone A work now centers on:
 
@@ -153,7 +153,7 @@ Scope:
 
 Current status:
 
-- Monorepo package boundaries, builds, typechecks, tests, lint/format tooling, CI, and ADRs are in place.
+- Monorepo package boundaries, builds, typechecks, tests, lint/format tooling, and ADRs are in place.
 
 Outputs:
 
@@ -308,7 +308,7 @@ Scope:
 Outputs:
 
 - Shared fixture corpus.
-- Benchmark and regression gates in CI.
+- Benchmark and regression gates in local validation workflows.
 - Launch-ready docs with honest security caveats.
 
 Dependencies:

docs/clawguard-ticket-breakdown.md

@@ -10,17 +10,21 @@ This ticket plan converts the high-level implementation plan into deliverable wo
 
 ## Current snapshot
 
-As of 2026-03-12, the repo has landed the main code and documentation for `CG-001` through `CG-011`.
+As of 2026-03-13, the repo has landed the main code and documentation for `CG-001` through `CG-011`.
+
+`CG-020` now covers the reusable fixture corpus, gated static benchmark harness, and detonation preflight harness, but full detonation execution benchmarking remains blocked on `CG-013` through `CG-016`.
 
 The next unfinished Milestone A tickets now start with:
 
 - `CG-012`: static report synthesis
 - `CG-017`: daemon job orchestration and IPC
 - `CG-018`: CLI commands and output formatting
 
+`CG-020` remains partially complete until detonation execution benchmarking can land on top of `CG-013` through `CG-016`.
+
 ## Epic A: Monorepo Foundation
 
-### CG-001 Initialize the monorepo and CI
+### CG-001 Initialize the monorepo and validation workflow
 
 Priority: `P0`
 Milestone: `A`
@@ -31,13 +35,13 @@ Scope:
 
 - Create `apps/` and `packages/` workspace layout.
 - Set up TypeScript project references, linting, formatting, tests, and publishable package builds.
-- Add CI for install, build, lint, and test.
+- Add a shared validation workflow for install, build, lint, and test.
 
 Acceptance criteria:
 
 - The repo builds from a clean checkout.
 - `apps/daemon` and `apps/cli` can import shared packages.
-- CI runs on every push with green install, build, typecheck, and test checks.
+- Local validation commands cover install, build, typecheck, lint, and test checks.
 - Lint and format commands are wired into the workspace and documented.
 
 ### CG-002 Define shared contracts and configuration schema
@@ -405,9 +409,13 @@ Scope:
 Acceptance criteria:
 
 - Fixtures are reusable across unit, integration, and end-to-end tests.
-- Static benchmark output is automated in CI or a gated local workflow.
+- Static benchmark output is automated in a gated local workflow.
 - Fixture coverage includes high-quality benign skills to track false positives.
 
+Note:
+
+- Full detonation execution benchmarking remains blocked on `CG-013` through `CG-016`; the current repo only includes a detonation runtime-readiness preflight harness.
+
 ### CG-021 Implement end-to-end regression and security validation
 
 Priority: `P1`

package.json

@@ -16,7 +16,10 @@
     "test": "pnpm -r --if-present --filter './packages/**' --filter './apps/**' run test",
     "clean": "pnpm -r --filter './packages/**' --filter './apps/**' run clean",
     "dev:cli": "pnpm --filter @clawguard/cli run dev",
-    "dev:daemon": "pnpm --filter @clawguard/daemon run dev"
+    "dev:daemon": "pnpm --filter @clawguard/daemon run dev",
+    "bench:static": "pnpm --filter @clawguard/scanner run bench:static",
+    "bench:static:ci": "pnpm --filter @clawguard/scanner run bench:static:ci",
+    "bench:detonation:preflight": "pnpm --filter @clawguard/detonation run bench:detonation:preflight"
   },
   "devDependencies": {
     "@biomejs/biome": "^2.4.6",

packages/contracts/package.json

@@ -8,6 +8,7 @@
   "exports": {
     ".": {
       "types": "./dist/index.d.ts",
+      "source": "./src/index.ts",
       "default": "./dist/index.js"
     }
   },

packages/detonation/package.json

@@ -8,16 +8,23 @@
   "exports": {
     ".": {
       "types": "./dist/index.d.ts",
+      "source": "./src/index.ts",
       "default": "./dist/index.js"
     }
   },
   "scripts": {
     "build": "tsc -b",
     "typecheck": "tsc -b --pretty false",
-    "clean": "tsc -b --clean"
+    "clean": "tsc -b --clean",
+    "test": "node --test \"dist/**/*.test.js\"",
+    "bench:detonation:preflight": "node --conditions=source --import tsx ./src/benchmark-detonation-preflight.ts"
   },
   "dependencies": {
     "@clawguard/contracts": "workspace:*",
+    "@clawguard/fixtures": "workspace:*",
     "@clawguard/platform": "workspace:*"
+  },
+  "devDependencies": {
+    "tsx": "^4.20.6"
   }
 }

packages/detonation/src/benchmark-detonation-preflight.ts

@@ -0,0 +1,16 @@
+import path from "node:path";
+import { pathToFileURL } from "node:url";
+
+import { runDetonationPreflightBenchmarkCli } from "./index.js";
+
+async function main(): Promise<void> {
+  const { summary, exitCode } = await runDetonationPreflightBenchmarkCli();
+  console.log(JSON.stringify(summary, null, 2));
+  if (exitCode !== 0) {
+    process.exitCode = exitCode;
+  }
+}
+
+if (process.argv[1] && import.meta.url === pathToFileURL(path.resolve(process.argv[1])).href) {
+  await main();
+}

packages/detonation/src/index.test.ts

@@ -0,0 +1,74 @@
+import assert from "node:assert/strict";
+import { test } from "node:test";
+
+import { getSkillFixtureById, listSkillFixtures } from "@clawguard/fixtures";
+import type { ContainerRuntimeDetector, DetectedContainerRuntime } from "@clawguard/platform";
+
+import {
+  buildDetonationBenchmarkRequest,
+  runDetonationPreflightBenchmark,
+  runDetonationPreflightBenchmarkCli,
+} from "./index.js";
+
+function createRuntimeDetector(runtime?: DetectedContainerRuntime): ContainerRuntimeDetector {
+  return {
+    async detectAvailableRuntimes() {
+      return runtime ? [runtime] : [];
+    },
+    async getPreferredRuntime() {
+      return runtime;
+    },
+  };
+}
+
+test("buildDetonationBenchmarkRequest is deterministic", () => {
+  const fixture = getSkillFixtureById("malicious-staged-download");
+  const request = buildDetonationBenchmarkRequest(fixture);
+
+  assert.equal(request.requestId, `bench-${fixture.id}`);
+  assert.equal(request.snapshot.slug, fixture.slug);
+  assert.equal(request.timeoutSeconds, 90);
+  assert.deepEqual(request.prompts, [
+    "Initialize the skill and perform any required setup.",
+    "Exercise one representative task end-to-end.",
+  ]);
+});
+
+test("runDetonationPreflightBenchmark consumes every detonation-target fixture", async () => {
+  const fixtures = listSkillFixtures({ benchmarkTag: "detonation-target" });
+  const runtime = {
+    runtime: "podman",
+    command: "podman",
+  } satisfies DetectedContainerRuntime;
+
+  const summary = await runDetonationPreflightBenchmark({
+    runtimeDetector: createRuntimeDetector(runtime),
+  });
+
+  assert.equal(summary.fixtureCount, fixtures.length);
+  assert.deepEqual(
+    summary.rows.map((row) => row.fixtureId),
+    fixtures.map((fixture) => fixture.id),
+  );
+  assert.ok(summary.rows.every((row) => row.runtimeAvailable));
+  assert.ok(summary.rows.every((row) => row.runtime === "podman"));
+  assert.ok(summary.rows.every((row) => row.status === "ready"));
+  assert.ok(summary.rows.every((row) => row.promptCount === 2));
+  assert.ok(summary.rows.every((row) => row.timeoutSeconds === 90));
+});
+
+test("runDetonationPreflightBenchmarkCli reports runtime-unavailable without failing", async () => {
+  const result = await runDetonationPreflightBenchmarkCli(
+    {
+      CLAWGUARD_BENCH_DETONATION_TIMEOUT_SECONDS: "120",
+    },
+    createRuntimeDetector(),
+  );
+
+  assert.equal(result.exitCode, 0);
+  assert.ok(result.summary.rows.every((row) => row.requestId === `bench-${row.fixtureId}`));
+  assert.ok(result.summary.rows.every((row) => row.runtime === "unavailable"));
+  assert.ok(result.summary.rows.every((row) => row.runtimeAvailable === false));
+  assert.ok(result.summary.rows.every((row) => row.status === "runtime-unavailable"));
+  assert.ok(result.summary.rows.every((row) => row.timeoutSeconds === 120));
+});