fix(findings): tie a suggestion to the span it replaces

evals/cassettes/ts-0045-kernel-bypass-hallucination/0001-5d5f08dc06f14946.json

@@ -4,13 +4,13 @@
   "model_answered": "deepseek/deepseek-v4-pro-0813",
   "schema_name": "tinysweeper_rule_extraction",
   "usage": {
-    "input_tokens": 1632,
-    "output_tokens": 3660,
+    "input_tokens": 3106,
+    "output_tokens": 5561,
     "cached_tokens": 1536,
     "embed_tokens": 0,
-    "cost_usd": 0.003231528
+    "cost_usd": 0.003111033
   },
   "value": {
-    "rules_markdown": "- Organize `src/` as single crate: one responsibility per module, core types in module-local `types.rs`, one trait per file in `src/ports/`.\n- Declare every CLI subcommand in `src/bin/tinysweeper.rs` even if milestone not landed; `src/server/` is the only production surface.\n- Store review policy as data in `presets/`: each preset is a folder with `preset.toml`, `README.md`, optional prompt overrides; never add a module for a preset.\n- Treat `vendor/tinyagents` as a git submodule: never edit it here; change upstream and bump the pin.\n- Keep one document per `src/` module at `docs/modules/<module>/README.md`.\n- Declare examples in `Cargo.toml` with `required-features` so credential-needing smoke tests never build in CI.\n- Default build is offline; link no HTTP client, test suite never touches network; network features must be gated (`harness`, `github`, `serve`).\n- Follow rustfmt output and Rust 2024 idioms; use `snake_case` for modules/files, `PascalCase` for types.\n- Return `Result<T>` using crate error type from `src/error.rs`.\n- Start every file with a `//!` module doc describing role and any feature gating.\n- Write comments that explain decisions, not code; call out load-bearing flags/ordering and why.\n- Add doc comments to public items; use `///` on Clap fields so it becomes help text.\n- Keep tests in-crate: `#[cfg(test)] mod tests` at bottom, move to sibling `test.rs` or `<name>_test.rs` when large; no `tests/` directory.\n- Provide always-compiled offline mock for every port; `MockForge` records intended writes for exact assertions.\n- Cover lane behaviour with golden tests: fixture diff + canned structured model response, asserting findings after filtering, dedupe, capping.\n- Maintain at least 80% coverage for meaningful library behaviour.\n- Keep every Markdown file, including guidelines, at 500 lines or fewer; split and link from module README.\n- Security: model never holds write token; write credentials minted only in `src/apply/` after all model calls return; leaving a change fails.\n- Security: never execute contributor code; read diff/tree only, no build, install, or running target repo scripts.\n- Security: treat PR bodies/comments/diffs as untrusted input, fence and label as data; model verdict advisory; only `src/apply/` and `src/automerge/` may mutate GitHub.\n- Security: report secrets by type and location only; never include value in comments, check-run summaries, or logs.\n- Use concise imperative commit subjects; keep commits small, one coherent validated slice each.\n- Pull requests state what changed, any behaviour change, and how it was verified."
+    "rules_markdown": "- src/ is the single crate; one responsibility per module, core types in module-local types.rs, and every port in src/ports/ is one trait in one file.\n- src/bin/tinysweeper.rs is the CLI; every subcommand is declared even if its milestone has not landed.\n- src/server/ is the only production surface; there is no GitHub Actions distribution path.\n- presets/ are review policy as data, not code; adding a preset is a new folder, never a new module.\n- vendor/tinyagents is a git submodule; never edit it here, change upstream and bump the pin.\n- docs/modules/<module>/README.md is one document per src/ module.\n- examples/ are declared explicitly in Cargo.toml with required-features so credential-needing smoke tests never build in CI.\n- The default build is offline; anything that needs the network goes behind a feature (harness, github, serve).\n- Follow rustfmt output and Rust 2024 idioms; use snake_case for modules/files and PascalCase for types.\n- Return Result<T> using the crate error type from src/error.rs.\n- Every file opens with a //! module doc describing its role and any feature gating.\n- Comments explain the decision, not the code; say why when a flag or ordering is load-bearing.\n- Public items carry doc comments; Clap fields use /// which becomes the help text.\n- Tests live in-crate: a #[cfg(test)] mod tests block at module bottom, moving to sibling test.rs or <name>_test.rs when they grow; no tests/ directory.\n- Every port has an always-compiled offline mock; MockForge records what would have been written so tests can assert on exact check runs and comments.\n- Lane behaviour is covered by golden tests: fixture diff plus a canned structured model response, asserting findings that survive filtering, dedupe, and capping.\n- Maintain at least 80% coverage for meaningful library behaviour.\n- Keep every Markdown file at 500 lines or fewer; split into focused files and link from module README when over.\n- The model never holds a write token; lanes run against a read-only checkout; write credentials minted only in src/apply/ after all model calls returned; a lane leaving a change in checkout fails.\n- Contributor code is never executed; read the diff and tree, do not build, install dependencies, or run target repository's scripts.\n- Pull request bodies, comments, and diffs are untrusted input; fence and label them as data in prompts; model verdict advisory; only deterministic policy in src/apply/ and src/automerge/ may mutate GitHub.\n- Secrets found by scanners are reported by type and location only; the value never reaches a comment, check-run summary, or log.\n- Use concise, imperative commit subjects; keep commits small; commit each coherent, validated slice on its own.\n- Pull requests state what changed, any behaviour change, and how it was verified."
   }
 }

evals/cassettes/ts-0045-kernel-bypass-hallucination/0002-17541401c04d839b.json

@@ -0,0 +1,17 @@
+{
+  "key": "17541401c04d839b",
+  "model_requested": "deepseek/deepseek-v4-pro-0813",
+  "model_answered": "deepseek/deepseek-v4-pro-0813",
+  "schema_name": "tinysweeper_critique",
+  "usage": {
+    "input_tokens": 2845,
+    "output_tokens": 552,
+    "cached_tokens": 0,
+    "embed_tokens": 0,
+    "cost_usd": 0.001717815
+  },
+  "value": {
+    "findings": [],
+    "summary": "The change adds a new public module `summarise` to `src/findings/mod.rs`. The declaration follows the repository's snake_case naming convention and does not introduce any apparent issues."
+  }
+}

evals/cassettes/ts-0045-kernel-bypass-hallucination/0003-1c93c25d9aadd0a7.json

@@ -0,0 +1,29 @@
+{
+  "key": "1c93c25d9aadd0a7",
+  "model_requested": "deepseek/deepseek-v4-pro-0813",
+  "model_answered": "deepseek/deepseek-v4-pro-0813",
+  "schema_name": "tinysweeper_critique",
+  "usage": {
+    "input_tokens": 2964,
+    "output_tokens": 2819,
+    "cached_tokens": 384,
+    "embed_tokens": 0,
+    "cost_usd": 0.003576222
+  },
+  "value": {
+    "findings": [
+      {
+        "body": "The diff turns on embeddings retrieval via an external provider (`provider = \"openrouter\"`) without any feature gating. The repository rule says the default build is offline and anything that needs the network goes behind a feature (harness, github, serve). Enabling this in `.tinysweeper.toml` makes network calls part of the default configuration even when the relevant feature is off, breaking offline use. Either set `enabled = false` here or ensure the code only reads this section when a network feature is active.",
+        "confidence": 0.5,
+        "existing_code": "enabled = true",
+        "late": false,
+        "path": ".tinysweeper.toml",
+        "rule": "network-off-by-default",
+        "severity": "medium",
+        "suggestion": null,
+        "title": "Disable embeddings retrieval by default or gate it behind a feature"
+      }
+    ],
+    "summary": "The change adds embeddings configuration; as written it enables network retrieval by default, which appears to violate the offline-default policy. I'd resolve that before merging."
+  }
+}

evals/cassettes/ts-0045-kernel-bypass-hallucination/0004-3df0bb3a4c9249d9.json

@@ -0,0 +1,16 @@
+{
+  "key": "3df0bb3a4c9249d9",
+  "model_requested": "deepseek/deepseek-v4-pro-0813",
+  "model_answered": "deepseek/deepseek-v4-pro-0813",
+  "schema_name": "tinysweeper_falsify",
+  "usage": {
+    "input_tokens": 1013,
+    "output_tokens": 176,
+    "cached_tokens": 0,
+    "embed_tokens": 0,
+    "cost_usd": 0.000593775
+  },
+  "value": {
+    "incorrect": []
+  }
+}

evals/cassettes/ts-0045-kernel-bypass-hallucination/0005-183c4d7693b78647.json

@@ -0,0 +1,48 @@
+{
+  "key": "183c4d7693b78647",
+  "model_requested": "deepseek/deepseek-v4-pro-0813",
+  "model_answered": "deepseek/deepseek-v4-pro-0813",
+  "schema_name": "tinysweeper_critique",
+  "usage": {
+    "input_tokens": 3187,
+    "output_tokens": 4694,
+    "cached_tokens": 0,
+    "embed_tokens": 0,
+    "cost_usd": 0.005470125
+  },
+  "value": {
+    "findings": [
+      {
+        "body": "The MongoDB community-server image (especially the UBI9 variant) typically does not include Python 3. Overriding the entrypoint to invoke `python3` will cause the container to fail to start with 'exec: python3: not found' if the interpreter is absent. Use a shell-based entrypoint or install Python before running the script.",
+        "confidence": 0.8,
+        "existing_code": "      [\"python3\", \"/usr/local/bin/kernel-bypass-entrypoint.py\", \"mongod\"]",
+        "path": "docker-compose.kernel-bypass.yml",
+        "rule": "missing-runtime-dependency",
+        "severity": "high",
+        "suggestion": null,
+        "title": "MongoDB container image may lack python3"
+      },
+      {
+        "body": "The mongo-secrets service inherits the original image entrypoint, which the file's comments say hard-refuses to start `mongod` on affected kernels. Setting `GLIBC_TUNABLES` alone may not bypass that check because the check is separate from the glibc tunable. If the base service uses the same entrypoint, it will fail to start. Override its entrypoint as well or confirm it does not run the kernel check.",
+        "confidence": 0.7,
+        "existing_code": "  mongo-secrets:\n    environment:\n      GLIBC_TUNABLES: glibc.pthread.rseq=0",
+        "path": "docker-compose.kernel-bypass.yml",
+        "rule": "inconsistent-kernel-bypass",
+        "severity": "high",
+        "suggestion": null,
+        "title": "mongo-secrets still uses the refusing entrypoint"
+      },
+      {
+        "body": "The MongoDB image's original entrypoint performs important setup (e.g., file permissions, keyfile generation, user/group setup) before executing `mongod`. Replacing it entirely with a custom Python script risks skipping those steps, leading to startup failures or security misconfigurations. Verify that the script replicates necessary initialization or use a wrapper that calls the original entrypoint with a flag to bypass only the kernel check.",
+        "confidence": 0.3,
+        "existing_code": "  mongod:\n    entrypoint:\n      [\"python3\", \"/usr/local/bin/kernel-bypass-entrypoint.py\", \"mongod\"]",
+        "path": "docker-compose.kernel-bypass.yml",
+        "rule": "entrypoint-replacement",
+        "severity": "medium",
+        "suggestion": null,
+        "title": "Overriding entrypoint drops original setup steps"
+      }
+    ],
+    "summary": "The overlay aims to bypass MongoDB's kernel restriction, but it relies on Python being present in the image and leaves `mongo-secrets` potentially failing, while discarding the original entrypoint's setup logic. These issues need to be addressed before merging."
+  }
+}

evals/cassettes/ts-0045-kernel-bypass-hallucination/0006-86efd049e0b944eb.json

@@ -0,0 +1,28 @@
+{
+  "key": "86efd049e0b944eb",
+  "model_requested": "deepseek/deepseek-v4-pro-0813",
+  "model_answered": "deepseek/deepseek-v4-pro-0813",
+  "schema_name": "tinysweeper_critique",
+  "usage": {
+    "input_tokens": 3663,
+    "output_tokens": 5259,
+    "cached_tokens": 0,
+    "embed_tokens": 0,
+    "cost_usd": 0.006168735
+  },
+  "value": {
+    "findings": [
+      {
+        "body": "When `rfind(' ')` returns `None`, the code appends an ellipsis to the full `limit`-character `head`, producing a result of `limit + 1` characters. This violates the documented contract (\"at most `limit` characters\") and may exceed downstream limits (e.g., GitHub check-run titles). The fix removes one character from `head` before appending the ellipsis, except when `limit` is 0, where an empty string should be returned.",
+        "confidence": 0.95,
+        "existing_code": "    let head: String = text.chars().take(limit).collect();\n    match head.rfind(' ') {\n        Some(space) => format!(\"{}…\", &head[..space]),\n        None => format!(\"{head}…\"),\n    }",
+        "path": "src/findings/summarise.rs",
+        "rule": "off-by-one",
+        "severity": "medium",
+        "suggestion": "    let head: String = text.chars().take(limit).collect();\n    match head.rfind(' ') {\n        Some(space) => format!(\"{}…\", &head[..space]),\n        None => {\n            let mut head = head;\n            if head.is_empty() {\n                String::new()\n            } else {\n                head.pop();\n                format!(\"{head}…\")\n            }\n        }\n    }",
+        "title": "Respect the character limit when no space is found"
+      }
+    ],
+    "summary": "The change adds `shorten` and `first_sentence` utilities with tests, but `shorten` can return a string one character longer than the requested limit when the truncated prefix contains no space."
+  }
+}

evals/cassettes/ts-0045-kernel-bypass-hallucination/0007-abbb7a6b9db04578.json

@@ -0,0 +1,17 @@
+{
+  "key": "abbb7a6b9db04578",
+  "model_requested": "deepseek/deepseek-v4-pro-0813",
+  "model_answered": "deepseek/deepseek-v4-pro-0813",
+  "schema_name": "tinysweeper_critique",
+  "usage": {
+    "input_tokens": 3420,
+    "output_tokens": 5319,
+    "cached_tokens": 0,
+    "embed_tokens": 0,
+    "cost_usd": 0.00611523
+  },
+  "value": {
+    "findings": [],
+    "summary": "The change renames GitHub App environment variables to match what the binary reads, adds API key forwarding, and mounts a read-only config file. No issues found."
+  }
+}

evals/cassettes/ts-0045-kernel-bypass-hallucination/0008-87d1384e18c9d053.json

@@ -0,0 +1,16 @@
+{
+  "key": "87d1384e18c9d053",
+  "model_requested": "deepseek/deepseek-v4-pro-0813",
+  "model_answered": "deepseek/deepseek-v4-pro-0813",
+  "schema_name": "tinysweeper_falsify",
+  "usage": {
+    "input_tokens": 1417,
+    "output_tokens": 456,
+    "cached_tokens": 384,
+    "embed_tokens": 0,
+    "cost_usd": 0.000847467
+  },
+  "value": {
+    "incorrect": []
+  }
+}

evals/cassettes/ts-0045-kernel-bypass-hallucination/0009-2d53b24abad09d41.json

@@ -0,0 +1,16 @@
+{
+  "key": "2d53b24abad09d41",
+  "model_requested": "deepseek/deepseek-v4-pro-0813",
+  "model_answered": "deepseek/deepseek-v4-pro-0813",
+  "schema_name": "tinysweeper_falsify",
+  "usage": {
+    "input_tokens": 1703,
+    "output_tokens": 1110,
+    "cached_tokens": 640,
+    "embed_tokens": 0,
+    "cost_usd": 0.001430425
+  },
+  "value": {
+    "incorrect": []
+  }
+}

evals/cassettes/ts-0045-kernel-bypass-hallucination/0010-d6bf343291f4be03.json

@@ -0,0 +1,28 @@
+{
+  "key": "d6bf343291f4be03",
+  "model_requested": "deepseek/deepseek-v4-pro-0813",
+  "model_answered": "deepseek/deepseek-v4-pro-0813",
+  "schema_name": "tinysweeper_critique",
+  "usage": {
+    "input_tokens": 3664,
+    "output_tokens": 6495,
+    "cached_tokens": 384,
+    "embed_tokens": 0,
+    "cost_usd": 0.007078842
+  },
+  "value": {
+    "findings": [
+      {
+        "body": "The file handle returned by `open()` is never closed. While the process will eventually close it on exit, the handle remains open during the entire execution of the patched entrypoint, which may run indefinitely if the entrypoint starts a long-lived process. Use a context manager to ensure it is closed immediately after reading.",
+        "confidence": 0.8,
+        "existing_code": "    source = open(ENTRYPOINT, encoding=\"utf-8\").read()",
+        "path": "deploy/mongo/kernel-bypass-entrypoint.py",
+        "rule": "resource-leak",
+        "severity": "low",
+        "suggestion": "    with open(ENTRYPOINT, encoding=\"utf-8\") as f:\n        source = f.read()",
+        "title": "Close the entrypoint file after reading"
+      }
+    ],
+    "summary": "The change adds a shim to bypass the MongoDB kernel compatibility check; it is generally sound, with one minor resource leak."
+  }
+}

evals/cassettes/ts-0045-kernel-bypass-hallucination/0011-7c0e6d892204ca20.json

@@ -0,0 +1,16 @@
+{
+  "key": "7c0e6d892204ca20",
+  "model_requested": "deepseek/deepseek-v4-pro-0813",
+  "model_answered": "deepseek/deepseek-v4-pro-0813",
+  "schema_name": "tinysweeper_falsify",
+  "usage": {
+    "input_tokens": 1663,
+    "output_tokens": 299,
+    "cached_tokens": 640,
+    "embed_tokens": 0,
+    "cost_usd": 0.000707455
+  },
+  "value": {
+    "incorrect": []
+  }
+}

evals/cassettes/ts-0045-kernel-bypass-hallucination/0012-b5380fa68e6677d3.json

@@ -0,0 +1,17 @@
+{
+  "key": "b5380fa68e6677d3",
+  "model_requested": "deepseek/deepseek-v4-pro-0813",
+  "model_answered": "deepseek/deepseek-v4-pro-0813",
+  "schema_name": "tinysweeper_security",
+  "usage": {
+    "input_tokens": 3642,
+    "output_tokens": 506,
+    "cached_tokens": 0,
+    "embed_tokens": 0,
+    "cost_usd": 0.00202449
+  },
+  "value": {
+    "findings": [],
+    "summary": "The new string truncation and sentence extraction helpers are pure and safe; no security issues introduced."
+  }
+}

evals/cassettes/ts-0045-kernel-bypass-hallucination/0013-3d5e8e65650775c9.json

@@ -0,0 +1,17 @@
+{
+  "key": "3d5e8e65650775c9",
+  "model_requested": "deepseek/deepseek-v4-pro-0813",
+  "model_answered": "deepseek/deepseek-v4-pro-0813",
+  "schema_name": "tinysweeper_security",
+  "usage": {
+    "input_tokens": 2824,
+    "output_tokens": 508,
+    "cached_tokens": 0,
+    "embed_tokens": 0,
+    "cost_usd": 0.0016704
+  },
+  "value": {
+    "findings": [],
+    "summary": "The change adds a new submodule `summarise` to the findings module. No security-sensitive code is introduced in this file; the change is safe to merge."
+  }
+}

evals/cassettes/ts-0045-kernel-bypass-hallucination/0014-fb11783005e5e9fd.json

@@ -0,0 +1,17 @@
+{
+  "key": "fb11783005e5e9fd",
+  "model_requested": "deepseek/deepseek-v4-pro-0813",
+  "model_answered": "deepseek/deepseek-v4-pro-0813",
+  "schema_name": "tinysweeper_security",
+  "usage": {
+    "input_tokens": 3399,
+    "output_tokens": 648,
+    "cached_tokens": 0,
+    "embed_tokens": 0,
+    "cost_usd": 0.002042325
+  },
+  "value": {
+    "findings": [],
+    "summary": "The change updates the compose environment to require the GitHub App id/private key and to forward only the relevant API keys, and mounts a read-only config file; no security issues found."
+  }
+}

evals/cassettes/ts-0045-kernel-bypass-hallucination/0015-3651492c72f6983f.json

@@ -0,0 +1,17 @@
+{
+  "key": "3651492c72f6983f",
+  "model_requested": "deepseek/deepseek-v4-pro-0813",
+  "model_answered": "deepseek/deepseek-v4-pro-0813",
+  "schema_name": "tinysweeper_security",
+  "usage": {
+    "input_tokens": 3166,
+    "output_tokens": 744,
+    "cached_tokens": 0,
+    "embed_tokens": 0,
+    "cost_usd": 0.00202449
+  },
+  "value": {
+    "findings": [],
+    "summary": "The overlay adds a workaround for running MongoDB on newer Linux kernels by setting GLIBC_TUNABLES and mounting a custom entrypoint script. The script itself is not part of this diff and cannot be reviewed here. The change is explicitly scoped to local development and introduces no new security issues that are visible from the diff."
+  }
+}

evals/cassettes/ts-0045-kernel-bypass-hallucination/0016-96d6b84e54ee3b43.json

@@ -0,0 +1,17 @@
+{
+  "key": "96d6b84e54ee3b43",
+  "model_requested": "deepseek/deepseek-v4-pro-0813",
+  "model_answered": "deepseek/deepseek-v4-pro-0813",
+  "schema_name": "tinysweeper_security",
+  "usage": {
+    "input_tokens": 2943,
+    "output_tokens": 1048,
+    "cached_tokens": 0,
+    "embed_tokens": 0,
+    "cost_usd": 0.002191965
+  },
+  "value": {
+    "findings": [],
+    "summary": "The change adds configuration for embeddings retrieval using an API key from the environment; no security issues introduced."
+  }
+}