Skip to content

Latest commit

 

History

157 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

finding-bridge

License: Apache-2.0 Python 3.12+ Canonical schema 0.5.0 SARIF 2.1.0 AI in the evidence path: none tests: 357 collected garak fixtures: 0.16.0 gate container

Turn AI red-team tool output into standard, sealed, provenance-stamped findings. Feed them into the systems your team already uses.

Think of it as pandoc for AI red-team findings. Attack tools find failures. finding-bridge is where a failure becomes a finding: normalized, provable, sealed, and ready to share.

It never replaces your tools. It feeds them.

Every line of code here was written by an AI (Claude, in Claude Code) under a human director who wrote none of it. How that was governed, measured rather than asserted, is in How this project was built.

Contents

What it does

  • Reads garak hitlogs and manual attack transcripts.
  • Seals harmful content. It is encrypted at rest and shown as a "safe metadata preview": length, line count, a keyed digest, and harm flags. Triagers can score findings without re-reading raw harm. Unsealing is always explicit and always logged.
  • Stamps every finding with hashes and a tamper-evident chain, so a finding stays credible after the model changes.
  • Waits for a human. Nothing becomes a confirmed finding until a person confirms it. The record shows who and when.
  • Emits Markdown packets and SARIF 2.1.0 that real tools accept, plus a provisional FLARE-AI report set (marked provisional in the file, because FLARE-AI has not published a schema yet).

No AI runs anywhere in this pipeline. No API key is needed, ever. The test suite proves it on every run.

Architecture

The sealing boundary is the line that matters. Raw model output crosses it once, on the way in, and comes back out only through an explicit, logged unseal.

flowchart LR
    subgraph sources[Sources]
        G[garak hitlog JSONL]
        T[attack transcript, text or JSON]
    end
    subgraph core[Deterministic core: no AI, no API key]
        I[ingest, adapters/in_]
        subgraph boundary[Sealing boundary: raw content encrypted at rest]
            S[(sealed store, Fernet blobs)]
            X[(exposure log, append-only)]
        end
        P[safe metadata preview]
        H[stamp: RFC 8785 hash and chain]
        D[dedup, exact match]
        C{human gate: confirm or reject}
        L[(ledger and head)]
    end
    subgraph out[Destinations]
        M[Markdown packet]
        SA[SARIF 2.1.0]
        TR[tracker JSON]
        F[FLARE-AI, provisional]
    end
    AI[caged --ai: prose suggestions only, off by default]
    G --> I
    T --> I
    I -->|raw content| S
    I --> P --> H --> D --> C --> L
    L --> M
    L --> SA
    L --> TR
    L --> F
    S -.->|unseal --explicit, every read logged| X
    AI -.->|reads the preview, never sealed content| C
Loading

Nothing to the right of the human gate carries raw content. Every emitted artifact carries the preview and metadata only.

See it in action

The five-minute tour

The repo ships synthetic example files. The "harmful" text in them is fake. Strings like SENTINEL-HARM-7001 are labeled stand-ins, never real model output.

finding-bridge ingest-garak schemas/fixtures/garak.synthetic.hitlog.jsonl
finding-bridge list
finding-bridge confirm <id-from-list>
finding-bridge verify
finding-bridge emit-markdown out/packet.md
finding-bridge emit-sarif out/findings.sarif
finding-bridge emit-flare out/findings.flare.json

What you will see, captured from a real run on 2026-08-25 (store and key in a scratch folder outside the repo; ids and hashes are store-local, so yours will differ):

$ finding-bridge ingest-garak schemas/fixtures/garak.synthetic.hitlog.jsonl
{"ingested": 3, "total_candidates": 3, "duplicates_marked": 1}
[exit 0]

$ finding-bridge list
fb-d867243a67f8131d  garak  [sealed content: 77 chars, 1 lines, keyed digest 2a9876b5; harm flags: garak-detector:mitigation.SyntheticDetector. Content is sealed; unseal is explicit and logged.]
fb-2fc91b3d90be8c23  garak  [sealed content: 77 chars, 1 lines, keyed digest 2a9876b5; harm flags: garak-detector:mitigation.SyntheticDetector. Content is sealed; unseal is explicit and logged.] duplicate-of fb-d867243a67f8131d
fb-0b8f4fd0e93add46  garak  [sealed content: 47 chars, 1 lines, keyed digest a67633f5; harm flags: garak-detector:promptinject.SyntheticDetector. Content is sealed; unseal is explicit and logged.]
[exit 0]

$ finding-bridge confirm fb-d867243a67f8131d
confirmed fb-d867243a67f8131d by MohdSaifHussain <263689115+MohdSaifHussain@users.noreply.github.com>
[exit 0]

$ finding-bridge verify
chain verifies clean
[exit 0]

$ finding-bridge emit-markdown out/packet.md
wrote out/packet.md
[exit 0]

$ finding-bridge emit-sarif out/findings.sarif
wrote out\findings.sarif and out\findings.fb.jsonl
[exit 0]

And two refusals, because how the tool refuses is part of the product:

$ finding-bridge confirm fb-0000000000000000
unknown-id: no candidate with id 'fb-0000000000000000'
[exit 1]

$ finding-bridge unseal sealed/2a9876b508a0d513
unseal-not-explicit: unseal of 'sealed/2a9876b508a0d513' requires explicit=True (charter: unsealing is always explicit and logged)
[exit 1]

The artifacts from a run like this, with the full transcript beside them, are committed under examples/.

See docs/USAGE.md for the full walk-through, every command, and the reason-code reference.

The worked examples

Install

Python 3.12 or newer.

git clone <this repo>
cd finding-bridge
pip install -e .

This gives you the finding-bridge command.

That is the developer route. The hash-verified route below is the secure route; the lock file is what makes it one, not ceremony.

To verify dependency hashes as well, install the locked dependencies first, then the wheel without dependencies. This is the route CI runs in a fresh venv on every push (.github/workflows/gate.yml):

pip install build
python -m build --wheel
pip install --require-hashes -r constraints.txt
pip install --no-deps dist/finding_bridge-1.0.0-py3-none-any.whl

constraints.txt locks every runtime dependency to an exact version with PyPI's hashes, and --require-hashes makes pip refuse anything that does not match. Hash-checking needs a wheel and --no-deps: pip cannot hash-check an install from a source directory, and once any requirement carries a hash pip requires one for all of them, so -c constraints.txt on a wheel install does not work either (finding F-6, STEP-06).

The container route (the image is digest-pinned, non-root, and carries git because the human gate needs it for identity; the store, the key and your gitconfig are mounted from outside):

docker run --rm -v "%CD%\store:/work/store" -v "%CD%\key:/home/fb/key" -v "%USERPROFILE%\.gitconfig:/home/fb/.gitconfig:ro" ghcr.io/mohdsaifhussain/finding-bridge:1.0.0 --store /work/store --key /home/fb/key/fb.key list

How this project was built

Every line of code here was written by an AI (Claude, in Claude Code) under a human director who wrote none of it. What makes this repository unusual is not that fact but the apparatus around it, which is measured below. Every figure names the command or file it is computed from, at commit time; a figure that did not compute cleanly is absent, not rounded.

  • Tests: 357 collected; 265 (74.2 percent) exercise the product, 92 (25.8 percent) exercise the governance instruments that keep the AI honest. Counted by node id from python -m pytest --collect-only -q. Governance means the instruments themselves: the gate guard, the overclaim scanner, the digest-comparison scan, the boundary table, the environment scrub, the AI-cage structural test, the badge-truth test, the installed-package proof, and the schema drift tests (tests/test_gate_guard.py, test_no_overclaim.py, test_no_inline_digest_compare.py, test_boundary_table.py, test_environment.py, test_ai_caged.py, test_readme_badges.py, test_installed_package.py, test_release_labels.py, three named drift tests in test_schema.py, and the fixture-currency test in test_real_shapes.py). RFC 8785 conformance vectors count as product.
  • Every governance tool was built after its failure class occurred. The rule census (docs/RULE-CENSUS.md, D-068) states it as a measurement: "Seven-for-seven: every rule that is now a check became one after it failed at least once." The method converts failures into instruments; the floor is one instance of each class, and the census says so.
  • Census: 68 numbered decisions inventoried; of the 23 rules in its classification tables, 18 are CHECK, 1 a partial check, 4 still HABIT (each already broken at least once, counted there); a further section lists the rules ruled correctly sentences. Counted from the census tables by section. A rule that failed twice became a tool both times (the gate-half-run rule, seven instances, converted to tools/gate.py; the digest-comparison rule, three failures, converted to one helper plus a scan).
  • Corrections: 13 entries, 6 of the director's errors and 7 of the AI's, each with the original claim quoted, what proved it wrong, and the direction it moved, including the director's own false alarm (C-007). Counted from the corrections table in DECISIONS.md.
  • Rulings: 88 numbered, across 7 phase contracts, each with declared review stops. Counted by grep -c "^## D-0" DECISIONS.md and ls docs/decisions/STEP-0*.md. Decisions the AI took alone: 4, all in the PROV register, all later ratified.
  • Escape rate: the per-phase builder evals (evidence/builder-eval-*) record defects that reached the director, with denominators of 15 and 6; too small for a trend, and the evals say so themselves. Not restated here as a trajectory.

The full method is on the record: DECISIONS.md, the phase contracts in docs/decisions/, and the corrections table, where both the AI's and the director's errors are recorded with direction named. The boundary, stated so the method is not overclaimed: the AI wrote the code; the human ruled, reviewed, and verified; neither alone produced this.

Honest numbers

Every figure here names the command that produced it and the date. If a figure and the tree disagree, the tree wins and the figure is wrong.

  • Tests: 356 passed, 1 skipped, run by python tools/gate.py on 2026-08-25 after the post-release closure phase with no API key in the environment (the suite scrubs key-bearing variables and proves it). The one skip is the Windows key file permission check, which needs a POSIX file mode.
  • Product versus governance tests: 265 versus 92. Governance tests check the project's own rules and record rather than finding behaviour (tests/test_gate_guard.py, test_no_overclaim.py, test_no_inline_digest_compare.py, test_installed_package.py, test_environment.py, test_readme_badges.py, and the rest of the governance list in the provenance section above). Counted by node id from python -m pytest --collect-only -q (92 of 357 collected).
  • Mutation testing, reported both ways (raw, and excluding the annotation-class equivalents, the frozen method of D-066). Last audit at the STEP-05 close, evidence/mutation-audit-step05-close.md: provenance 226/341 = 66.3 percent raw, 226/242 = 93.4 percent adjusted; sealing 139/151 = 92.1 percent both ways. Carried unmeasured since their last audit: schema 10/26 = 38.5 percent, dedup 49/63 = 77.8 percent. Of the surviving mutants, 111 of 125 are judged equivalent by the builder's reasoning, not by a machine; that number is a stated limit, not a footnote.
  • Built by an AI under a human director: see the provenance paragraph at the top of this file; the figures here are the AI's measurements and the director's re-runs, and the corrections table records where either was wrong.

Notation

The record uses short prefixes. Decoded once, here:

Prefix Meaning Where
D-nnn a numbered ruling by the director DECISIONS.md
OB-n an obligation carried by name until discharged obligations register in DECISIONS.md
DEV-n a numbered deviation from a ratified phase contract the docs/decisions/STEP-nn-*.md file it deviates from
PROV-n a provisional decision taken alone, pending ratification PROV register in DECISIONS.md
C-nnn a correction: original claim quoted, correction, proof, direction corrections table in DECISIONS.md
STEP-nn a phase contract docs/decisions/

Where the record lives

Every decision, limit, and open obligation is written down: DECISIONS.md, docs/PROJECT_CHARTER.md, and the phase contracts in docs/decisions/.

  • SOP.md: the runbook. Every procedure was executed before it was written: init, ingest, the gate with and without --ai, verify with a 2am reason-code table, unseal with the exposure log read back, every emit, backup and restore, the rotation walk, and the incident path for a verify failure you did not expect.
  • docs/STANDARDS.md: field-by-field alignment with OWASP Top 10 for LLM Applications 2026, the OWASP GenAI Red Teaming Guide 1.0, Google SAIF, MITRE ATLAS 5.6.0 and NIST AI 600-1, from fetched sources, with the non-alignments stated.

Honest limits (short form)

  • The preview is metadata: length, line count, a keyed digest, and harm flags. It is not a summary of the content. A meaning-level summary would need AI, and no AI is allowed in this pipeline.
  • Tamper-evidence bound: the hash chain and its head detect accident, drift and casual edit. They do not defend against an attacker with write access to both the ledger and its head at once.
  • Finding ids are local to one store. Two analysts ingesting the same file get different ids.
  • Duplicate detection is exact-match only. Similar-but-not-identical findings are not clustered.
  • The encryption key can be rotated (rotate-key), and the rotation is recorded in the ledger as a supersession event. The separate key that produces sealed references is permanent and is not rotated.
  • Input files are capped at 10 MiB.
  • The sealed key file is not permission-locked by the tool on Windows; the operator does it with icacls (see docs/USAGE.md).
  • The chain head has no external trust anchor. Verification is against the store's own head, so the bound above is the whole guarantee (OB-4, due the first time a store crosses a trust boundary).
  • The parsers have not been fuzzed against data at volume that this project did not generate (OB-5, scoped out until that happens).
  • The FLARE-AI export is provisional: FLARE-AI has published no machine-readable schema, so the field names come from its paper.
  • The grey-scale idea behind the preview is research-informed, not research-proven: the cited evidence is image-moderation research, not a red-teaming trial (charter section 6).
  • The mutation figures above are the builder's measurement, and most surviving mutants are dispositioned by the builder's reasoning.

The full list, in user language, is in docs/USAGE.md.

License

Apache-2.0. See LICENSE and NOTICE.

About

Turn AI red-team tool output into standard, sealed, provenance-stamped findings. Feed them into the systems your team already uses.

Topics

Resources

Code of conduct

Contributing

Security policy

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Used by

Contributors

Languages