Static analysis toolchain for neural network weights - inspect, diff, fingerprint, scan, and anomaly-check model checkpoints without running them.
Pentimento (art history): a visible trace of an earlier painting beneath the layers of a finished work. Fine-tunes, merges, grafts, and implants all leave pentimenti in weight space. This project builds the tooling to see them.
Status: Phase 1 complete, Phase 2/3/4 core mechanisms implemented, calibration pending.
Every heuristic or uncalibrated output is marked EXPERIMENTAL and carries confidence: uncalibrated until it has been measured against a reference corpus - see Status below and docs/roadmap.md for the phase-by-phase detail.
We treat model weights the way the industry treated binaries in 1995: as opaque blobs we either run or don't.
Every security and provenance question about a checkpoint today is answered dynamically - run it, probe it, eval it. But dynamic analysis of a model is insufficient for the same reason dynamic analysis of a binary is: a backdoor that triggers on one rare input will never appear in your eval suite, exactly like malware that sleeps until a date check passes. Meanwhile the supply chain has exploded: millions of unvetted checkpoints on public hubs, fine-tunes of fine-tunes with no recorded lineage, and enterprises deploying downloaded weights with no equivalent of an antivirus scan.
What binary security built over 30 years - disassemblers, differs, signature scanners, provenance databases - simply does not exist for weights:
- Existing model scanners (modelscan, picklescan) check the serialization format for code-execution payloads. They never look at the weights semantically.
- Mechanistic interpretability looks inside models, but requires running them, and is aimed at science rather than practitioner tooling.
- Model-merging tools (mergekit) manipulate weights but do not analyze them.
- Academic work on weight fingerprinting and static backdoor detection exists, but as scattered one-off paper repos, not a unified, maintained toolchain.
Pentimento is a working attempt at that missing static toolchain.
Requires Python 3.11+.
$ pip install -e .This installs the pnt CLI and the pentimento Python package.
For development (tests, lint, type-check):
$ pip install -e '.[dev]'The commands below are real output, captured against a tiny synthetic llama-shaped checkpoint (2 layers, hidden size 16) built with the test-suite's safetensors helper - not hand-written mockups.
$ pnt inspect model.safetensors Tensors
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━┳━━━━━━━━━━┳━━━━━━━┳━━━━━━━┓
┃ Name ┃ Role ┃ Block ┃ Shape ┃ Dtype ┃ Quant ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━╇━━━━━━━━━━╇━━━━━━━╇━━━━━━━┩
│ lm_head.weight │ unembed │ - │ (64, 16) │ F32 │ f32 │
│ model.embed_tokens.weight │ embed │ - │ (64, 16) │ F32 │ f32 │
│ model.layers.0.self_attn.q_pr… │ attn_q │ 0 │ (16, 16) │ F32 │ f32 │
│ model.layers.0.mlp.gate_proj.… │ mlp_gate │ 0 │ (32, 16) │ F32 │ f32 │
│ ... │ ... │ ... │ ... │ ... │ ... │
│ model.norm.weight │ norm │ - │ (16,) │ F32 │ f32 │
└────────────────────────────────┴──────────┴───────┴──────────┴───────┴───────┘
╭──────────────────────────────── Architecture ────────────────────────────────╮
│ claimed: llama │
│ matched: llama │
│ blocks: 2 │
╰──────────────────────────────────────────────────────────────────────────────╯
No findings.
claimed comes from untrusted metadata (a config.json sidecar or GGUF KV table); matched is recovered from tensor-name template matching against the actual structure.
A mismatch between the two is itself a finding (wir.arch_mismatch), because a metadata lie is evidence.
$ pnt diff base.safetensors finetune.safetensors Touched layers by role/block
┏━━━━━━━━━━┳━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Role ┃ Block ┃ Tensors ┃ Mean rel. norm ┃ Max rel. norm ┃
┡━━━━━━━━━━╇━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ attn_q │ 0 │ 1 │ 1.379 │ 1.379 │
│ mlp_down │ 0 │ 1 │ 1.422 │ 1.422 │
│ ... │ ... │ ... │ ... │ ... │
│ norm │ - │ 1 │ 0 │ 0 │
└──────────┴───────┴─────────┴────────────────┴───────────────┘
╭───────────────────────────────── Change classification ──────────────────────────────────╮
│ change type: mixed (EXPERIMENTAL) │
│ recovered rank: - │
│ rule trace: │
│ [ ] identical: 16 touched tensor(s), 0 structural change(s) │
│ [ ] lora_graft: touched tensors are not all 2-D with a small, consistent, low │
│ recovered rank over a narrow role set │
│ [ ] full_finetune: only 76% of tensors touched, or the touched set is uniformly low-rank│
│ fallback: no preceding rule matched the observed delta-rank/locality/dtype profile │
╰────────────────────────────────────────────────────────────────────────────────────────────╯
findings: 1
Findings
┏━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━┓
┃ Severity ┃ Method ┃ Message ┃ Tensors ┃ Confidence ┃ Invariance ┃ Experimental ┃
┡━━━━━━━━━━╇━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━┩
│ MEDIUM │ diff.classific… │ delta profile │ - │ uncalibrated │ none │ EXPERIMENTAL │
│ │ │ classified as │ │ │ │ │
│ │ │ mixed ... │ │ │ │ │
└──────────┴─────────────────┴────────────────┴─────────┴──────────────┴────────────┴──────────────┘
Every classification decision shows its rule trace: which rule matched, which ones were checked and rejected, and why.
change type is a heuristic over rank/locality/dtype, not a calibrated detector - it ships EXPERIMENTAL per design invariant 5, and the CLI says so on every line that mentions it.
$ pnt lineage finetune.safetensors --index reference-checkpoints/ Lineage ranking: spectral_product
(GhostSpec, arXiv:2511.06390)
┏━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━┓
┃ Reference ┃ Score ┃ Confidence ┃
┡━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━┩
│ ref2.safetensors │ 0.9899 │ uncalibrated │
│ ref3.safetensors │ 0.9896 │ uncalibrated │
│ base.safetensors │ 0.9893 │ uncalibrated │
└──────────────────┴────────┴──────────────┘
Lineage ranking: aligned_cka (AWM, arXiv:2510.06738)
┏━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━┓
┃ Reference ┃ Score ┃ Confidence ┃
┡━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━┩
│ ref3.safetensors │ 0.1380 │ uncalibrated │
│ base.safetensors │ 0.1229 │ uncalibrated │
│ ref2.safetensors │ 0.1052 │ uncalibrated │
└──────────────────┴────────┴──────────────┘
findings: 2
Findings
┏━━━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┓
┃ Severity ┃ Method ┃ Message ┃ Tensors ┃ Confidence ┃ Invariance ┃ Experimental ┃
┡━━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━┩
│ LOW │ fingerprint.… │ backend │ - │ uncalibrated │ residual_rot… │ EXPERIMENTAL │
│ │ │ 'aligned_cka' │ │ │ norm_scale │ │
│ │ │ ranks │ │ │ │ │
│ │ │ 'ref3...' as │ │ │ │ │
│ │ │ nearest │ │ │ │ │
│ LOW │ fingerprint.… │ backend │ - │ uncalibrated │ head_permuta… │ EXPERIMENTAL │
│ │ │ 'spectral_pr… │ │ │ residual_rot… │ │
│ │ │ ranks │ │ │ │ │
│ │ │ 'ref2...' as │ │ │ │ │
│ │ │ nearest │ │ │ │ │
└──────────┴───────────────┴───────────────┴─────────┴──────────────┴───────────────┴──────────────┘
Every backend cites its published method and reports which symmetries it quotients (invariance_tags, e.g. residual_rotation, head_permutation).
Scores are ranked, not thresholded: confidence: uncalibrated means no reference corpus with known lineage has been run through this backend yet, so the ranking is directional evidence, not a calibrated match probability.
$ pnt scan finetune.safetensors --base base.safetensors Rule outcomes
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Rule ┃ Status ┃ Reason ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ lora_backdoor_spectral_v1 │ NOT_MATCHED │ no selected tensor satisfied the condition │
│ lora_refusal_removal_v1 │ NOT_APPLICABLE │ select resolved no tensors for roles ['attn_q', │
│ │ │ 'attn_v'] on this model │
└───────────────────────────┴────────────────┴─────────────────────────────────────────────────────┘
No findings.
Every rule reports its outcome even when it does not fire: NOT_MATCHED (evaluated, condition false), NOT_APPLICABLE (its preconditions, like requiring a declared base, were not met), or a finding.
Seed rules ship in rules/ as plain YAML; --rules <dir> layers in your own.
$ pnt anomaly build-corpus ref1.safetensors ref2.safetensors ref3.safetensors -o corpus --name demo-llama
wrote corpus 'demo-llama' (3 member(s)) to corpus
$ pnt anomaly check candidate.safetensors --corpus corpus╭───────────────────────────────────────── Anomaly check ──────────────────────────────────────────╮
│ architecture: llama │
│ has baseline: True │
│ baseline found: 42 statistic(s) beyond threshold 0.99 │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
(The 42 above is inflated by the toy example: the "candidate" checkpoint here is a wholly re-randomized tensor set, not a subtle drift, so nearly every per-tensor statistic trips the 99th-percentile threshold against a 3-member reference corpus.
Findings are always relative to a named, versioned corpus at a stated empirical percentile threshold; with no corpus for an architecture, pnt anomaly reports "no baseline" instead of guessing.)
Every subcommand supports --json (SARIF-inspired JSON on stdout, human output moved to stderr) and --fail-on {low,medium,high,critical}, and every run exits one of:
| Code | Meaning |
|---|---|
0 |
ran cleanly, nothing at or above the threshold |
1 |
at least one finding at or above --fail-on |
2 |
usage/input error - the analysis could not run at all |
3 |
the analysis ran but was not applicable (e.g. no reference corpus for this architecture) |
$ pnt inspect model.safetensors --json --fail-on high > report.json{
"$schema": "https://raw.githubusercontent.com/oasis-tcs/sarif-spec/master/Schemata/sarif-schema-2.1.0.json",
"runs": [
{
"properties": { "analysisKind": "inspect", "inputs": ["model.safetensors"] },
"results": [],
"tool": { "driver": { "name": "pentimento", "rules": [], "version": "0.1.0" } }
}
],
"version": "2.1.0"
}| Module | What it does | CLI surface |
|---|---|---|
pentimento.loader |
Streaming safetensors/GGUF readers and a restricted pickle-metadata parser (no unpickling, ever) | (backing all commands) |
pentimento.wir |
The Weight IR: WeightGraph, streaming per-tensor features (moments, spectra), architecture template matching |
(backing all commands) |
pentimento.canon |
Canonicalization: invariant product spectra, linear CKA, best-effort alignment mode | (backing diff, lineage) |
pentimento.diff |
Streamed tensor deltas, low-rank graft recovery, experimental change classification | pnt diff |
pentimento.fingerprint |
Spectral-product and aligned-CKA backends, lineage index, calibration hooks | pnt lineage |
pentimento.rules |
YAML rule language v1, sandboxed expression engine, seed rules | pnt scan |
pentimento.anomaly |
Reference-corpus building and population-relative outlier detection | pnt anomaly |
pentimento.report |
SARIF-inspired JSON, rich text rendering, CI exit-code contract | (shared by all commands) |
pentimento.cli |
Thin typer veneer wiring the above to pnt |
pnt |
Full module design, the Weight IR, and the symmetry catalog: docs/architecture.md. Rule language spec: docs/rules.md.
We have written down the limits up front, because a security tool that overclaims is worse than no tool.
-
It is not a backdoor oracle. Goldwasser, Kim, Vaikuntanathan and Zamir (2022) proved that cryptographically undetectable backdoors can be planted in some settings. No static tool can promise to catch a maximally resourced adversary, and we do not. Like YARA and antivirus, the goal is to raise attacker cost and reliably catch known implant families and low-effort attacks - which is the overwhelming majority of real-world abuse. See the threat model for the explicit adversary tiers we do and do not cover.
-
It does not output legal conclusions. Weight similarity is evidence, not proof. Pentimento reports calibrated findings ("spectral fingerprint match to Llama-3.1-8B, confidence 0.97, method AWM-style invariants") and never strings like "license violation." Turning evidence into a claim is a job for humans with lawyers.
-
Naive diffing is known to be broken, so we don't do it. Weight space has large symmetry groups - neuron/head permutations, scaling, and full orthogonal rotations of the residual stream (the computational-invariance result underlying SliceGPT) - under which two functionally identical checkpoints look numerically unrelated. Permutation-only alignment (Git Re-Basin-style) is provably insufficient for transformers, so Pentimento's primary path is invariant features (singular value spectra, invariant attention-product signatures, CKA) that quotient out the symmetry group analytically, with explicit alignment as a residual-tagged best-effort mode. The full symmetry catalog, with per-claim verification status, is in docs/architecture.md.
-
"Statistical outlier" requires a reference population, not vibes. Anomaly detection is defined strictly relative to empirical reference corpora built from public-hub checkpoints of the same architecture, shipped with measured false-positive rates. No corpus for your architecture yet means Pentimento says "no baseline" instead of guessing.
-
It must work at 70B+ scale, so nothing ever loads a full model. Every analysis is defined over streaming, memory-mapped, layer-at-a-time access to safetensors/GGUF - the loader contract forbids materializing the checkpoint.
Implemented and tested (323 tests, ~95% coverage, ruff + mypy --strict clean, CI on Python 3.11 and 3.12):
- Streaming loaders for safetensors, GGUF, and a restricted pickle-metadata parser.
- The
WeightGraphIR with streaming per-tensor features and architecture template matching. - Canonicalization: invariant product spectra, linear CKA, best-effort alignment mode.
- Diff: per-tensor deltas, low-rank graft recovery, change classification.
- Fingerprint: spectral-product and aligned-CKA backends, a lineage index, calibration hooks.
- Rules engine v1: YAML rule schema, a sandboxed expression evaluator, seed rules in
rules/. - Anomaly: reference-corpus building and population-relative outlier detection.
- Report: SARIF-inspired JSON, rich human-readable text, CI exit codes 0/1/2/3.
- A working
pnt inspect | diff | lineage | scan | anomalyCLI over all of the above.
Not yet done, and explicitly out of scope for a "v1" claim:
- No reference corpora built from real public-hub checkpoints exist yet -
pnt anomalyandpnt lineagework against whatever corpus/index you build yourself, but nobody has shipped a curated hub-scale one. - No fingerprint or anomaly backend has measured false-positive/false-negative rates on a held-out labeled corpus; every such finding is marked
EXPERIMENTALwithconfidence: uncalibrateduntil that changes (design invariant 5, docs/architecture.md). - Validation against real hub LoRA fine-tunes (rank/target-module recovery against published adapter configs) and a measured 70B-scale run are still open Phase 1 exit criteria - see docs/roadmap.md.
- No external rule contributions or separate
pentimento-rulesrepository yet.
Read docs/roadmap.md for the full phase-by-phase status and open exit criteria.
| Serialization scanners (modelscan, picklescan) | Merge/manipulation tools (mergekit, ckpt) | Academic fingerprinting repos (SeedPrints, AWM, GhostSpec) | Mech interp | Pentimento | |
|---|---|---|---|---|---|
| Looks at weight values semantically | ✗ | partial | ✓ | ✓ | ✓ |
| Runs without inference | ✓ | ✓ | mostly | ✗ | ✓ |
| Unified toolchain, not one paper's method | ✓ | ✓ | ✗ | ✗ | ✓ |
| Community-extensible signatures | ✗ | ✗ | ✗ | ✗ | ✓ |
| Honest threat model published | partial | n/a | rarely | n/a | ✓ |
Pentimento treats the academic work as its standard library, not competition: fingerprinting methods from the literature are pluggable backends behind one CLI and one report format. Citations and per-paper positioning: docs/related-work.md.
| Doc | Contents |
|---|---|
| docs/architecture.md | Module design, the Weight IR, canonicalization, streaming loader contract, methodology verification status |
| docs/threat-model.md | Adversary tiers, in/out of scope, theoretical limits, detection philosophy |
| docs/rules.md | The weight-signature rule language spec |
| docs/related-work.md | Annotated prior art: papers, tools, and what Pentimento borrows from each |
| docs/roadmap.md | Phased plan, current status, and open exit criteria |
| docs/faq.md | Preemptive answers to the sharpest objections |
Contributions wanted: reference-corpus methodology, fingerprinting/rule backends from the literature we missed, adversarial review of the threat model, and (once the loader corpus grows) fuzz seed corpora. See CONTRIBUTING.md and SECURITY.md.