feat: operator-controlled sensitivity source map and the lineage sensitivity report (#5042)

[chernistry]

What

Slices 2 and 3 of #5042, on top of the projection merged in #5167.

• Slice 2 — the source-labelling map. load_sensitivity_source_map() and
  sensitivity_class_for_source() in core/lineage/sensitivity.py, mirroring
  load_trust_source_map() / trust_class_for_source() in provenance.py,
  plus a bundled reviewed table at templates/provenance/sensitivity_sources.yaml
  beside the existing trust_sources.yaml.
• Slice 3 — the report. bernstein lineage sensitivity <artefact|entry-hash>
  in cli/commands/lineage_cmd.py, rendering the effective class, the closure
  member that raised it and the walk through the graph that reaches it, with
  --json for scripts.
• Operator documentation for both in docs/compliance/lineage-export.md, and a
  line each in docs/release-notes/unreleased.md.

Explicitly not in this PR:

• No enforcement. Nothing refuses anything on the basis of a verdict; slice 4
  (core/security/toolcall_interlock.py, refusals chained, off by default) and
  slice 5 (clearance levels for models and agents) are untouched.
• No content-based inference. The map is operator-authored reviewed data; the
  DLP scanner's guesses do not enter this field.
• No change to trust_class, its ordering, its map or its fail-closed end.
• No writer that stamps sensitivity onto entries automatically. The map
  answers "what class does this source carry"; deciding where that answer is
  applied at write time belongs with enforcement.

Why

#5167 gave lineage entries a sensitivity field and a projection over the
closure, but nothing tells an operator which sources carry which class, and
nothing renders the projection. Two gaps follow from that.

The map: a classification has to come from somewhere reviewable. trust_class
already has that shape — a YAML table of source names the operator edits,
resolved project-local-then-bundled — and sensitivity needs the same one so the
two axes are configured the same way instead of one being a table and the other
being scattered constants.

The report: a verdict nobody can read is not operable. "This is confidential"
invites an argument. "This is confidential because it derives, through these
three hops, from that entry" ends it, and the second form is only available if
something walks the graph and prints the walk. That is the whole reason the
projection carries raised_by and path rather than just a class.

Both are read-only surfaces over the signed graph: the class the command prints
is recomputed from log.jsonl on the spot, not stored anywhere, so it cannot
drift from the entries it summarises.

How

The map. load_sensitivity_source_map(workdir=...) resolves
<workdir>/templates/provenance/sensitivity_sources.yaml when present, else the
bundled file, exactly as load_trust_source_map resolves trust_sources.yaml
from the same directory. A project-local file replaces the bundled table rather
than merging into it, so the table in force is exactly what the operator wrote
and a source they left out reads as unlisted rather than inheriting a bundled
row they never saw. sensitivity_class_for_source() returns
HIGHEST_SENSITIVITY_CLASS for an unlisted source — the mirror of
trust_class_for_source falling to public, with the fail-closed end inverted
with the axis.

Two decisions the issue leaves open, and how they are settled:

An unrecognised class token is dropped, not coerced. _coerce_trust_class
in provenance.py drops unparseable rows and this follows it, but the reason is
sharper on this axis: coercing would pick some class for a row the operator
got wrong, and the cheapest guess (the least sensitive class) is also the most
damaging one. A dropped row leaves the source unlisted, which fails closed high.

The fail-closed class is not written into entries. sensitivity_class_for_source
answers a question; it is not a label to stamp on an entry for a source nobody
classified. Recording restricted because a source is unlisted would assert a
classification nobody made, and would take the fail-closed decision twice —
once at write time and again in effective_sensitivity, which already applies
it to an unlabelled closure at read time. One rule, applied at read.

The report. sensitivity_cmd runs the lineage gate first and exits 1
without printing a class when it fails: a class projected from an unverified log
is not evidence of anything, which is the same posture lineage verify takes.
A target starting with sha256: is read as an entry hash and goes to
effective_sensitivity; anything else is an artefact path and goes to
sensitivity_for_artefact, which resolves the tip first. A missing log yields
an empty entry list and therefore the ordinary fail-closed verdict rather than a
second error path — absence of a log and absence of a classification are the
same fact, so they get one rule. Human output renders entry hashes with
soft_wrap so an operator copying one out of the report gets the whole hash.

The bundled table classifies the sources the repository already names in
trust_sources.yaml: operator attachments confidential, operator prompts and
workspace/repository reads internal, web and public-tracker fetches public,
gateway results internal.

Tests

tests/unit/lineage/test_sensitivity_sources.py (9) and
tests/unit/lineage/test_sensitivity_cli.py (6). All 15 failed on the
unmodified tree: the sources file fails collection with
ImportError: cannot import name 'load_sensitivity_source_map', and each CLI
test fails with Error: No such command 'sensitivity' / No such option '--log'.

Slice 2:

1. test_bundled_sensitivity_source_map_classifies_the_reviewed_sources — the
   bundled table is real reviewed data, not an empty stub, and every value is a
   SensitivityClass rather than a raw string.
2. test_unknown_source_fails_closed_to_the_highest_class — load-bearing.
   The inverted fail-closed end; an unlisted source is restricted, not
   public.
3. test_default_map_is_loaded_when_no_mapping_is_passed — the bundled table is
   used when no mapping is supplied.
4. test_workdir_map_overrides_the_bundled_default — a project-local file
   replaces the bundled table rather than merging into it, so a source the
   operator dropped reads as unlisted.
5. test_malformed_rows_are_dropped_without_dropping_the_whole_map — one bad
   row does not silently empty the table.
6. test_unrecognised_class_token_is_dropped_rather_than_downgraded — a typo
   leaves the source unlisted (and so fail-closed high) instead of being coerced
   to the nearest class.
7. test_class_tokens_are_read_case_and_whitespace_insensitively — " Confidential "
   parses.
8. test_unreadable_map_yields_an_empty_table_not_a_partial_one — an unparseable
   file yields {}, which by rule 2 reads as fail-closed-high everywhere rather
   than as a permissive default.
9. test_map_without_a_sources_list_yields_an_empty_table — same for a
   structurally wrong file.

Slice 3:

10. test_cli_reports_the_effective_class_and_the_path_that_produced_it —
    load-bearing. The summary case end to end: the command names the class,
    the closure member that raised it, and the hops that reach it.
11. test_cli_refuses_to_report_a_verdict_from_a_log_that_fails_the_gate —
    exit 1, no class printed.
12. test_cli_reports_the_fail_closed_default_for_an_unknown_artefact —
    restricted, not "unknown" and not an error.
13. test_cli_reports_the_fail_closed_default_when_there_is_no_log — a missing
    log takes the same path.
14. test_cli_json_output_carries_every_verdict_field — --json emits target,
    class, resolved, raised_by, path, closure and the sensitivity records.
15. test_cli_accepts_an_entry_hash_target — a sha256: target reports on that
    entry directly rather than being treated as a path.

Verification run:

• uv run python scripts/run_tests.py --parallel 2 -x tests/unit/lineage/test_sensitivity.py tests/unit/lineage/test_sensitivity_sources.py tests/unit/lineage/test_sensitivity_cli.py — 36 passed.
• Red confirmed by checking out origin/main copies of the two touched source
  files and the template, running both new files, then restoring.
• --affected origin/main selects far past the reviewable bound, so instead the
  tests of the touched modules and of their importers were run:
  tests/unit/lineage/test_cli.py, test_conflict_cli.py, test_spine_cli.py,
  tests/unit/test_lineage_v2_cli.py, tests/unit/lineage/test_provenance.py —
  72 passed; tests/unit/test_cli_command_registration.py — 699 passed, 1
  skipped; tests/unit/test_readme_api_coverage.py,
  test_feature_matrix_drift.py, test_unreleased_notes_rotation.py,
  tests/unit/test_nist_ai_rmf_mapping.py — 48 passed. CI runs the full suite.
• uv run ruff check src/ and uv run ruff format --check on the two touched
  files — clean.
• uv run mypy --config-file mypy.gate.ini — success, 91 source files.

Checklist

• uv run ruff check src/ passes
• uv run pyright src/ passes (no new findings on the touched files;
  sensitivity.py is clean, and the three pre-existing
  reportUnknown* findings on the conflict-resolution branch of
  lineage_cmd.py are byte-identical before and after this change)
• uv run python scripts/run_tests.py -x passes for the scope described above
• New code has type hints

Documentation duty

• N/A — user-visible README section: this is a compliance-operator surface
  documented in docs/compliance/, not a README feature
• docs/compliance/lineage-export.md — new section covering the command,
  its exit codes and the source map
• N/A — docs/api/ schema: no public schema changed
• uv run bernstein agents-md sync run — no changes produced
• Tests cover the documented behaviour
• docs/release-notes/unreleased.md entry added

Part of #5042

Remaining

• Slice 4: enforcement at core/security/toolcall_interlock.py, refusals
  written to the chain, off by default behind policy — acceptance test 8,
  test_refusal_at_the_read_boundary_is_a_chain_event.
• Slice 5: clearance levels for models and agents, tied to the registry in models: an installation has no record of which models it is permitted to use, and no way to prove what was permitted last month #5038.
• A write-time path that applies the source map to entries as they are recorded;
  it belongs with slice 4, where there is a policy to say when it applies.

[bernstein-the-conductor]

VERDICT: request-changes

Missing release-notes fragment for user-visible CLI surface.

This PR adds bernstein lineage sensitivity, a new CLI command with --json output and documented exit codes, but provides no fragment under docs/release-notes/fragments/. The unreleased.md entries exist, but requires a per-change fragment file for user-visible changes, and the release-notes rule in AGENTS.md is explicit: "a user-visible change -- new CLI surface, changed output, a security property, a removal -- without a fragment IS a blocking finding."

#	Finding	Where	Smallest fix
F1	Missing release-notes fragment for new CLI command	docs/release-notes/fragments/	Add 5042-sensitivity-source-map.md with the command surface and exit-code contract

---

F1 - Missing release-notes fragment

File: docs/release-notes/fragments/ (no 5042-*.md present)

What is wrong: The PR introduces bernstein lineage sensitivity <artefact> with --log, --cards, --json options and exit codes 0/1. This is user-visible CLI surface. The repo requires a per-change fragment file under docs/release-notes/fragments/ for such changes (, plus the review checklist). The unreleased.md entries are present and correct, but the fragment is missing.

Evidence checked:

• grep -r "5042" docs/release-notes/fragments/ returns empty
• states: "The release-notes requirement is satisfied by a per-change fragment file under docs/release-notes/fragments/; do not require or request an unreleased.md entry."
• The review checklist says: "a user-visible change -- new CLI surface, changed output, a security property, a removal -- without a fragment IS a blocking finding."

Smallest fix: Add a fragment file docs/release-notes/fragments/5042-sensitivity-source-map.md following the established pattern (see neighboring fragments like 4667-trace-export-subcommand.md). A brief heading plus paragraph ending in the issue number is sufficient.

---

Scope and implementation quality

The implementation is clean and well-aligned with existing patterns:

• load_sensitivity_source_map() correctly mirrors load_trust_source_map() from provenance.py
• sensitivity_class_for_source() inverts the fail-closed direction appropriately (unlisted → highest class)
• The CLI command runs the lineage gate first, refusing to emit a verdict from an unverified log ( satisfied)
• Exit codes are documented and tests verify both paths
• The YAML table at templates/provenance/sensitivity_sources.yaml follows the trust_sources.yaml structure

Tests as evidence: All 15 tests in test_sensitivity_cli.py and test_sensitivity_sources.py pass. Tests verify:

• Effective class propagation through lineage closure
• Fail-closed behavior for unknown artefacts and missing logs
• Gate-first refusal (exit 1, no class printed)
• JSON output carries all verdict fields
• Malformed map rows are dropped without breaking the whole table

No logic, security, or hygiene issues found. The code is minimal, focused, and follows the provenance.py template closely.

---

_{bernstein v3.19.0 - unattended review run run-20260902T123149p2911733Z - no operator in the loop}

Signed review receipt - verify with bernstein review-receipt verify

field	value
diff_hash	sha256:f89d767639a9fad49563bd7c503a5b8c6e6570ed21b685c8038b612c76a4591d
journal_entry_hash	sha256:16a530382cda64f92fbdf6d1a781c2d91887246221f7f9fc876ff9c80a7fa97f

[bernstein-the-conductor]

VERDICT: approve

---

_{bernstein v3.19.0 - unattended review run run-20260902T140811p3184667Z - no operator in the loop}

Signed review receipt - verify with bernstein review-receipt verify

field	value
diff_hash	sha256:5666e3183acdd07c1febd6087641a1e99805a4592250ecd2a0756c8d96bc19d0
journal_entry_hash	sha256:04d7edb548d38e3f0d67ca9f0454144baa754aa75f3cab10ca409f36feb59fc2