Skip to content

security: kb.explain_ranking leaks status-filtered (archived/retracted) claim and page text #650

Description

@philluiz2323

What happened

explain_ranking() in src/vouch/explain_ranking.py correctly labels a
retracted/superseded/redacted claim or an archived page with
gate: "status-filtered", but still hands back its full, live text in the
summary field.

# src/vouch/explain_ranking.py:246
live = _filter_live_hits(store, scoped)
snapshots.append(_Snapshot.of("status_filter", live))
...
# src/vouch/explain_ranking.py:291
summaries = {_key(h): h[2] for h in scoped}

summaries is built from scoped — the candidate set after viewer/scope
filtering (filter_hits) but before _filter_live_hits, which is what
actually drops retracted/superseded/redacted claims and archived pages.
Every candidate's summary is looked up from this pre-status-filter dict
regardless of which stage removed it, so a status-filtered candidate keeps
its live summary even though the gate says it was dropped.

The comment directly above this line explains the intended invariant:

# Summaries come from the *scoped* set: a candidate the viewer cannot
# retrieve keeps its gate attribution — that is the point of the report —
# but not its text. `kb.search` and `kb.context` withhold that summary for
# this viewer, so this surface must not hand it back.

#640 implemented exactly this invariant for the scope_filter stage
(viewer-permission scoping) — re-sourcing summaries away from the raw
fused set so a scope-filtered candidate's text is withheld. But scoped
(the fix's chosen source) is itself downstream of scope filtering only;
it's still upstream of _filter_live_hits, so the same invariant was
never extended to the status_filter stage.

What you expected

A candidate reported with gate: "status-filtered" should have its
summary withheld (empty string), matching how kb.search and
kb.context already exclude retracted/archived content entirely — the
same standard explain_ranking already applies to scope-filtered
candidates.

Reproduction

import json
from vouch.storage import KBStore
from vouch.models import Claim, ClaimStatus, ClaimType
from vouch.explain_ranking import explain_ranking
import tempfile, pathlib

d = pathlib.Path(tempfile.mkdtemp())
store = KBStore.init(d)
src = store.put_source(b"evidence", title="s")
store.put_claim(Claim(
    id="c1", text="jwt legacy signing key is hunter2-example-retracted",
    type=ClaimType.FACT, status=ClaimStatus.SUPERSEDED,
    evidence=[src.id], approved_by="human",
))
result = explain_ranking(store, query="jwt", limit=5)
print("hunter2-example-retracted" in json.dumps(result))  # True
for c in result["candidates"]:
    if c["id"] == "c1":
        print(c["gate"], repr(c["summary"]))
        # status-filtered 'jwt legacy signing key is hunter2-example-retracted'

The secret payload is present verbatim in the JSON response despite the
gate correctly reporting the candidate as filtered out. All 31 existing
tests in tests/test_explain_ranking.py pass — test_retracted_claim_ reports_status_filtered and test_archived_page_reports_status_filtered
only assert the gate value, never summary, so this gap was never
caught.

Environment

  • vouch version: test branch @ current HEAD
  • Python version: 3.11+
  • OS: any
  • Host: any — reachable via kb.explain_ranking (MCP), JSONL, and
    vouch explain-ranking CLI

.vouch/ state

Not required to reproduce — the repro above uses a fresh temp KB.

Anything else

This is the same "archived/retracted content leaking into a read surface"
bug class fixed roughly a dozen times elsewhere (digest, salience, context,
search, hot-memory, triage, followups-due, verify/doctor) — but
explain_ranking.py is a brand-new module (#628, hardened for the
scope-filter case in #640) that never got this specific instance of the
fix. It's a genuine confidentiality bug: content retracted/superseded/
redacted/archived — sometimes specifically because it was a secret or
incorrect — is fully readable through a diagnostic surface any caller with
query access can invoke, defeating the point of those statuses.

Suggested fix: source summaries from live (post-status-filter) instead
of scoped (post-scope-filter only) — live already exists at line 246
and represents exactly the set that survived both the scope and status
gates, matching what kb.search/kb.context expose. Add a test mirroring
test_scope_filtered_candidate_does_not_carry_its_summary (from #640) for
a SUPERSEDED/ARCHIVED claim and an archived page, asserting
summary == "" and that the secret text is absent from
json.dumps(result).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions