`pipeline_output.json`: `results.vulnerable` is not recomputed after the caller/callee dedup (183 vs 175 findings)

[gadievron]

Evidence provenance (added 2026-08-22). The run-derived figures in this issue come from a
private scan that a reader cannot reproduce (raptor-e2e-20260818, a 9.85-hour run against
a Python target). They are reported for completeness, not as independently checkable evidence.
Every code claim below is pinned to public source at b5019628 and is checkable; where a
conclusion rests on the private numbers rather than on the code, treat it as my report of what
that run showed. This label was missing when the issue was filed.

Summary

build_pipeline_output runs _dedup_caller_callee over the confirmed findings and builds
findings from the deduped list, but results.vulnerable is computed from the metrics block and
never re-derived. The same file therefore reports results.vulnerable: 183 alongside 175 entries in
findings, and 175 disclosure documents on disk. The dedup itself is logged, not silent.

The same results dict has a second, independent defect: it folds protected into safe and so
carries no protected key at all.

Evidence

libs/openant-core/core/reporter.py:331 (HEAD b501962), inside build_pipeline_output:

    confirmed = _dedup_caller_callee(confirmed, all_results, call_graph_path)

core/reporter.py:129-198 is the helper; its last statement announces what it removed
(core/reporter.py:194-198):

    removed = len(confirmed) - len(deduped)
    print(f"[Report] Deduplicated {removed} caller/callee finding(s)", file=sys.stderr)
    return deduped

On this run it fired and logged:

grep -n "Deduplicated" run/scan.log
#   12098:[Report] Deduplicated 8 caller/callee finding(s)

findings_data is then built from the deduped confirmed list (core/reporter.py:333-334), but
results at core/reporter.py:562-570 is built from metrics, which was read once at
core/reporter.py:304 and is never touched by the dedup:

        "results": {
            "vulnerable": metrics.get("vulnerable", 0) + metrics.get("bypassable", 0),
            "safe": metrics.get("safe", 0) + metrics.get("protected", 0),
            "inconclusive": metrics.get("inconclusive", 0),
            # F13: errored units are part of `total` (see units_analyzed above), so the
            # results buckets must include them or they cannot reconcile to `total`.
            "errors": metrics.get("errors", 0),
            "total": total_units,
        },

The file contradicts itself

RUN=~/.openant/projects/gadievron/raptor-e2e-20260818/scans/7dbf9c691d7/python
python3 -c "
import json, sys
d = json.load(open(sys.argv[1]))
print('results        :', d['results'])
print('len(findings)  :', len(d['findings']))" $RUN/pipeline_output.json
ls $RUN/report/disclosures | wc -l

results        : {'vulnerable': 183, 'safe': 5292, 'inconclusive': 0, 'errors': 0, 'total': 5475}
len(findings)  : 175
175

183 - 175 = 8, exactly the count in the log line. 175 disclosure documents were produced, matching
findings, not results.vulnerable.

Second defect in the same dict: protected is folded into safe

core/reporter.py:564 adds metrics["protected"] into safe and emits no protected key.
report/prompts/summary.txt:20-26 asks for a five-row verdict table:

| Verdict | Count |
|---------|-------|
| Vulnerable | {n} |
| Safe | {n} |
| Protected | {n} |
| Inconclusive | {n} |
| Error | {n} |

The report producer is given pipeline_output.json as its input data, and that input has no
protected key and errors: 0. The produced report/SUMMARY_REPORT.md, lines 13-19:

13	| Verdict | Count |
14	|---------|-------|
15	| Vulnerable | 183 |
16	| Safe | 5292 |
17	| Protected | 0 |
18	| Inconclusive | 0 |
19	| Error | 0 |

Both non-zero rows are absent from the report:

python3 -c "import json,sys; print(json.load(open(sys.argv[1]))['metrics'])" $RUN/results_verified.json
#   {'total': 5475, 'vulnerable': 183, ..., 'protected': 414, 'safe': 4878, 'errors': 0}
python3 -c "import json,sys; print(json.load(open(sys.argv[1]))['summary']['metrics'])" $RUN/scan.report.json
#   {'total': 5475, 'vulnerable': 1, ..., 'protected': 408, 'safe': 4884, 'errors': 48, ...}
python3 -c "import json,sys; print(json.load(open(sys.argv[1]))['summary'])" $RUN/verify.report.json
#   {..., 'needs_review': 134, 'error_count': 48}

Protected | 0 is printed where the metric blocks report 414 (results_verified.json) / 408
(scan.report.json); Error | 0 is printed where scan.report.json and verify.report.json both
report 48. The protected count cannot be recovered from results at all, because the fold at
:564 is lossy.

(The 183 and errors: 0 figures inherited from metrics are themselves wrong for a separate
reason — the post-verify metrics recount in core/verifier.py:359-373 — filed as #284. This
issue is about results not being re-derived after the dedup, and about the lossy protected fold.)

Why it matters

pipeline_output.json is the reporting layer's single input, and it is self-inconsistent: the
headline count and the enumerable findings in the same object disagree by 8. A consumer that reports
results.vulnerable and a consumer that lists findings produce different answers from the same
file, and there is no field that explains the difference. Within this repo the disclosure count (175)
and the summary headline (183) diverge for exactly this reason.

The protected fold is a smaller but plainer loss: a verdict the pipeline computes and the report
template asks for is destroyed one line before it would have been written.

Suggested fix

1. Re-derive results.vulnerable from the deduped list rather than from metrics — e.g.
   "vulnerable": len(findings_data) — or, if the pre-dedup count is wanted, keep both under
   distinct names (vulnerable, vulnerable_before_dedup) plus a deduplicated count so the
   difference is explicit rather than contradictory.
2. Have _dedup_caller_callee return (deduped, removed_keys) so the caller can adjust counts and
   record what was collapsed, instead of the count only reaching stderr.
3. Emit protected as its own key in results rather than folding it into safe; the summary
   template already has a row for it.
4. Add an invariant test on build_pipeline_output: results["vulnerable"] == len(output["findings"])
   on a fixture where the dedup fires.

Corrected 2026-08-21 — items 1, 3 and 4 above conflict with merged PR #230. Do not apply them
as written.

results is a partition of units, not a list of findings. tests/report/test_results_bucket_reconciliation.py
states the invariant in its own docstring — "the pipeline_output results block must reconcile to
its own total", where total counts all units — and asserts
vulnerable + safe + inconclusive + errors == total.

• Item 1 ("vulnerable": len(findings_data)) sets a unit bucket to a finding count. Those
  are different populations, so it breaks the partition and the tests that pin it.
• Item 3 ("rather than folding") breaks test_folds_bypassable_and_protected_then_still_reconciles,
  which pins the fold with assert r["vulnerable"] == 2 and r["safe"] == 3 # folded. An additive
  change — leave safe as-is and emit a new protected key alongside — is compatible; replacing the
  fold is not.
• Item 4's invariant results["vulnerable"] == len(output["findings"]) is unsatisfiable for the
  same reason as item 1.

What survives: the self-contradiction this issue reports is real — results.vulnerable (183)
and len(findings) (175) are printed side by side and read as the same quantity. But the fix is to
make the distinction explicit (name the two populations differently, or emit a deduplicated
count), not to force one number onto the other. Item 2 stands unchanged. Item 5 below is
independent of all of this.

5. Add a cycle guard to _dedup_caller_callee (promoted from the note below on 2026-08-21 — it
   was filed as "Latent", which was wrong): skip a removal where the caller is itself in
   remove_keys, or where the callee is a caller of its caller. Executing the real function on a
   mutually-recursive pair sharing a non-zero CWE returns [] — both findings are deleted. The
   removal is logged (core/reporter.py:197 prints [Report] Deduplicated N caller/callee finding(s)), so this is not silent — it is mislabelled: a double-deletion is reported as
   ordinary deduplication, and the count alone gives a reader no way to tell the two apart. The
   outcome is still a false negative. Add a fixture for that pair.

Reproducible on demand, same helper

Re-scoped 2026-08-21. Filed under "Latent" — the wrong word. The failure is reproducible on
demand; only its occurrence in a given repository is data-dependent. Now fix item 5 above;
this section is the supporting detail.

_dedup_caller_callee (core/reporter.py:129-198) has no cycle guard. It builds remove_keys from
reverse_call_graph entries with exactly one caller, and two mutually-recursive findings sharing the
same non-zero CWE each name the other as their sole caller — so both land in remove_keys and both
are deleted.

Executing the real function on such a pair returns [] — both findings are deleted. As above
this is logged rather than silent, but the log is a bare count that does not distinguish a
double-deletion from ordinary deduplication. For a security scanner the outcome is a false negative.

What was data-dependent is only whether such a pair occurs: none did on this run, and the 8 removals
reconcile exactly to scan.log:12098. That makes it unobserved here, not latent in the code.

Fix: skip a removal where the caller is itself in remove_keys, or where the callee is a caller
of its caller — and add a fixture for the mutually-recursive same-CWE pair.

What I am not claiming

• The 8 dropped findings are not dropped silently. core/reporter.py:197 logs them and the log
  line is present at scan.log:12098. The defect is that results.vulnerable is not adjusted, so
  the emitted JSON self-contradicts.
• I am not claiming the 8 removals were wrong. I did not audit whether each collapsed pair was a
  genuine caller/callee duplicate; the dedup may be doing exactly what it intends.
• SUMMARY_REPORT.md is produced by an LLM from report/prompts/summary.txt. I am attributing the
  Protected | 0 and Error | 0 rows to the shape of the input data (results has no protected
  key and carries errors: 0), not asserting a deterministic code path from :564 to those exact
  characters.
• The blast-radius statement is scoped to consumers within this repo.

[gadievron]

Two follow-ups from a verification pass: one item should be promoted out of the "Latent" note into the
fix list, and one scope caveat.

The cycle guard is not latent in the code — only in this run's data

I filed the missing cycle guard in _dedup_caller_callee as a latent hazard. A reviewer executed the
real function on a mutually-recursive pair sharing a non-zero CWE — the case where each finding names
the other as its sole caller — and both findings were deleted, returning [].

So the failure is reproducible on demand; what was latent is only whether such a pair occurs in a
given repository (none did here — the 8 removals reconcile exactly to scan.log:12098). For a
security scanner, a silent double-deletion of two mutually-recursive findings is a false negative, and
it should sit in the numbered fix list rather than a closing note:

Skip a removal where the caller is itself in remove_keys, or where the callee is a caller of its
caller — and add a fixture for the mutually-recursive same-CWE pair.

Scope caveat: this fix leaves the headline number wrong

Making results.vulnerable consistent with len(findings) closes the self-contradiction inside
pipeline_output.json, but the resulting number is still 175 unadjudicated units presented as
confirmed vulnerabilities — because the count it is being reconciled against is inherited from
results_verified.json, which is #284's defect. In this run, 182 of those 183 records are incomplete
and 48 carry an error.

So this issue and #284 should be fixed together, or this one should be fixed second. On its own it
makes the file self-consistent about a number that is still wrong.

Everything else reproduced: 183 vs 175 vs 175 disclosure files, the scan.log:12098 "Deduplicated 8"
line, and the lossy protected fold (safe 5292 = 4878 + 414).

[gadievron]

Fix opened in #381 (CI green, all checks passing). results.vulnerable is now re-derived from the deduped findings list (results.vulnerable == len(findings), with vulnerable_before_dedup + a deduplicated delta so the difference is explicit), and protected is emitted as its own key — the lossy safe-fold that printed Protected | 0 is gone. The unit-partition closes including the dedup delta (F13 extended).