feat(correlate): Graph Phase 2 — provenance-carrying cross-API correlation engine - #207
Merged
Conversation
…aph P2) correlate_surfaces(a, b) -> CorrelationResult reports which of A's outputs correlate to B's inputs and WHY, on the §13.2 provenance ladder (never a learned float). Builds ON the shipped primitives (graph._entity_of / _sig_corroborates / genericity floor; compose's DECLARED join; hints' vocab). Ladder: tier 3 DECLARED (cross-API plan-eligible only when BOTH sides are customer-confirmed; provider-x-gecko alone -> candidate, guardrail 3/4) · tier 2 signature-corroborated (INTRA-API only, the §13.6 cross-API zero- false-high gate) · tier 1 INFERRED-name (intra plan-eligible; cross-API -> quarantined candidate, never auto-joined) · tier 0 (type is a HARD gate). Control-plane (invariant #1): the scorer's ONLY input is a frozen Correland carrying {name, type, value-domain signature, name/declared entity, provenance} — a payload/overlap/cardinality field literally cannot be passed. Adds Surface.correlate façade + `gecko correlate <specA> <specB>` CLI (thin, report-only). Two Pattern-B falsifiable proofs (offline, $0): no DECLARED hint -> candidate (tier<=1) even when name AND signature match; one confirmed entity on both sides -> tier 3 plan-eligible and cross_plan recovers the chain first-plan-correct. 9 new tests; full suite 1889 passed. Next (Phase 2b): correlate_surfaces tool on CatalogMcpSurface (aggregator). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… declared BLOCKING (Graph P2): correlate gated a provider-only x-gecko DECLARED entity to plan_eligible=False (a candidate), but compose.cross_plan read graph.declared — the MERGED provider∪customer vocab with provenance stripped — so a malicious provider self-declaration minted an EXECUTABLE cross-API chain (pulling a value from a victim surface into the attacker's param). plan_eligible was advertised as the trust gate but was not enforced by the actual planner. Fix (fail-closed, one provenance source): - graph.py: add SurfaceGraph.confirmed — the CUSTOMER-vouched subset of declared (injected declared_hints / the workspace confirm store, NEVER provider x-gecko). Populated in build_graph; carried into serialize() so an upgrade is content-addressed. declared stays the merged (untrusted) vocab, unchanged. - client.py: pass confirmed=self._declared_hints into build_graph. - compose.cross_plan: require the entity customer-confirmed on BOTH the consuming param's surface (conf_target) and the producing field's surface (_confirmed_field_producers reads graph.confirmed). A provider-only entity resolves to None — the same honest refusal as an unmatched input. plan_eligible is now the sole plan gate everywhere. - correlate._declared_split: read graph.confirmed instead of reconstructing from client._declared_hints — correlate and compose now share ONE provenance source, so the report and the executable plan can never disagree. Backward compat: declared unchanged; confirmed is additive. A graph with no customer hints has empty confirmed -> cross_plan refuses all cross-joins (fail-closed: nothing is customer-vouched). Tests: - test_correlate: new failing-first test_cross_plan_refuses_provider_only_declared (the probe pair -> None), test_confirmed_subset_is_customer_only_not_provider. - test_compose: the three cross-plan tests relied on provider x-gecko driving an executable plan — that was encoding the vulnerability. Updated to inject a CUSTOMER confirmation (MARKET_HINTS); test_cross_plan_needs_both_sides_declared sharpened to prove the SUPPLIER side must be confirmed too. Adversarial matrix (all refuse except both-confirmed): both provider-only, supplier confirmed only, consumer confirmed only -> None; both customer-confirmed -> chain recovers first-plan-correct. Full suite 1891 passed, 1 skipped; ruff + mypy clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Graph Phase 2 (#202 roadmap) — the heart of Video 2.
correlate_surfaces(a, b)reports which of API A's outputs correlate to API B's inputs, and why, with provenance — building on the shippedcompose.cross_plan/graph/hintsprimitives.The honest core
CorrelationBasisrides every link (entity, tier, provenance, fields, thesignalsthat fired, confidence) — the human-readable "why".The security gate found and fixed a real hole
defi-security review caught that
correlate.pycorrectly refused a provider-only self-declaration, but the executable plannercross_planread the merged DECLARED vocab with provenance stripped — so a malicious provider declaring an entity could mint an executable cross-API chain (value misroute). Fixed fail-closed:SurfaceGraph.confirmed= the customer-vouched subset (fromgecko graph confirm), one provenance source for both engines.cross_plannow requires both sides customer-confirmed; provider-only → refused (None), same as an unmatched input.Guardrails
Control-plane typed input (
Correlandstructurally can't carry a response value / traffic);to_dict()emits structure only (no payload/secret/address); tier stays honest provenance,plan_eligibleis the sole plan gate everywhere.Proof (Pattern B, offline)
No DECLARED → the cross-API token link is a quarantined candidate; one customer-confirmed entity on both sides → tier-3 plan-eligible +
cross_planrecovers the chain first-plan-correct. Provider-only → refused by report and planner.Surface
Surface.correlate(other)+gecko correlate <specA> <specB>. Next (Phase 2b): the aggregator MCP toolcorrelate_surfacesonCatalogMcpSurface.Tests:
test_correlate.py+ updatedtest_compose.py(three tests were encoding the vulnerability — now use customer confirmations). Gate: ruff + mypy clean · full suite 1891 passed / 1 skipped.🤖 Generated with Claude Code