[Bug]: FactVerifier allows LLM fallback to overwrite deterministic verdicts

[rahuldass19]

QWED Version

5.0.0

Python Version

3.11.9

Operating System

Windows

Which engine is affected?

Fact Verifier

Input that caused the bug

from qwed_new.core.fact_verifier import FactVerifier

verifier = FactVerifier(use_llm_fallback=True)

claim = "The treaty was signed in 1842."
context = """
The treaty negotiations began in 1840.
Sources disagree on the final signing year.
Some references mention 1841, others 1843.
"""

result = verifier.verify_fact(
    claim=claim,
    context=context,
    min_confidence=0.95,
    provider="openai",  # any configured provider path that enables fallback
)

print(result)


### Expected behavior

When deterministic fact verification is low-confidence or inconclusive, the engine must fail closed.

Expected behavior:
- low-confidence deterministic analysis must not be upgraded into a final fact verdict by an LLM
- external model output may be attached as advisory reasoning only
- final verdict must remain deterministic, explicit, and non-pass unless proven from deterministic checks

QWED principle:
Risky interpretation may inform analysis, but it must never decide truth.


### Actual behavior

`FactVerifier.verify_fact()` currently allows LLM fallback output to overwrite the deterministic verdict when confidence is below the threshold.

Relevant code in `src/qwed_new/core/fact_verifier.py`:

```python
if confidence < min_confidence and self.use_llm_fallback and provider:
    methods_used.append("llm_fallback")
    llm_result = self._llm_fallback(claim, context, provider)
    if llm_result:
        verdict = llm_result.get("verdict", verdict)
        confidence = max(confidence, llm_result.get("confidence", 0) * 0.8)
        reasoning += f"\n\nLLM Analysis: {llm_result.get('reasoning', '')}"

This means:

• deterministic verification can end in one verdict
• an external LLM can replace that verdict
• the final engine output can become model-decided rather than verifier-decided

Additional context

Severity

HIGH

Why this violates QWED philosophy

QWED’s core principles are explicit:

• fail-closed by default
• zero trust in LLM outputs
• no heuristic acceptance of results
• verification must be deterministic and explicit

This code path violates those principles because the LLM is not just providing commentary. It is allowed to modify the final fact verdict.

That turns the fallback model into a truth-decider inside a verifier.

Attack / bypass scenario

An attacker can craft:

• low-signal claims
• ambiguous supporting context
• semantically noisy passages that keep deterministic scores below the confidence threshold

Once the engine enters the fallback path, an LLM response can steer the final verdict toward SUPPORTED or REFUTED, even though the deterministic verifier could not prove that outcome.

This creates a trust-leak from advisory model output into the core verification boundary.

Fix direction

Do not allow LLM fallback to mutate the final verifier verdict.

Required direction:

1. keep deterministic verdict ownership inside the deterministic verifier only
2. preserve LLM output as advisory metadata or separate analysis
3. if deterministic confidence is below threshold, return a non-pass state such as INCONCLUSIVE rather than a model-derived final verdict
4. ensure methods_used distinguishes proof-bearing methods from advisory methods

Suggested regression tests

• low-confidence deterministic path + LLM fallback -> final verdict must not change
• LLM says SUPPORTED while deterministic path is inconclusive -> final status must remain non-pass
• deterministic high-confidence path -> fallback must not run
• fallback reasoning may be attached, but final fact verdict must remain deterministic

QWED Principle To Preserve

An LLM may explain uncertainty, but it must never resolve truth on behalf of the verifier.

[rahuldass19]

Cross-reference: Diagnostic Architecture (#204)

This issue is the same structural family as #134 (ImageVerifier VLM fallback) — model output overwriting a deterministic verdict with confidence = max(confidence, llm_result.get("confidence", 0) * 0.8). The fix is identical in shape, but FactVerifier has one pre-#204 feature worth noting: methods_used: List[str] (line 39) already distinguishes "llm_fallback" from deterministic methods ("semantic_similarity", "keyword_overlap", "entity_matching", "negation_detection"). This is a partial analog of #204's advisory_checks — but it's a list of strings, not a structured separation of advisory vs proof-bearing.

Intersection point

#133 fail path	#204 Layer	What's missing today
verdict = llm_result.get("verdict", verdict) (line 191)	Layer 3	LLM verdict overwrites deterministic verdict — no proof_ref distinction: deterministic SUPPORTED (from entity matching) has evidence; LLM SUPPORTED has none. Both produce the same verdict string
confidence = max(confidence, llm_result.get("confidence", 0) * 0.8) (line 192)	Layer 3	confidence is model self-assessment in a verdict-deciding position — #204 forbids this. The * 0.8 discount keeps it as a verdict signal; under #204 it moves to advisory_checks
reasoning += f"\n\nLLM Analysis: {llm_result.get('reasoning', '')}" (line 193)	Layer 1 + Layer 3	Model reasoning concatenated into verification reasoning — #204 forbids model reasoning in diagnostics. Agent sees "LLM Analysis: <model opinion>" mixed with deterministic evidence
methods_used.append("llm_fallback") (line 187)	Layer 2	Good pre-#200 pattern — but it's a flat string list. No advisory_only: True flag, no constraint_id, no separation from proof-bearing methods at the type level
Return is raw dict (line 195-207) despite FactVerificationResult dataclass existing (line 35)	ALL	Even the existing dataclass isn't used — the engine returns an ad-hoc dict, making any future diagnostic consumer need an N-specific adapter

The deeper structural problem

_calculate_verdict() (line 470) produces 4 deterministic verdicts with different evidence qualities:

• REFUTED via negation conflict (line 515) — strong deterministic evidence
• SUPPORTED via aggregate score (line 518) — deterministic but score-based
• NEUTRAL (line 521) — no strong evidence either way
• INSUFFICIENT_EVIDENCE (line 524, 527) — explicit fail-closed

Today, all 4 + the LLM-overwritten verdict return the same {"verdict": ..., "confidence": ..., "reasoning": ...} shape. An auditor cannot distinguish "REFUTED by negation conflict" (strong proof) from "SUPPORTED by LLM fallback" (no proof) without parsing the reasoning string. Under #204, these must be structurally separated by proof_ref presence and advisory_checks population.

Conformance requirement

When implementing the fix direction ("Do not allow LLM fallback to mutate the final verifier verdict"):

1. Return type must be DiagnosticResult from Structured Verification Diagnostics — Architectural Completion of 3-Layer Diagnostic Model #204. The verdict states map:

   • Deterministic SUPPORTED → DiagnosticResult(status="VERIFIED", agent_message="Claim supported by evidence", developer_fields={"methods": ["semantic_similarity", "entity_matching"], "evidence": {...}, "constraint_id": "fact_verifier.deterministic"}, proof_ref=<hash of evidence>)
   • Low-confidence + LLM fallback → DiagnosticResult(status="UNVERIFIABLE", agent_message="Claim could not be deterministically verified", developer_fields={"deterministic_verdict": "INSUFFICIENT_EVIDENCE", "deterministic_confidence": 0.4, "advisory_checks": {"llm_verdict": "SUPPORTED", "llm_confidence": 0.65, "advisory_only": True, "constraint_id": "fact_verifier.llm_advisory_only"}, "methods_used": ["semantic_similarity", "llm_fallback"]}, proof_ref=None)
   • Deterministic REFUTED → DiagnosticResult(status="VERIFIED", agent_message="Claim refuted by evidence", developer_fields={"method": "negation_detection", "evidence": {...}}, proof_ref=<hash>)

2. methods_used must be promoted to structured advisory_checks. The existing list is a good foundation. Under Structured Verification Diagnostics — Architectural Completion of 3-Layer Diagnostic Model #204, each entry gains advisory_only: bool and constraint_id: str:

   developer_fields = {
       "methods_used": [
           {"name": "semantic_similarity", "advisory_only": False, "constraint_id": "fact_verifier.semantic_similarity"},
           {"name": "llm_fallback", "advisory_only": True, "constraint_id": "fact_verifier.llm_advisory_only"},
       ],
   }

   This makes the fix direction's "ensure methods_used distinguishes proof-bearing methods from advisory methods" structurally enforced — advisory_only is a field, not a caller convention.

3. confidence must not survive into DiagnosticResult.status. Same as [Bug]: ImageVerifier allows VLM fallback to decide final verification verdicts #134 — Structured Verification Diagnostics — Architectural Completion of 3-Layer Diagnostic Model #204 forbids confidence in verdict-deciding position. The max(confidence, llm_confidence * 0.8) pattern must be replaced by: deterministic confidence → developer_fields.deterministic_confidence (advisory); LLM confidence → developer_fields.advisory_checks.llm_confidence (advisory). status is set by proof presence, not confidence arithmetic.

4. reasoning field must split — same as [Bug]: ImageVerifier allows VLM fallback to decide final verification verdicts #134:

   • Deterministic evidence description → developer_fields.evidence
   • LLM analysis → developer_fields.advisory_checks.llm_reasoning (marked advisory_only=True)
   • Agent-facing summary → agent_message (agent-safe, no model reasoning)

5. Regression tests must assert on DiagnosticResult fields:

   # low-confidence + LLM fallback → final status must remain non-pass
   result = verifier.verify_fact(claim, context, min_confidence=0.95, provider="openai")
   assert result.status == "UNVERIFIABLE"
   assert result.proof_ref is None
   assert result.developer_fields["advisory_checks"]["advisory_only"] is True
   
   # deterministic high-confidence → fallback must not run
   result = verifier.verify_fact(claim="2+2=4", context="2+2 equals 4", min_confidence=0.7)
   assert result.status == "VERIFIED"
   assert result.proof_ref is not None
   assert all(not m["advisory_only"] for m in result.developer_fields["methods_used"])

Sequencing

Per QWED Principle 12 — #204 takes precedence. Fix #204 first to establish DiagnosticResult with proof_ref, advisory_checks, and the methods_used → structured promotion pattern, then implement #133's LLM/deterministic separation against that contract.

Fixing #133 in isolation would likely replace the fallback block with if confidence < min_confidence: verdict = "INSUFFICIENT_EVIDENCE" — correct behavior, but FactVerificationResult would still carry confidence and reasoning as core fields, and the existing methods_used list would remain a flat string list without advisory_only enforcement. #204 addresses both the return-type gap and the methods_used structural promotion.