accept signal only at tail of output, to handle self-corrected emissions

[iatsiuk]

problem

during the first claude review phase, claude can emit a signal marker, immediately realize the mistake in plain text, and keep working correctly - but ralphex has already captured the premature signal and terminates the run as failed.

real log excerpt (opus 4.7, first claude review):

[16:21:39] 0 errors remaining (warnings are pre-existing). running tests once more to confirm:
[16:21:44] 5/5 pass. committing the fixes:
[16:21:58] <<<RALPHEX:TASK_FAILED>>>
[16:21:58] wait - issues were found AND fixed, so per path b i should NOT output any signal. let me correct that.
[16:21:58] issues were found and fixed. stopping here per path b - the external loop will run another review iteration to verify the fixes.

failed: 2026-04-20 16:21:59 (9m23s) - runner: first review: review failed (FAILED signal received)

what actually happened: claude found 3 issues, fixed them, ran tests (5/5 pass), committed, then mistakenly emitted TASK_FAILED, then self-corrected in the next blocks quoting "path b" from the prompt verbatim. per path b in review_first.txt:83-87, no signal should have been emitted at all - another review iteration should run to verify the fixes. ralphex aborted the whole run instead.

prior art

the existing approach on similar bugs is known:

• Claude evaluation emits CODEX_REVIEW_DONE despite codex reporting findings #92 "claude evaluation emits CODEX_REVIEW_DONE despite codex reporting findings" - fixed with prompt only, code-level guard explicitly rejected as "ralphex second-guessing the llm"
• Review phase emits REVIEW_DONE signal multiple times #93 "review phase emits REVIEW_DONE signal multiple times" - fixed with prompt only
• claude review loop doesn't exit when no issues found #69 "claude review loop doesn't exit when no issues found" - mixed fix, added HEAD hash fallback for a different scenario

in this case the prompt is already explicit (review_first.txt:83-87 describes path b in plain text, claude even quotes it back verbatim), yet opus 4.7 still emits the signal first and self-corrects after. further prompt tuning may reduce the frequency but cannot rule out mid-output mistakes - the model has no way to "unsay" a marker it already emitted.

proposed approach (non-intrusive code fix)

instead of the current stream-time behavior in pkg/executor/executor.go:401-403:

if sig := detectSignal(text); sig != "" {
    signal = sig
}

which captures the first signal marker and never releases it, scan the accumulated output from the tail after claude exits:

1. find the last occurrence of any <<<RALPHEX:...>>> marker in the full output.
2. inspect the text that follows the marker until EOF.
3. if the trailing text is empty, whitespace, or very short (e.g. < 50 chars), accept the signal.
4. if substantial prose follows the marker, treat it as a self-correction and ignore the signal (same effect as no signal - the outer loop decides next step, typically another iteration).

this does not parse semantics, does not second-guess the llm's decision, and does not introduce keyword heuristics. it only answers one mechanical question: "did the model stop talking right after emitting the marker, or did it keep going?" a signal that is not the end of the output was, by observation, never intended as the final verdict.

why this is different from #92

#92's rejected guard re-evaluated the codex output in go (re-parsing findings) to override claude's decision. the proposed tail check does the opposite: it trusts whatever claude said last, and treats mid-output markers as incomplete drafts. there is no re-evaluation of the review itself.

edge cases

• valid pattern "signal is last line": still accepted.
• valid pattern "short punctuation/whitespace after signal": accepted via small threshold.
• pathological case where claude legitimately writes a long postamble after a valid signal: the outer loop runs one extra iteration, which is the same cost as the current false-positive on the other side.
• could be combined with a prompt tweak requiring the signal to be the final non-empty line, which makes the tail check the enforcement mechanism rather than a heuristic.

[umputun]

I don't like this approach. It addresses a model-level problem at the wrong level and effectively changes the signal contract from "marker appears in output" to "marker must be at the tail", which is a real behavior change with real risks. I've seen plenty of runs where claude emits a proper, intended signal and then follows it with a short explanation or summary — with the tail check that whole class of runs starts getting rejected as "self-corrected". And on top of that, custom review scripts and codex wrappers that do signal-then-summary would break silently.

Also, any threshold (50 chars, "substantial prose", etc.) is a heuristic we'd have to live with. #92 and #93 were intentionally fixed at the prompt level to avoid exactly this kind of second-guessing, and I'd rather stay consistent with that.

I'd prefer to push back on the model first. The Path B instruction is already explicit, but opus 4.7 still misfired — so the prompt clearly isn't strong enough for it yet. Worth experimenting with:

• making the final-line rule explicit: "if you emit a signal marker it MUST be the final non-empty line of your output, nothing after it"
• if prompt tightening isn't enough, we could introduce a new explicit cancel signal (doesn't exist today — would be something like <<<RALPHEX:CANCEL>>>). Narrow, explicit, no heuristics. But only worth doing if plain prompt wording doesn't hold
• tightening the Path B wording so the model doesn't reach for a signal reflexively after fixing

If prompt-only tweaks don't move the needle on opus 4.7 across a few runs, we can revisit. But I don't want to ship a tail-position check as the primary defense.

[umputun]

transfering to discussions