fix(server): plain-text call:<verb>{} tool-call detection (parser + emitter wiring)

[easel]

Summary

Closes the loop on Gemma-4-style plain-text tool emissions reaching /v1/messages clients as proper Anthropic tool_use content blocks. Combines the parser delta from the earlier #323 (which was closed and folded into PR #285) with the previously-missing emitter wiring discovered on 2026-06-01.

Both layers are required for the fix to work end-to-end:

• Parser (cherry-picked from fix(server): parse gemma's call:<verb>{} plain-text tool emissions #323 + the underscore-prefix follow-up): adds pattern 6 — call:<ns>?<verb>{relaxed-JSON args} — to tool_parser.cpp so plain-text Gemma emissions like call:get_country_info{country: "France"} or _call:... (SentencePiece tokenizer artifact) parse to structured ToolCalls.
• Emitter wiring: the SSE emitter only entered TOOL_BUFFER mode when the model emitted the literal <tool_call> XML opener. For plain-text emissions it stayed in CONTENT mode and never invoked parse_tool_calls. Added a CONTENT-mode finalize branch that runs the parser on accumulated text when a call: substring is present, hoists matches into tool_calls_, strips the spans from accumulated_text, and flips finish_reason to tool_calls so the existing Anthropic/OpenAI serialization paths emit tool_use content blocks.

Empirical signal

Live smoke test on 2026-05-31 against lucebox-hub:fac7e0f-cuda12 (easel/feat/lucebox-docker tip, which had the parser merged but not yet the emitter wiring):

POST /v1/messages with tools=[get_country_info] →
  stop_reason: "end_turn"
  content[0]: {"type": "text", "text": "_call:get_country_info{country: \"France\"}..."}

Expected (and after this fix):

  stop_reason: "tool_use"
  content[N]: {"type": "tool_use", "name": "get_country_info", "input": {"country": "France"}}

The original PR #323 didn't catch this because its tests called parse_tool_calls directly rather than going through the emitter; the parser was correct but unreachable in the live path.

History

• fix(server): parse gemma's call:<verb>{} plain-text tool emissions #323 (closed, folded into feat(lucebox): docker stack + CLI + bench/profile + harness + luce-bench in-tree #285): the parser layer landed on feat/lucebox-docker.
• This PR brings BOTH layers to main together — they're inseparable for functional correctness, and the emitter wiring on its own has no parser to invoke against Luce-Org/main.
• feat(lucebox): docker stack + CLI + bench/profile + harness + luce-bench in-tree #285 (feat/lucebox-docker) carries the equivalent changes via separate commits; whichever lands first wins and the other auto-resolves.

Out of scope

• Gemma4 / Laguna soft-close ports (separate work in PR feat(server): soft-close thinking termination via logit-ratio peek #326)
• Docker image rebuild + live-service e2e validation (reviewer / operator, post-merge)
• Streaming SSE: scope decision documented in the plan doc; non-streaming path is the priority

Files

• Plan: docs/experiments/sse-emitter-content-mode-tool-parse-plan.md (includes codex review verbatim)
• Plan (parser): docs/experiments/server-call-verb-tool-parser-plan.md (from fix(server): parse gemma's call:<verb>{} plain-text tool emissions #323 cherry-pick)
• Parser: server/src/server/tool_parser.{cpp,h}
• Emitter wiring: server/src/server/sse_emitter.cpp
• Tests: server/test/test_server_unit.cpp (+14 parser, +9 emitter)

Tests

23 new test cases across parser + emitter layers. All passing in a standalone driver build (the sub-agent's CMake build was interrupted by harness timeout before completing; reviewer should confirm via cmake --build server/build --target test_server_unit && server/build/test_server_unit).

🤖 Generated with Claude Code

[cubic-dev-ai]

3 issues found across 6 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="server/src/server/tool_parser.cpp">

<violation number="1" location="server/src/server/tool_parser.cpp:248">
P2: String rewrite misses escaping inner double quotes. Relaxed JSON like single-quoted text with `"` inside can fail parse and silently lose a tool call. Escape `"` when normalizing non-double-quoted strings.</violation>
</file>

<file name="server/src/server/sse_emitter.cpp">

<violation number="1" location="server/src/server/sse_emitter.cpp:49">
P2: Pre-check blocks digit-start verbs. Parser accepts them, so valid plain-text calls can be skipped. Allow digits in first verb char.</violation>

<violation number="2" location="server/src/server/sse_emitter.cpp:703">
P2: Stripping accumulated_content_ here can desync Responses stream. Deltas already sent raw call text, but done/completed now send stripped text.</violation>
</file>

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[cubic-dev-ai]

P2: String rewrite misses escaping inner double quotes. Relaxed JSON like single-quoted text with " inside can fail parse and silently lose a tool call. Escape " when normalizing non-double-quoted strings.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At server/src/server/tool_parser.cpp, line 248:

<comment>String rewrite misses escaping inner double quotes. Relaxed JSON like single-quoted text with `"` inside can fail parse and silently lose a tool call. Escape `"` when normalizing non-double-quoted strings.</comment>

<file context>
@@ -161,6 +168,134 @@ static const std::regex & re_tool_code() {
+        if (in_str) {
+            // Inside a string we already opened. Mirror escapes verbatim.
+            if (c == '\\' && i + 1 < payload.size()) {
+                rewritten += c;
+                rewritten += payload[i + 1];
+                i += 2;
</file context>

Suggested change

	rewritten += c;
	if (in_str != '"' && c == '"') {
	rewritten += "\\\"";
	} else {
	rewritten += c;
	}

[cubic-dev-ai]

P2: Pre-check blocks digit-start verbs. Parser accepts them, so valid plain-text calls can be skipped. Allow digits in first verb char.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At server/src/server/sse_emitter.cpp, line 49:

<comment>Pre-check blocks digit-start verbs. Parser accepts them, so valid plain-text calls can be skipped. Allow digits in first verb char.</comment>

<file context>
@@ -23,6 +24,47 @@ static bool has_request_tools(const json & tools) {
+    while ((pos = text.find("call:", pos)) != std::string::npos) {
+        size_t v = pos + 5;  // step past "call:"
+        if (v < text.size() &&
+            (std::isalpha((unsigned char)text[v]) || text[v] == '_')) {
+            size_t w = v;
+            while (w < text.size() &&
</file context>

Suggested change

	(std::isalpha((unsigned char)text[v]) || text[v] == '_')) {
	(std::isalnum((unsigned char)text[v]) || text[v] == '_')) {

[cubic-dev-ai]

P2: Stripping accumulated_content_ here can desync Responses stream. Deltas already sent raw call text, but done/completed now send stripped text.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At server/src/server/sse_emitter.cpp, line 703:

<comment>Stripping accumulated_content_ here can desync Responses stream. Deltas already sent raw call text, but done/completed now send stripped text.</comment>

<file context>
@@ -608,6 +650,134 @@ std::vector<std::string> SseEmitter::emit_finish(int completion_tokens,
+            // the pre-strip text in earlier deltas; this only affects
+            // the final accumulated_text() consumed by the response
+            // builders in http_server.cpp.
+            accumulated_content_ = parsed.cleaned_text;
+
+            fr = "tool_calls";
</file context>