fix(sgl-router): mirror the engine's tool_choice default in the K3 render

[yan-lgtm]

Problem

An absent tool_choice is not auto. The engine's protocol layer fills the field in before the chat template ever runs — ChatCompletionRequest.set_tool_choice_default in python/sglang/srt/entrypoints/openai/protocol.py:

if values.get("tool_choice") is None:
    if values.get("tools") is None and not _has_message_level_tools(values.get("messages")):
        values["tool_choice"] = "none"
    else:
        values["tool_choice"] = "auto"

kimi_k3::resolve_render_opts read the field literally, so an omitted tool_choice mapped to ToolChoice::Unset and skipped the preamble that encoding_k3 emits for tool_choice=none:

<|open|>message role="system" type="tool-choice"<|sep|>The system is invoked with `tool_choice=none`.
You MUST NOT call any tools in the next message.<|close|>message<|sep|><|end_of_msg|>

That is 38 tokens, dropped on essentially every plain chat request — clients rarely send tool_choice explicitly. And because the router forwards its ids as input_ids, the shortened prompt is what the engine actually generated from, so the model never saw "You MUST NOT call any tools in the next message."

Evidence

Measured on a live two-replica Kimi-K3 endpoint, identical messages, engine-direct (localhost:30000) vs through the router:

request	engine	router
omits tool_choice	142	104	diverges
tool_choice: "none"	142	142	match
tool_choice: "auto"	104	104	match

Decoding return_prompt_token_ids from both paths showed the tool-choice block as the entire difference. The router's 107 raw ids were byte-identical to the reference apply_chat_template output — the encoder is faithful to encoding_k3; what it did not mirror was the protocol layer's default.

A second effect worth noting: this split the prefix cache by modality. Text-only turns are rendered by the router (no preamble) while image-carrying turns deliberately fall through to the engine (preamble present), so the same conversation hashed to two different prefixes across an image boundary — which undercuts cache-aware routing precisely where a long multimodal conversation should benefit most.

Fix

resolve_render_opts now distinguishes an absent/null tool_choice from a present one and resolves the absent case through a new declares_tools helper mirroring the engine predicate. Two details of the Python are load-bearing and pinned by tests:

• Top-level tools is tested by is None, not truthiness — an explicit tools: [] is not None in Python, so it still yields auto. Testing emptiness would emit a block the engine never emits.
• Message-level tools count only on system / developer roles, and there by truthiness (bool(msg.get("tools"))), so tools: [] on a system message does not count.

An explicit tool_choice still wins in both directions; required, none, auto, and a named-function object are unchanged.

Why the existing tests missed it

The fixture-driven parity tests build RenderOpts from FixtureOpts::to_render_opts, which mirrors the reference encoder, and the reference never sees the protocol layer's defaults. No fixture can express "the field was absent and the server filled it in." Pinned with a request-level test instead: absent_tool_choice_mirrors_the_engine_protocol_default.

Testing

• cargo test --lib — 854 passed, 0 failed, 1 ignored. Includes all three reference-encoder parity tests (segment_parity_with_reference_encoder, render_parity_with_reference_encoder, rendered_text_parity_with_reference_encoder) and the new test.
• cargo clippy --lib --all-features -- -D warnings — clean.
• cargo fmt --check — clean.

Not done: the router image was not rebuilt and re-measured against the live endpoint. The correctness chain is the live measurement above plus the unit test — an explicit tool_choice: "none" already rendered router=142=engine, and the new code maps absent-plus-no-tools onto that same ToolChoice::None.

🤖 Generated with Claude Code

---

CI States

Latest PR Test (Base): ⚠️ Run #30671627874
Latest PR Test (Extra): ⚠️ Run #30671627725