[rollout] Agentic server roadmap

[guapisolo]

Token-In-Token-Out (TITO) is Miles' token-preserving path for multi-turn agentic rollout: each turn remains a bit-perfect prefix of the next, so training consumes exactly the tokens emitted by the inference engine. This issue tracks model support and the remaining session-server roadmap.

User guide: Agentic Chat Templates (TITO). Blog: No Token Left Behind: Demystifying TITO in Miles.

Model support on main

Select a model family explicitly with --tito-model; Miles does not auto-detect it. Every named family owns one FIXED_TEMPLATE contract for its renderer and required chat-template kwargs. Named families require --use-session-server; use default for a native or custom template.

--tito-model	Example models
qwen3	Qwen3
qwen35	Qwen3.5
qwennext	Qwen3-Thinking-2507, Qwen3-Next-Thinking
glm47	GLM-4.7, GLM-5
nemotron3	NVIDIA Nemotron 3 Super / Ultra
kimi25	Kimi K2.5
kimi26	Kimi K2.6
minimax_m25	MiniMax-M2.5
minimax_m27	MiniMax-M2.7
deepseekv32	DeepSeek-V3.2
deepseekv4	DeepSeek-V4
default	Native or custom templates (best effort)

Rollout status

Landed on main

• Multi-instance session server — one owning instance per session; 16 instances benchmarked at 8.2x the throughput of one.
  • (2/N) feat(session): scale out as N session-server instances on a port range #1659 — multi-instance scale-out
  • (3/N) test(session): session-server overhead benchmark on the multi-instance topology #1660 — overhead benchmark
• General code cleanup

Implementation complete in draft, not on main

• Server-side sample collection and assembly — process records on the owning server and return only assembled samples.
  • (1/3) refactor(session): move sample assembly into the session package #1758 — move sample assembly into the session package
  • (2/2) refactor(session): assemble training samples on the session server; records never leave it #1759 — server-side collection and assembly (draft)

TODO

• Black-box harness support via Session Server v2 — trajectory-tree support for retries, compaction, branches, and per-leaf merging, plus end-to-end coverage.
  • feat(session): session server v2 — opt-in trajectory-tree serving; v1 preserved byte-exact #1771 — Session Server v2 (draft)