feat(tilert): add decode backend and native router

[cquil11]

Adds the TileRT decode backend and native P/D router on top of #33.

The supported pairing is vLLM prefill + TileRT decode + TileRT router. TileRTRouterFrontend.validate enforces the engine pairing and topology during configuration loading, before Slurm submission. Other routers reject TileRT workers through their engine requirements; per-role engines with Dynamo are rejected in #33.

• Explicit role engines and images; no top-level engine.
• Allocated worker addresses and ports, with worker health checked before router startup.
• B200/NIXL and MI355X/Mooncake recipes and launch-contract regression tests.

The vLLM image must contain TileRTConnector. Validation does not inspect installed connector versions or prove KV-layout compatibility. Supporting another prefill engine requires a compatible TileRT protocol integration and extending this frontend validation; the shared role dispatch remains engine-neutral.

Stack 2/2. Shared role orchestration is in #33.

Validation through the InferenceX ports:

• B200 / NIXL — 1k1k and 8k1k passed, 16/16 requests each; 8k1k power validated across both nodes.
• MI355X / Mooncake — full 3,600-second AgentX profile passed, 271 successful requests and zero errors; required prefill metrics validated.

Ports: shared integration #3551, MI355X #3552, B200 #3553.

---

Note

Medium Risk
Introduces a new mixed-engine launch path (vLLM + TileRT) with KV transport coupling that validation does not fully prove; benchmark metrics/env behavior changed for custom AIPerf jobs on logical workers.

Overview
Adds TileRT as a first-class decode engine and tilert-router as a static P/D frontend for disaggregated recipes where vLLM prefill hands off KV via TileRTConnector and TileRT decode runs tilert.pd_vllm.decode_server. srtctl owns HTTP/control ports and router wiring (--vllm-url, combined --decode specs); config validation enforces one prefill, ≥1 single-node decode workers, no multi-frontend nginx, and rejects incompatible engine/router pairings at load time.

Documentation, schema docs, and B200/NIXL and MI355X/Mooncake example recipes ship with the integration. Custom-benchmark AIPerf metrics discovery now asks the frontend for worker_metrics_port per process (TileRT exposes prefill only), while SRT_*_ENDPOINTS still lists all routable workers; vLLM-router multi-node DP metrics behavior is covered by a regression test.

^{Reviewed by Cursor Bugbot for commit b421420. Bugbot is set up for automated code reviews on this repo. Configure here.}

[cursor]

Stale Bugbot comment from a previous run.

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 2 potential issues.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit f2b1e91. Configure here.}

[cursor]

Router setup misses patches fallback

Medium Severity

build_bash_preamble runs only /configs/<setup_script> and treats a missing file as a hard router-start failure. Workers, Dynamo, and the vLLM router also try /configs/patches/<setup_script> and continue with a warning when the script is absent. A recipe whose setup lives in the patches mount, or exists only in worker images, will launch workers and then fail when tilert-router starts.

 

^{Reviewed by Cursor Bugbot for commit f2b1e91. Configure here.}

[cursor]

DP prefill evades router validation

Low Severity

validate only checks resources.num_prefill == 1, but build_router_command rejects any collected worker list that is not exactly one prefill HTTP URL. vLLM per_node data parallel exposes one API per node-local pool (http_port > 0), so a single logical prefill worker on multiple nodes passes configuration loading and then fails when the router is built.

Additional Locations (1)

• src/srtctl/frontends/tilert_router.py#L93-L98

 

^{Reviewed by Cursor Bugbot for commit f2b1e91. Configure here.}

[cquil11]

Closing because the TileRT decode backend and native router were merged upstream in NVIDIA#529.