feat(frontends): add the llm-d router

[cquil11]

• Add frontend.type: llm-d: Envoy and the Endpoint Picker (EPP) for aggregated and prefill/decode vLLM jobs.
• Generate worker discovery and router configs after workers pass health checks; gate readiness on Envoy and EPP.
• Run a pd-sidecar per decode worker, with allocated proxy ports and NIXL KV transfer.
• Add frontend.epp_config, docs/llm-d.md, and aggregated/disaggregated/DP-rank examples.
• Route to vLLM DP ranks: data-parallel-external-lb: true launches every rank as its own vllm serve --data-parallel-rank on its own port, one EPP endpoint (and decode sidecar) per rank, on one or several nodes.
• Route by KV-cache events: roles.<role>.kv_events connects every worker's (or rank's) vLLM publisher to the EPP's precise-prefix-cache-producer on one socket srtctl owns, with topics naming each endpoint, and points the token-producer at a worker's render endpoint.
• Without frontend.epp_config, run the scorers of llm-d's no-Kubernetes guide (prefill and decode profiles for P/D); the EPP's own default never applied, since srtctl's file always carries discovery.
• Map a decode MultiConnector (NIXL next to OffloadingConnector CPU offloading) to the nixlv2 sidecar protocol.
• Scope: vLLM only, one router replica, static discovery (a failed worker or sidecar ends the job). KV events use the EPP's global socket: v0.11 file discovery has no per-endpoint rankIndex, so per-pod subscription and vLLM's multi-port DP supervisor are not used.
• Reuse the SMG frontend's worker readiness gate. Follow llm-d-router v0.11.0's file-discovery API.

flowchart TD
    Client --> Envoy
    Envoy <-->|Endpoint selection<br/>over gRPC| EPP
    Envoy -->|Aggregated request| Agg["vLLM aggregate worker"]
    Envoy -->|Disaggregated request| Sidecar["Decode P/D sidecar"]
    Sidecar -->|Run prefill| Prefill["vLLM prefill worker"]
    Sidecar -->|Decode with<br/>transfer parameters| Decode["vLLM decode worker"]
    Prefill -.->|KV cache via NIXL| Decode

Loading

Validation

AgentX through InferenceX #3741, with image quay.io/rh-ee-imarkov/llm-d-nokube-vllm:dspark-0814-nightly (EPP and sidecar v0.10.0). MiniMax-M3 NVFP4 at concurrency 8 unless noted. The first four rows ran cd129c2e; the rest ran 0c209710, the same commits before the rebase onto upstream main.

Scenario	Hardware	Run
DeepSeek-V4-Pro 1P1D, DEP8 prefill / DEP8 decode, NIXL, c64	GB200	37396765122
DeepSeek-V4-Pro aggregated, DEP8, c16	GB200	37396830944
MiniMax-M3 1P1D, TP4 / TP4, NIXL, c8	GB300	37396770618
MiniMax-M3 aggregated, 2x TP4, c8	GB300	37396773335
Precise prefix, aggregated 2x TP4	GB300	37553005166
Precise prefix, 1P1D TP4 / TP4	GB300	37553008202
DP ranks, one node, DP2 x TP2, precise prefix	GB300	37553011355
DP ranks, two nodes, DP4 x TP2, precise prefix	GB300	37553014497
DP ranks, 1P1D DP2 x TP2 each, precise prefix	GB300	37553017562
Precise prefix over CPU offload, 1P1D TP4, NIXL + OffloadingConnector in a MultiConnector	GB300	37553020113
No epp_config (guide scorers), aggregated 2x TP4	GB300	37553023865
Baseline rerun, aggregated 2x TP4	GB300	37553026821

[cursor]

Stale Bugbot comment from a previous run.

[github-actions]

src/srtctl diff: +1066 -370

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

There are 3 total unresolved issues (including 2 from previous reviews).

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 96fae46. Configure here.}

[cquil11]

FYI:
This diff is just general cleanup as this is mainly duplicated logic
For instance _get_int_flag functionality is achieved via _get_parallel_size