[R3]: Move to new vLLM routed experts format

[S1ro1]

PR is ready - verified with verifiers/renderers (we need to pin to main), however waiting: vllm-project/vllm#39568 to be included in vLLM release - expected 0.21.1

• expose choices[i].routed_experts from the prime-rl vLLM token path as compact raw-uint8 JSON payloads: {"data": base64(raw_bytes), "shape": [...]}
• stitch P/D routed experts by joining RequestOutput.prompt_routed_experts with per-completion decode experts before serializing
• keep routed-experts data opaque through verifiers during token truncation; prime-rl decodes at the orchestrator boundary
• preserve this branch's routed-experts source of truth: RoutedExperts(data, shape, dtype), explicit dtype maps, and _pack_routed_experts / _unpack_routed_experts for multi-turn stitching
• update trainer packing/loading to slice, append, pad, and reconstruct the RoutedExperts transport struct with torch.frombuffer
• pin vllm-router to 0.1.25 for the matching raw-uint8 schema and add pybase64
• pin deps/verifiers to the cleaned production-equivalent routed-experts response path from verifiers PR [Feat] Multi-lora layer, data packing, optimizer, broadcast and scheduler #1433
• disable vLLM async scheduling only for the NIXL routed-experts capture path, where async scheduling leaves placeholder sampled-token state during capture
• fix rendered multi-node orchestrator args to use the student client config keys

Related PRs

• Router payload format: Feat: downstream perf improvements - bump .25 router#34
• Router release trigger: release: v0.1.25 router#35
• Verifiers payload format: [Router Replay]: Improve performance by removing Pydantic validation verifiers#1394
• Verifiers cleanup against main: Clean routed experts response path verifiers#1433

Verification

• uv sync --all-extras
• uv run ruff check src/prime_rl/inference/patches.py src/prime_rl/inference/vllm/routed_experts.py
• bash -n /beegfs/outputs/qwen3-30b-a3b-router-replay-diag-r3-v3-clean/rl.sbatch
• 3-node Qwen3 30B A3B routed-experts validation, Slurm job 19354, output /beegfs/outputs/qwen3-30b-a3b-router-replay-diag-r3-v3-clean, node 53 excluded. Orchestrator completed 5/5 steps using the direct renderer rollout client; trainer completed steps 0-3 with finite grad norms and began step 4 before the generated script terminated remaining processes after orchestrator completion.
• Log scan for Failed to merge, Rollout error, Aborted rollout, ERROR, Traceback, and Exception under the validation output logs returned no matches.

---

Note

Medium Risk
Medium risk because it changes the on-wire/transport representation of routed_experts end-to-end (inference responses, orchestrator parsing, batching/packing, and trainer tensor reconstruction), and adds a vLLM VllmConfig.__post_init__ monkey patch that could affect inference startup/validation in disaggregated setups.

Overview
Updates routed-experts handling end-to-end to a new compact format. The vLLM /inference/v1/generate path now captures per-choice routed_experts and serializes it as {"data": base64(raw uint8 bytes), "shape": [...]} (via new inference/vllm/routed_experts.py), replacing the prior numpy/list-style encoding.

Propagates the new representation through the training pipeline. The orchestrator decodes the compact payload to numpy for step stitching, then packs it into a new transport.types.RoutedExperts struct (raw bytes + shape + dtype) carried by TrainingSample/MicroBatch; trainer batching now slices/appends/pads this byte payload and reconstructs tensors with torch.frombuffer.

Adds safety and compatibility guardrails. RL config validation now rejects router replay when inference.kv_cache_offload is enabled, and inference installs a vLLM monkey patch to allow routed-experts capture when using the NIXL KV connector (while still rejecting unsupported PP/v2 runner cases). Dependencies are updated to add pybase64 and bump vllm-router to 0.1.25, with tests updated accordingly.

^{Reviewed by Cursor Bugbot for commit c13b0b3. Bugbot is set up for automated code reviews on this repo. Configure here.}

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit aa1fc36. Configure here.}