[Perf] Benchmark Miles vs VERL training throughput across FSDP and Megatron backends

[vincentzed]

Summary

Propose a controlled, reproducible throughput benchmark of Miles vs VERL, run as a backend matrix {Miles, VERL} × {FSDP, Megatron}, so we have an apples-to-apples number instead of the scattered, hard-to-interpret reports inherited from slime.

Today Miles has no published head-to-head throughput comparison with VERL. The only data points live upstream in slime, and they (a) are mostly Megatron-backend or rollout-side, not FSDP-isolated, and (b) conflate the training backend with rollout/colocate effects, which makes them hard to act on.

Why now / prior art (upstream slime)

Miles is a slime fork, so these are the closest existing signals:

• slime#1072 — "Slime RL Training is 4× Slower Than VERL on the Same Model" — Qwen3-1.7B-Base, 8×H800, Megatron TP=2, same dataset/rollout-size as VERL. Reporter sees perf/wait_time_ratio ≈ 0.5 and high perf/rollout_time. Notably the slowdown is attributed to disaggregated placement (rollout engine idle while the actor computes); a later comment reports colocate mode on B300 at ~170s/step (rollout 120s). So the headline "4× slower" is not a training-backend result — it's placement + rollout.
• slime#385 — experiment — Qwen3-0.6B, 2×H200, colocate: slime rollout_time 245s / step 287s vs VERL generation ~9s / step ~33s, with GPU util observed at 0% during rollout. Suspected long-tail / config (non-power-of-2 batch sizes) rather than compute.
• slime#199 — "slime vs verl on moe model" — slime stuck in /health checks while VERL is already generating; right-tailed long generations; partial rollout suggested as mitigation.
• slime#114 — "comparison with other frameworks" — explicit ask for a VERL comparison; answer was "async should outperform VERL but it's untested." Still untested.

Takeaway: the existing evidence is dominated by rollout time, wait-time, colocate-vs-disaggregated, and long-tail generation — not by the FSDP/Megatron training step. A clean benchmark needs to isolate the training compute from the rollout pipeline.

We already have the instrumentation

Miles inherits slime's perf metrics, so the numbers needed for this comparison are already emitted:

• miles/utils/train_metric_utils.py → perf/step_time, perf/wait_time_ratio, perf/actor_train_tflops (3 × fwd_flops / actor_train_time), perf/actor_train_tok_per_s, perf/log_probs_tflops, perf/ref_log_probs_tflops.
• miles/ray/rollout/metrics.py → perf/rollout_time, perf/*tokens_per_gpu_per_sec, perf/longest_*sample_tokens_per_sec, plus a generation-time breakdown.
• In flight: [feat] add performance monitoring metrics for training and rollout #1020 (add performance monitoring metrics for training and rollout, adds prometheus_utils.py) and perf: optimize micro-batch calculation #403 (optimize micro-batch calculation).

VERL equivalents for the comparison: timing_s/step, timing_s/generation_timing/*.

We can do both backends (incl. Megatron)

Miles ships two training backends, selected by --training-backend {megatron, fsdp} (miles/utils/arguments.py:131):

• miles/backends/megatron_utils/ — primary/production backend.
• miles/backends/experimental/fsdp_utils/ — FSDP (currently under experimental/).

scripts/run_mcore_fsdp.py already parameterizes train_backend: Literal["fsdp", "megatron"] with colocate=True and a fixed model/config — i.e. we can hold the dataset, rollout size, GPU layout, and placement constant and flip only the training backend. VERL likewise supports both FSDP and Megatron, so the matrix is well-defined. #1486 (fsdp(scripts): RL scripts for representative dense + MoE models) adds the per-model FSDP RL launchers that can serve as the harness.

Proposed benchmark

Hold everything constant; vary only framework × backend.

• Matrix: {Miles, VERL} × {FSDP, Megatron}
• Models: one small dense (Qwen3-1.7B-Base or Qwen3-4B, matching slime#1072) + one MoE (Qwen3-30B-A3B, matching slime#199)
• Fixed: DAPO-math-17k, GRPO, rollout 128×32, seq len (8k or 16k), same GPU count (e.g. 8×H100/H800), colocate (and separately disaggregated, since that's where slime#1072 regressed)
• Report (per step, steady state):
  • perf/step_time vs VERL timing_s/step
  • perf/rollout_time vs VERL timing_s/generation_timing/*
  • perf/actor_train_tflops / MFU and perf/actor_train_tok_per_s (isolates the training step from rollout)
  • perf/wait_time_ratio (placement / pipeline-bubble cost)
  • rollout tokens_per_gpu_per_sec

Separating training-step throughput (TFLOPs/MFU, tok/s) from rollout/wait gives us an actionable answer: is any gap in the FSDP/Megatron training compute, or in the rollout + scheduling path (as the slime reports suggest)?

Asks

1. Do maintainers already have internal Miles-vs-VERL numbers (FSDP and/or Megatron) that could be published?
2. Is there interest in landing this as a standard benchmark under scripts/ (building on fsdp(scripts): RL scripts for representative dense + MoE models #1486 / run_mcore_fsdp.py)?
3. Any known config that should be the recommended default for the small-dense case to avoid the slime#1072/update code #385 rollout-idle pathology (e.g. colocate + dynamic batch size + partial rollout)?

Happy to contribute the harness/scripts if there's interest.

---

Filed after searching radixark/miles (issues + PRs + comments) and THUDM/slime; no existing Miles issue covers a Miles-vs-VERL throughput comparison.