Summary
With fault tolerance enabled, a replacement for an updatable SGLang rollout engine is added to the router before the current actor weights are transferred to it. During that interval, the router can dispatch generation requests to an engine that is still serving the configured checkpoint weights.
Current sequence
On current main (ab97f759969c5288685887ea9dd6a7455251159c):
RayTrainGroup.update_weights() calls recover_updatable_engines() before broadcasting update_weights.ServerGroup.recover() starts and initializes each replacement.- Engine initialization immediately registers the replacement with the router.
- Recovery returns, and only then does the actor-side
update_weights broadcast run.
A continuously active rollout producer can therefore send a request through the router between steps 3 and 4. The replacement initially serves hf_checkpoint; before the first Miles weight update, SGLang reports its initial weight version as "default" rather than the numeric actor version.
Impact
- A trajectory may contain tokens generated from stale checkpoint weights while being treated as a current-policy rollout.
- Consumers that require numeric actor weight versions may fail when they receive
"default".
- The window grows if weight transfer is delayed or fails after recovery.
Expected behavior
Automatically recovered engines whose model receives actor weight updates should remain unavailable to routed generation traffic until the current actor weight update has completed successfully. If weight transfer fails, the replacement should remain unavailable.
Initial startup, explicitly started cells, and model servers configured not to receive actor weight updates do not necessarily need this gate.
As a diagnostic guardrail, observing "default" from an updatable engine after startup should fail loudly because it indicates that an unsynchronized engine became routable; silently requeueing the response would hide the invariant violation.
Suggested direction
Separate engine initialization from router registration:
- Start automatically recovered updatable engines without registering them.
- Perform the actor-to-engine weight update.
- Register the replacements only after the complete update broadcast succeeds.
- Make registration idempotent so a failed registration can be retried without exposing an engine before synchronization.
This avoids canceling in-flight trajectories on healthy engines and does not require pausing the rollout producer.
Regression test
A focused test can stop an updatable engine slot, invoke RayTrainGroup.update_weights(), and assert this ordering:
- Replacement initialization completes.
- No router registration occurs yet.
- Actor weight update completes for the replacement.
- Router registration occurs.
The failure-path variant should assert that an exception during weight transfer leaves the replacement unregistered.
Summary
With fault tolerance enabled, a replacement for an updatable SGLang rollout engine is added to the router before the current actor weights are transferred to it. During that interval, the router can dispatch generation requests to an engine that is still serving the configured checkpoint weights.
Current sequence
On current
main(ab97f759969c5288685887ea9dd6a7455251159c):RayTrainGroup.update_weights()callsrecover_updatable_engines()before broadcastingupdate_weights.ServerGroup.recover()starts and initializes each replacement.update_weightsbroadcast run.A continuously active rollout producer can therefore send a request through the router between steps 3 and 4. The replacement initially serves
hf_checkpoint; before the first Miles weight update, SGLang reports its initial weight version as"default"rather than the numeric actor version.Impact
"default".Expected behavior
Automatically recovered engines whose model receives actor weight updates should remain unavailable to routed generation traffic until the current actor weight update has completed successfully. If weight transfer fails, the replacement should remain unavailable.
Initial startup, explicitly started cells, and model servers configured not to receive actor weight updates do not necessarily need this gate.
As a diagnostic guardrail, observing
"default"from an updatable engine after startup should fail loudly because it indicates that an unsynchronized engine became routable; silently requeueing the response would hide the invariant violation.Suggested direction
Separate engine initialization from router registration:
This avoids canceling in-flight trajectories on healthy engines and does not require pausing the rollout producer.
Regression test
A focused test can stop an updatable engine slot, invoke
RayTrainGroup.update_weights(), and assert this ordering:The failure-path variant should assert that an exception during weight transfer leaves the replacement unregistered.