Repository navigation
Conversation
Rubiks/Wuji play scenes use the base ManagerBasedEnvCfg but are built via ManagerBasedRlEnv, which reads cfg.device — crashing with AttributeError because device was defined only on the RL subclass ManagerBasedRlEnvCfg. Move device: str = "cuda" onto the base cfg so every env cfg carries it; drop the now-redundant subclass duplicate. Reference audit confirmed the only cfg.device reads are in ManagerBasedRlEnv.__init__; all other .device access is on the runtime env.device property. Add headless regression tests and update the config reference docs.
fix(envs): move device cfg field onto base ManagerBasedEnvCfg
P8/P9: `genelab play GeneLab-Rubiks-Play-v0 --agent zero --steps 5` (and the Wuji equivalent) crashed in env construction — first on `cfg.device` (fixed superficially in #153), then on `cfg.decimation`, etc. Root cause is architectural, not a missing field. Rubiks/Wuji are force-driven scene-playback demos whose cfgs subclass the non-RL base ManagerBasedEnvCfg and have their own RegisteredTask.play() → RubiksPlayEnv/WujiHandPlaybackEnv.play() that builds a Genesis scene directly (no managers, no policy). `_dispatch_play` already routes to task.play() when no RL flags are present, so the bare command worked — but RL-only flags (--agent/--checkpoint/--num-envs/--prof) forced the fall-through to play_task → build_env → ManagerBasedRlEnv, which requires the full RL cfg surface these demos don't have. Bolting more fields onto the base cfg would only move the crash to the next field. Fix at the routing layer: gate RL routing on whether the play env cfg is a ManagerBasedRlEnvCfg. Non-RL tasks always run their own .play(); RL-only options are meaningless for a fixed scene demo, so they are warned-and-ignored rather than crashing. --steps/--vis/--gpu and dotted overrides still apply (they are resolved into the cfg before dispatch). Defense-in-depth: build_env now raises a clear TypeError naming the required surface instead of a cryptic AttributeError. Tests: - test_examples_{rubiks,wuji}: exercise the real `genelab play … --agent zero --steps 5` dispatch path with Genesis stubbed (headless), asserting the scene demo runs, honors --steps, and is NOT routed through play_task. These guard the actual P8/P9 bug, which the prior #153 cfg-only tests did not. - test_cli_routing: non-RL task + --agent runs the demo with a warning; build_env rejects a non-RL cfg. Added an RL-capable fake task so the existing agent-routing tests still validate the RL path (they previously asserted the buggy contract on a non-RL cfg). Docs: clarify in cli/play-train.{en,zh} that policy options apply only to RL tasks; scene demos run built-in playback.
Wrap the External-Fake-RL-Task-v0 play args list to satisfy ruff format --check.
fix(cli): route non-RL scene demos away from the RL play helper (P8/P9)
…ps NaN-ing The Curriculum-Showcase G1 spawned ~0.77 m buried in the RandomRough heightfield: terrain_levels_vel (run on every reset) and the runner teleport wrote the base to the cell origin at z=0 (= terrain.pos[2]), ignoring the surface elevation and the robot's 0.76 m standing height. Feet ended at z=-0.72 under a +0.05 m surface; that interpenetration blew up the rigid contact solver — instant NaN at dt=0.005, an unconverging multi-minute grind at dt=0.002. dt was a red herring. - terrains/importer: add surface_height_at(xy) heightfield lookup. - mdp/curriculums: terrain_levels_vel reseats the base at surface + init_pos.z + spawn_clearance instead of the bare cell origin. - showcase runner: teleport reseats at standing height too, so the full 400-step demo survives its teleports. - showcase cfg: dt 0.005->0.002, decimation 4->10. The G1 is independently NaN-unstable at dt=0.005 even from a clean stance; this matches the known-good Velocity/Tracking-Flat configs. Control rate stays 50 Hz and the 2 s episode (100 control steps) is unchanged. - rl/runner: play_task caps scripted (zero/random) playback at simulation.steps, so `play TASK --agent zero --steps N` stops after N steps instead of looping forever headless (--steps was silently ignored on the RL play path). Trained playback stays unbounded. Verify: `play GeneLab-Curriculum-Showcase-v0 --agent zero --steps 5` exits 0, no NaN; full 400-step showcase exits 0 with curriculum promotion (levels 0->4). pytest: 507 passed.
fix(showcase): seat G1 on terrain surface so the Curriculum-Showcase stops NaN-ing
`genelab export <trainable-task>` builds the play env via
`resolve_env_cfg(task_id, play=True)`, and trainable tasks set
`simulation.vis=play` so their play_env opens the viewer. Export only probes
obs shapes — it never renders — so on a display-less CI / server building that
env raised `GenesisException: No display detected`, with no CLI override to
disable it.
Force `env_cfg.simulation.vis = False` right after resolving, mirroring what
`eval_task` already does ("eval is always headless"). The `episode_length_s`
clamp eval applies is not needed here: export collects no episodes.
Add a regression test that drives the real `export_task` with env build
monkeypatched out (no Genesis runtime) and asserts the cfg handed to
`build_env` has the viewer off and the .ts + .metadata.json are produced.
fix(export): force vis=False so export runs headless
…gain `genelab export <rsl_rl-task>` failed for every rsl_rl checkpoint with "backend 'rsl_rl' did not expose an actor module": `_extract_rsl_rl_actor` read `runner.alg.actor_critic`, but current rsl_rl's `PPO.__init__` stores `self.actor` / `self._raw_actor` directly and no longer has `actor_critic`, so extraction returned `(None, num_obs)` and `_export.py` raised. Source the actor from `alg._raw_actor` (falling back to `alg.actor`) and use the actor's `as_jit()` export wrapper, which exposes the flat `forward(obs) -> deterministic action` contract (learned obs normalizer baked in) that the exporter wraps with env-side scale/clip. The legacy `alg.actor_critic.actor` / `act_inference` path is kept as a fallback for older rsl_rl. Verified headless (QD_GRAPH=0 QT_QPA_PLATFORM=offscreen) against a real inverted-pendulum checkpoint: `-f torchscript` and `-f onnx` both exit 0 and write loadable .ts / .onnx + .metadata.json (obs_dim 8 -> action_dim 1, dynamic batch). Tests build a real rsl_rl MLPModel and assert extraction + full export, plus the legacy and missing-actor paths. Docs (eval-and-export EN/ZH) updated to describe the new extraction path.
fix(export): extract rsl_rl actor from alg directly so export works again
`genelab export <sb3 SAC+HER task>` (e.g. GeneLab-Franka-Pick-And-Place-v0) failed with `AssertionError: Expected dict, got <class 'torch.Tensor'>`: goal-conditioned tasks have a gymnasium Dict observation (observation / achieved_goal / desired_goal) and the SAC actor's CombinedExtractor consumes all keys, but the exporter traced the actor with a single flat tensor that SB3's preprocess_obs rejects. Fix (single flat input, reconstruct the Dict inside the actor): - exporter: generalize the single-group resolver to `_resolve_obs_specs(env, groups)` with a global cursor across groups; `export_policy` gains `goal_groups`, concatenating [policy, achieved_goal, desired_goal] into one flat `obs`. Metadata now records `obs_dim` + each group's `start` offset. - sb3 backend: `_extract_sb3_actor` returns `_Sb3DictActor` for Dict spaces — it splits the flat tensor (in observation/achieved_goal/desired_goal order) back into the Dict and calls `policy._predict(dict, deterministic=True)`. - cli: `export_task` passes the HER goal groups from `agent_cfg.her`. - exporter ONNX: use `dynamo=False` (legacy tracer) — the torch.export-based default (torch>=2.9) can't trace SAC's `Normal` construction; the legacy tracer matches the torchscript path and handles every backend. Verified locally (no Genesis): a synthetic SAC MultiInputPolicy exports to both .ts and .onnx with loadable, runnable artifacts and correct multi-group metadata; the extracted actor matches `policy._predict` on the equivalent Dict. Docs (eval-and-export EN/ZH) updated: sb3 extraction, the flat HER layout, the new metadata schema, and removal of the stale "Dict obs unsupported" limitation.
fix(export): support SB3 SAC+HER Dict-obs export via flat concat
Trainable tasks set `vis=play` in their play_env, so `play --agent trained` opens the Genesis viewer by default and aborts with "No display detected" on a display-less server. `--headless` is the inverse of `-v/--vis`: it rewrites to `env.simulation.vis=false`. The two are mutually exclusive (raises SystemExit). Mirrors the existing `-v/--vis` shortcut in `parse_run_args`. Updates help text + snapshots and the CLI docs (en/zh) with a headless-server playback note.
feat(cli): add --headless flag to force vis=false
The skrl backend was implemented and unit-tested but no registered task used
it, so `genelab train/play/eval/export` never exercised skrl at runtime. Add
`GeneLab-Inverted-Pendulum-Skrl-v0`: it reuses the single inverted-pendulum
env/robot but supplies a `SkrlAgentCfg` (skrl PPO) instead of the rsl_rl runner
cfg — and since the backend is chosen purely by agent-cfg type, that routes the
task to the skrl backend.
- examples: add `inverted_pendulum_skrl_agent_cfg()` (mirrors the rsl_rl PPO
recipe; obs_group=policy / state_group=critic, which the env already emits)
and `InvertedPendulumSkrlTask` registering the new id. `SkrlAgentCfg` is a
pure dataclass, so registration/listing never imports skrl — only training
does.
- backend: `SkrlBackend.train()` now writes a final checkpoint after
`trainer.train()`. skrl only checkpoints every `checkpoint_interval` steps
(`"auto"` -> `timesteps // 10`, == 0 for a smoke `--max-iterations 2`), so
short runs otherwise finished with nothing to eval/play. Mirrors rsl_rl's
unconditional final `model_{it}.pt` save; benefits every skrl task.
- tests: registration + agent-cfg-type asserts (no skrl needed), plus an
skrl-guarded train->checkpoint->eval round-trip on a fake env. Verified the
optional-dep guard: with skrl blocked the registration tests pass and the
round-trip skips.
- docs: list the task and its skrl train/eval commands (EN + ZH).
feat(examples): register skrl-backed inverted-pendulum task
ForceTorqueSensor / ForceTorqueSensorCfg were implemented and unit-tested but no registered task instantiated one, so it was never exercised at runtime. Wire it into the play-only Sensors-Showcase (Franka) — the lowest-blast-radius route: it adds a sensor to the scene only, touching no trainable task's obs/reward (T1–T5 reproduction unaffected). - sensors/env_cfg.py: add `ForceTorqueSensorCfg(name="arm_joint_ft", joint_names_expr=r"^joint[1-7]$")` to the scene — the 7 actuated arm joints the scripted joint1 sweep drives. `data.force` is (num_envs, 7). The env builds and updates every scene sensor each step, so this alone exercises it during a play rollout. - sensors/runner.py: read + log the joint force/torque in `_dump`, mirroring the camera/IMU/frame-transformer evidence dump. - tasks.py: extend the Sensors-Showcase description (and fix a stale "six"->"seven" task count). - tests/test_showcase_sensors.py: assert the showcase env cfg registers the FT sensor and that, bound against a Franka-style fake env (no Genesis), it resolves exactly the 7 arm joints and yields (num_envs, 7) force data. Mirrors tests/test_sensor.py's fake-env pattern. - docs: note joint force/torque in the showcase capabilities table (EN + ZH).
feat(showcase): exercise ForceTorqueSensor in the Sensors-Showcase
MlpResidualActuator / MlpResidualActuatorCfg were implemented and unit-tested but no robot/task used one, so the residual code path never ran at runtime. Add a play-only showcase task `GeneLab-MlpResidual-Actuator-Showcase-v0` that drives the Franka arm with an MlpResidualActuator (DC-motor torque base + a learned residual) WITH a real network_file, so the MLP-residual path executes (not the DC-motor fallback). Play-only, so no trainable task's obs/reward is touched. - actuators/residual_net.py: `residual_net_path()` materializes a tiny, deterministic TorchScript Linear(2,1) (residual = -0.05*joint_vel) under CACHE_DIR and returns its path — no committed binary. Always rewrites, so changing the weights can't leave a stale cached net. - actuators/env_cfg.py: `mlp_residual_actuator_showcase_env_cfg()` reuses the actuator showcase and swaps the arm group to MlpResidualActuatorCfg(network_file=..., residual_scale=1.0). The residual is tiny vs the 87 Nm budget, so it can't destabilize the demo. - actuators/runner.py: MlpResidualActuatorShowcaseRunner with a distinct log slug. - tasks.py: register the new task (eight showcase tasks now). - tests/test_showcase_mlp_residual.py: assert the cfg wires the residual (network_file set, residual_scale>0), the generated net round-trips, and the loaded residual actually changes the actuator torque vs the network_file=None fallback at an unsaturated operating point. Mirrors tests/test_actuator.py; no Genesis needed. - docs: actuator concept doc gets a runnable-example pointer; showcase capabilities table notes the MLP-residual variant (EN + ZH).
…wcase feat(showcase): exercise MlpResidualActuator in a new actuator showcase
…ext API) `collect_demos` aborted with `AttributeError: 'ManagerBasedRlEnv' object has no attribute 'robot_state'` — `demo_fsm.py` still used the pre-vNext `env.robot_state.*` accessor removed by the Articulation refactor (ADR-0019). This blocked Franka SAC+HER reproduction: the prefill .npz was never produced, so training failed with FileNotFoundError on GENELAB_SB3_DEMO_PATH. Route both reads through the already-cached `self._art = env.articulations["robot"]`, mirroring the task's own mdp.py: - `_ee_pos`: `self._art.data.link_pos[:, self._hand_idx, :]` - `_gripper_width`: `self._art.data.joint_pos.index_select(1, self._finger_joint_idx)...` `env.robot_state` was the prior name for the primary robot's state object, so this is behavior-preserving (same tensors, same joint/link indexing). Refactor `collect_demos.main()` into a pure `collect(env, fsm, steps) -> dict` plus `save_demos(out, path)` (env construction + CLI stay in `main`), so the collection pipeline is testable without a Genesis runtime. Behavior of `python -m ...collect_demos` is unchanged. Add `tests/test_franka_collect_demos.py`: drives the real `FrankaPickAndPlaceFsm` + `collect()` against a fake env exposing the current `articulations[...].data.*` API, then round-trips the .npz and asserts the ten HER-prefill keys + shapes. `step()` exercises `_ee_pos`; `_gripper_width` (not used by step) is asserted directly. Before the fix these raised the `robot_state` AttributeError.
fix(franka): route demo FSM state through articulations[...].data (vNext API)
`_download_to` called `urlopen` with no timeout, so on a host with no network (or one that needs an http_proxy) asset fetches hung forever — taking `genelab list/train` (which resolve asset-zoo robots) down with them. Pass a per-socket-op timeout (default 30s, override via GENELAB_DOWNLOAD_TIMEOUT; 0/neg disables) to `urlopen`, applying to connect and each blocking read, and map the resulting TimeoutError to AssetDownloadError with a network/proxy hint. It is not a total-download budget, so healthy slow transfers are unaffected.
fix(utils): time-bound asset downloads so a blocked network fails fast
…dle) `Genelab-Velocity-Flat-Unitree-G1-v0` trained at ~100 s/iter with the GPU at 0% util — a ~50x regression vs the ~2.4 s/iter reference. Root cause: the G1 env cfgs never set `SimulationCfg.gpu`, which defaults to False, so `InteractiveScene` initialized Genesis with the **CPU backend** (`backend = gs.gpu if sim_cfg.gpu else gs.cpu`). The entire humanoid physics ran on CPU while the policy/tensors sat on cuda — GPU idle, CPU-bound. (The showcase tasks already set `gpu=True`; the G1 training cfgs were missed in the vNext refactor.) Set `gpu=True` on both G1 cfgs (`unitree_g1_velocity_env_cfg` and `unitree_g1_tracking_env_cfg`). Measured on an idle RTX 5060 Ti @1024 envs (QD_GRAPH=0, headless): - env.step: 1481 ms (CPU backend) -> 52 ms (GPU) — ~28x. - `genelab train` end-to-end: ~100 s/iter @0% GPU -> ~1.9 s/iter @76-86% GPU, matching the reference-runs.md ~2.4 s/iter target. Add a guard test (`test_g1_tasks_run_simulation_on_gpu_backend`) asserting both G1 tasks' train + play env cfgs set `simulation.gpu=True`, so this can't silently regress again. Runs on CI without a Genesis runtime (pure cfg check).
fix(g1): run Genesis on the GPU backend (was CPU — ~50× slower, GPU idle)
The Franka SAC+HER pick-and-place env cfg never set `SimulationCfg.gpu`, which defaults to False — so Genesis initialized on the CPU backend and ran the sim on CPU while the policy/tensors sat on cuda (the same regression just fixed for G1, #165). num_envs=64 makes the absolute cost smaller than G1, but it's still the wrong backend. Set `gpu=True` on the Franka env cfg. Verified on an RTX 5060 Ti: the env builds and steps on the GPU backend (9.5 ms/step @64 envs; HER Dict obs intact). Add a guard test asserting the task's train + play env cfgs set `simulation.gpu=True` (pure cfg check, runs on CI without Genesis).
…d footgun) `SimulationCfg.gpu` defaulted to False while env `device` defaults to "cuda", so a task that set `device="cuda"` but didn't also set `gpu=True` silently ran the whole sim on the CPU backend — GPU idle, ~50x slower. This bit G1 (#165) and Franka (#166); any future cuda task would trip the same wire. Make `gpu` default to `None` = "follow the device": `_resolve_use_gpu(gpu, device)` returns the explicit flag when set, else `"cuda" in device`. The scene already receives `device_hint`, so no new plumbing. Now a `device="cuda"` env runs the sim on the GPU with no separate opt-in; `device="cpu"` (or explicit `gpu=False`) stays on CPU; the distributed path still force-sets `gpu=True`. Verified on an RTX 5060 Ti: a `gpu=None` + `device="cuda"` env builds on the GPU backend (9.6 ms/step, GPU active); the CPU-backend tests (`device="cpu"`, `gpu=False`) still build on CPU. Existing explicit `gpu=True` cfgs (G1, showcases) are unaffected. Tests: parametrized `_resolve_use_gpu` unit test (None follows device; True/False override) + assert the new `gpu=None` default. Refresh the G1 guard-test docstring that referenced the old `False` default.
fix(franka): run Genesis on the GPU backend (same CPU-backend bug as G1)
fix(sim): default Genesis backend to follow device (remove CPU-backend footgun)
…ion caveats Surfaced while reproducing the reference numbers: - SimulationCfg.gpu defaults to False (CPU backend) — a footgun that leaves the GPU idle and training ~50-100x slower (fixed for G1/Franka via gpu=True; custom tasks must set it too). - Hopper needs QD_GRAPH=0, which cripples contact-heavy sims — prefer non-Hopper. - `--gpus N` barely speeds up G1; run one seed per GPU instead. - 4096-env RL is CPU-bound; concurrent trainings oversubscribe the host (wall-clock only — rewards stay deterministic).
docs(reference-runs): GPU-backend / Hopper / multi-GPU reproduction caveats
KraHsu
added a commit
that referenced
this pull request
Jun 5, 2026
Release dev → main: runtime-validated fixes + features (PRs #153–168)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Promotes 36 commits (PRs #153–168) from
devtomain— the output of a multi-session runtime validation campaign (real-GPU smoke + reproduction on RTX 5060 Ti / H200 / RTX 4090). All merged via CI-green PRs and runtime-verified.Validation summary
play --agent trained→ export (ts+onnx) ✅ across rsl_rl + sb3.reference-runs.md): 4/4 reproduced —IP 39.99×3, DIP 59.96/59.88/59.98, G1-Velocity 113.35/92.51/91.78 (vs 112.42/93.42/92.03), Franka success 0.99/0.99/0.93.
What landed (by theme)
--headlessflag), fix(utils): time-bound asset downloads so a blocked network fails fast #164 (asset-download timeout).robot_stateAPI).Notes for reproducers (see #168)
SimulationCfg.gpudefaults to CPU (footgun → setgpu=True); Hopper needsQD_GRAPH=0(slows contact sims);--gpus Ngives ~no G1 speedup (run one seed/GPU); 4096-env RL is CPU-bound.🤖 Generated with Claude Code