Repository navigation
Convert all eight showcases to real-time display - #198
Merged
Merged
Conversation
… runner ShowcaseRunner gains the shared infrastructure the realtime showcase pass needs: - A SpringCfg "marionette" that suspends a robot link via Genesis solver force/torque (gravity feed-forward + PD hold + uprighting), so passive humanoids stay standing. - Real-time pacing of the play loop (gated on the viewer) so motion reads naturally. - A _post_step hook for per-step debug drawing (camera frustum, ray casts, ...). - A guard so the final dump is skipped once the viewer closes (a camera-backed dump would otherwise raise "Viewer already closed").
Genesis ships a live Matplotlib image viewer (MPLImagePlot) but the recording layer did not expose it. Add MPLImagePlotCfg so a CameraSensor source can stream its rgb / depth frame to a live image window (the image counterpart to the line plotters), wire it through register/resolve, and require a camera source for it. Also rename the ambiguous _PLOTTER_CFGS to _LINE_PLOT_CFGS (image plots are plotters too), only enforce the camera-source rule for sensor-name sources (not callables), and drop the redundant `from __future__ import annotations` in this package (the forward refs are already quoted). Docs and tests updated.
GeneLab's analytic RayCastSensor only intersects the ground plane / heightfield (the fast path for terrain height-scan). MeshRayCastSensor wraps Genesis's native Raycaster (BVH ray-cast against real meshes) so rays hit arbitrary rigid bodies. The two coexist with a clear division of responsibility. The sensor follows the CameraSensor pattern: it registers the Genesis raycaster in pre_build_genesis and reads it in _compute_data, flattening the pattern's ray grid to a uniform (num_envs, num_rays, ...) payload. Thin MeshGridPattern / MeshSphericalPattern specs mirror Genesis's patterns and defer construction to build time (Genesis patterns need gs.init()); genesis stays a lazy import. RigidObjectCfg gains use_visual_raycasting to opt a body into the visual BVH. Public surface (genelab.lab / sensor __all__), snapshot test, and sensor docs updated.
Replace the per-step disk dumps (PNG frames + frame.log) with live windows: the IMU, frame-transformer, and joint force/torque sensors stream to live PyQt curves declared as scene recordings; the wrist camera renders RGB + depth into a pyqtgraph image window (Qt, smooth) and its view frustum is drawn in the main viewer. Adds table props so the camera sees real geometry, and fixes the camera offset_quat so it looks along the gripper-forward axis (it was 90 degrees off). Nothing is written to disk.
Swap the analytic RayCastSensor (which only hit the ground) for MeshRayCastSensor so the rays hit real box / sphere props placed in front of the robot, scanning along the gripper-forward axis as joint 1 sweeps. The hit point cloud renders live in a dedicated GPU pyqtgraph 3D-scatter window (one colour per sensor); the old distances.log dump is gone. Nothing is written to disk.
The README listed six tasks; the package registers eight. Add the MlpResidual-Actuator and Recording showcases, fix the Sensors entry (ForceTorqueSensor), and refresh the layout tree.
Replace the static G1 + air_time.log dump with an in-place march: a virtual pelvis spring (SpringCfg) holds the passive robot upright while the runner scripts an alternating high-knee step (hip + knee + ankle flexion per side). Each foot's air-time streams to a live PyQt plot, and a contact-state sphere is drawn under each foot (green = planted, red = airborne, radius ~ contact force). Nothing is written to disk.
Replace the dropped G1 + terrain.log dump with a guided tour: a virtual pelvis spring holds the G1 upright while the runner teleports it cell to cell across the 1x5 sub-terrain row (flat -> stairs -> rough -> slope -> wave), re-anchoring the spring at each cell. The main viewer follows the robot. A downward pelvis ray-cast scans the surface; the runner draws the rays in the scene and renders the hit point cloud (pyqtgraph GL, height-coloured) plus a live height cross-section. The scan grid is densified to 11x11. Nothing is written to disk.
The raycast, sensors, and terrain runners each carried the same lazy-init pyqtgraph window boilerplate (mkQApp, one-time build, best-effort exception fallback, processEvents). Hoist it into genelab_showcase._viz.LazyQtWindows: runners now pass a build callback to ``ensure(...)`` and pump events via ``process()``. Behaviour is unchanged; the next showcase that needs a window reuses the helper instead of copying the guard logic.
…level histogram Replace the disk-dumping curriculum demo with a real-time scene: 25 passive Unitree-G1 robots tile a 5x5 grid, one per cell, held upright by the virtual spring (vertical + uprighting) plus a kinematic planar pin. Each grid row is a distinct, escalating terrain type -- flat, wave, random-rough, pyramid stairs, slope -- so the difficulty gradient reads from across the scene. Five RandomRoughCfg rows would have collapsed to one shape because Genesis keys terrain parameters by type, so distinct types are required. The runner evolves a per-robot logical curriculum level (promote/demote), floats a head marker coloured by level over each robot, and renders the live level distribution as a pyqtgraph histogram. Markers are drawn at root_pos + envs_offset because the multi-env render offset, not the local per-env root position, is where the viewer shows each robot. Nothing is written to disk.
…ve sliders Replace the disk-dumping actuator showcases with real-time pyqtgraph windows and fix the "moves once then sticks" behaviour. The stick was the Python force-channel PD buzzing (under-damped) at the coarse 0.01 dt: at the finer 0.005 dt (decimation 4, 50 Hz control) with arm damping 40 the reported velocity matches the finite-difference velocity, so joint 1 genuinely tracks the sine instead of jittering. Both showcases now drive joint 1 with an accumulated-phase sine and plot the commanded target, the achieved position and the tracking error live, with four sliders (kp, kv, amplitude, period) that write straight into the actuator PD gains and reshape the sine. The MlpResidual showcase becomes a side-by-side comparison: two Frankas via the env's `robots` dict -- `robot` (MlpResidual) and `robot_ideal` (plain IdealPD) -- driven from the same scripted target through separate action terms, their joint-1 responses overlaid. env.step only refreshes the primary articulation's data cache, so the runner refreshes every robot it reads (the non-primary one otherwise reads stale zeros). The residual net is retuned to a position-error compensator (80 * pos_error) so the residual visibly tightens tracking, more so as the kp slider is lowered. Nothing is written to disk.
…recorder warning Default the Recording showcase to live plots only (both data sources now drive PyQt plots): it writes nothing to disk unless GENELAB_SHOWCASE_RECORD_FILES is set, in which case the lin-acc CSV and a per-episode orientation NPZ are added. This stops the default run from silently filling the disk; the module docstring documents the switch. Also silence Genesis's repeated "start_thread(): Processor thread already exists" warning. Its RecorderManager.reset() restarts the still-running processor thread on every save_on_reset flush, logging the warning each episode; the bridge now filters that one message on the Genesis logger just around its reset() call. This benefits any env with threaded save_on_reset recorders, not only the showcase.
KraHsu
added a commit
that referenced
this pull request
Jun 5, 2026
Convert all eight showcases to real-time display
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Convert all eight bundled
genelab_showcaseexamples from silent disk-dumping to real-time, on-screen display. Every showcase now visualises live (smooth Qt / pyqtgraph) instead of writing files the user didn't ask for and couldn't see.Global direction applied to all eight:
LazyQtWindowshelper and a virtual-spring base class (gravity comp + uprighting + pinning) keep the runners consistent.The eight showcases
MeshRayCastSensorhit point cloud (GL scatter)Library improvements (reused beyond the showcases)
MeshRayCastSensor— Genesis-backed BVH mesh ray-caster (coexists with the analytic terrain raycaster);RigidObjectCfg.use_visual_raycasting.MPLImagePlotCfg— live-image output in the recording layer.start_thread(): Processor thread already existswarning aroundsave_on_resetflushes (benefits any threaded-recorder env).Notable fixes found along the way
dt=0.01; stabilised withdt=0.005/ decimation 4 / arm damping 40.env.steponly refreshes the primary articulation, so the two-arm runner refreshes every robot it reads.RandomRoughCfgrows collapse to one shape (Genesis keys terrain params by type) — used five distinct terrain types for a visible difficulty gradient; markers drawn atroot_pos + envs_offsetto match the multi-env render.Testing
Full suite: 659 passed (headless, 0 failures). Each showcase was visually reviewed and passed a 3-agent quality review before commit.
🤖 Generated with Claude Code