Skip to content

Commit c2e252a

Browse files
authored
Merge pull request #198 from KraHsu/dev/showcase-realtime-revamp
Convert all eight showcases to real-time display
2 parents c8c71ac + e37d4f5 commit c2e252a

33 files changed

Lines changed: 1730 additions & 553 deletions

‎docs/concepts/recording.en.md‎

Lines changed: 8 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ configs into Genesis recorders during build, then binds them to the live env aft
1010

1111
```text
1212
RecordingCfg(source, field, env_idx)
13-
└── PyQtPlotCfg / MPLPlotCfg / NPZFileCfg / CSVFileCfg / VideoFileCfg
13+
└── PyQtPlotCfg / MPLPlotCfg / MPLImagePlotCfg / NPZFileCfg / CSVFileCfg / VideoFileCfg
1414
```
1515

1616
## Sources
@@ -28,12 +28,17 @@ returns a scalar or when a file should capture the full batch.
2828

2929
| Output | Use case |
3030
|---|---|
31-
| `PyQtPlotCfg` | Live PyQtGraph plots. |
32-
| `MPLPlotCfg` | Live Matplotlib plots. |
31+
| `PyQtPlotCfg` | Live PyQtGraph line plots (time series). |
32+
| `MPLPlotCfg` | Live Matplotlib line plots (time series). |
33+
| `MPLImagePlotCfg` | Live Matplotlib image window for a `CameraSensor` source — `field="rgb"` for the frame, `field="depth"` for the depth map. |
3334
| `NPZFileCfg` | Compressed array dumps at cleanup/reset. |
3435
| `CSVFileCfg` | Row-oriented streaming output. |
3536
| `VideoFileCfg` | Camera frames to `.mp4`. |
3637

38+
Line plots (`PyQtPlotCfg` / `MPLPlotCfg`) consume 1-D channel data; `MPLImagePlotCfg` consumes a 2-D
39+
frame and requires a camera source. All plot windows auto-detect the display and no-op on a headless
40+
host.
41+
3742
## Where to continue
3843

3944
- [Sensors](sensors.md)

‎docs/concepts/recording.zh.md‎

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ Recording 把选定运行时信号变成实时曲线、文件或视频,而不
88

99
```text
1010
RecordingCfg(source, field, env_idx)
11-
└── PyQtPlotCfg / MPLPlotCfg / NPZFileCfg / CSVFileCfg / VideoFileCfg
11+
└── PyQtPlotCfg / MPLPlotCfg / MPLImagePlotCfg / NPZFileCfg / CSVFileCfg / VideoFileCfg
1212
```
1313

1414
## 数据源
@@ -25,12 +25,15 @@ RecordingCfg(source, field, env_idx)
2525

2626
| Output | 用途 |
2727
|---|---|
28-
| `PyQtPlotCfg` | 实时 PyQtGraph 曲线。 |
29-
| `MPLPlotCfg` | 实时 Matplotlib 曲线。 |
28+
| `PyQtPlotCfg` | 实时 PyQtGraph 折线图(时间序列)。 |
29+
| `MPLPlotCfg` | 实时 Matplotlib 折线图(时间序列)。 |
30+
| `MPLImagePlotCfg` | `CameraSensor` 源的实时 Matplotlib 图像窗口——`field="rgb"` 显示画面,`field="depth"` 显示深度图。 |
3031
| `NPZFileCfg` | cleanup/reset 时写压缩数组。 |
3132
| `CSVFileCfg` | 行式流式输出。 |
3233
| `VideoFileCfg` | 相机帧写 `.mp4`。 |
3334

35+
折线图(`PyQtPlotCfg` / `MPLPlotCfg`)消费一维通道数据;`MPLImagePlotCfg` 消费二维帧,且要求相机源。所有绘图窗口都会自动检测显示器,无显示器时静默跳过。
36+
3437
## 继续阅读
3538

3639
- [传感器](sensors.md)

‎docs/concepts/sensors.en.md‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,8 @@ This lazy cache keeps repeated observation and reward reads consistent within on
2121
| `CameraSensor` | RGB-D camera attached to a link. |
2222
| `ContactSensor` | Per-link contact force and optional air/contact time. |
2323
| `FrameTransformerSensor` | Target frame poses relative to a source frame. |
24-
| `RayCastSensor` | Grid, ring, or hemisphere ray patterns. |
24+
| `RayCastSensor` | Analytic ray-vs-ground / heightfield (grid, ring, hemisphere patterns) — the fast O(1) path for terrain height-scan. |
25+
| `MeshRayCastSensor` | Genesis BVH ray-cast against real scene meshes — rays hit arbitrary rigid bodies (props, obstacles). Targets must set `use_visual_raycasting=True`. |
2526
| `TerrainHeightSensor` | Downward height scan for terrain-aware locomotion. |
2627
| `SelfContactSensor` | Self-collision/contact signals. |
2728
| `RootAngularMomentumSensor` | Root angular momentum diagnostics. |

‎docs/concepts/sensors.zh.md‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,8 @@ Genesis 不会通过 MJCF sensor block 暴露所有信号,因此 GeneLab 提
1919
| `CameraSensor` | 挂到 link 上的 RGB-D 相机。 |
2020
| `ContactSensor` | 每 link 接触力和可选 air/contact time。 |
2121
| `FrameTransformerSensor` | target frame 相对 source frame 的姿态。 |
22-
| `RayCastSensor` | grid、ring、hemisphere ray pattern。 |
22+
| `RayCastSensor` | 解析法射线 vs 地面 / heightfield(grid、ring、hemisphere pattern)——地形高度扫描的 O(1) 快路。 |
23+
| `MeshRayCastSensor` | Genesis BVH 射线对真实场景 mesh 求交——射线能打到任意刚体(道具、障碍)。目标需设 `use_visual_raycasting=True`。 |
2324
| `TerrainHeightSensor` | 地形感知 locomotion 的向下 height scan。 |
2425
| `SelfContactSensor` | 自碰撞/自接触信号。 |
2526
| `RootAngularMomentumSensor` | root angular momentum 诊断。 |

‎examples/genelab_showcase/README.md‎

Lines changed: 7 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,18 @@
11
# GeneLab visual showcase
22

3-
Play-only GeneLab extension that wires every M1–M4 building block into a six-task viewer
3+
Play-only GeneLab extension that wires every M1–M4 building block into an eight-task viewer
44
demo. Each task drops a real robot (Franka or Unitree G1) into a `ManagerBasedRlEnv`,
55
runs a scripted action loop, and dumps the relevant sensor / curriculum / terrain state
66
so the visual behaviour can be checked by eye.
77

8-
- `GeneLab-Sensors-Showcase-v0` — Franka with `CameraSensor` (RGB+depth → PNG), `IMUSensor`, `FrameTransformerSensor`.
8+
- `GeneLab-Sensors-Showcase-v0` — Franka with `CameraSensor` (RGB+depth → PNG), `IMUSensor`, `FrameTransformerSensor`, and `ForceTorqueSensor`.
99
- `GeneLab-RayCast-Showcase-v0` — Franka with three `RayCastSensor` instances mounted side by side (`GridPattern` + `RingPattern` + `HemispherePattern`), all three dumped in parallel.
1010
- `GeneLab-Contact-Showcase-v0` — Unitree G1 with `ContactSensor`, air-time tracking on both feet.
1111
- `GeneLab-Terrain-Showcase-v0` — Unitree G1 dropped on a 1×5 row that tiles every built-in sub-terrain (`flat` + `stairs` + `rough` + `slope` + `wave`) in a single scene.
1212
- `GeneLab-Curriculum-Showcase-v0` — Unitree G1 on a 5×5 RandomRough grid, scripted XY drift drives `terrain_levels_vel` promotion.
1313
- `GeneLab-Actuator-Showcase-v0` — Franka with `IdealPDActuator` on the arm (force-channel control), prints joint-tracking error against a sine target.
14+
- `GeneLab-MlpResidual-Actuator-Showcase-v0` — Franka with `MlpResidualActuator` on the arm (DC-motor base + TorchScript residual network).
15+
- `GeneLab-Recording-Showcase-v0` — Franka with live PyQt / Matplotlib plots and NPZ / CSV data dumps from an IMU.
1416

1517
## Installation
1618

@@ -55,12 +57,13 @@ examples/genelab_showcase/
5557
└── src/genelab_showcase/
5658
├── tasks.py # register() + RegisteredTask dispatcher
5759
├── runner.py # ShowcaseRunner base class
58-
├── sensors/ # Camera + IMU + FrameTransformer
60+
├── sensors/ # Camera + IMU + FrameTransformer + ForceTorque
5961
├── raycast/ # Grid / Ring / Hemisphere
6062
├── contact/ # G1 foot air-time
6163
├── terrain/ # 5 sub-terrains + mixed
6264
├── curriculum/ # terrain_levels_vel
63-
└── actuators/ # ImplicitPD / IdealPD / DCMotor
65+
├── actuators/ # IdealPD (force control) + MlpResidual (DC-motor + residual)
66+
└── recording/ # live PyQt/MPL plots + NPZ/CSV IMU dump
6467
```
6568

6669
The runtime entry point is `genelab_showcase.tasks:register`, auto-discovered by
Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
"""Lazy pyqtgraph window helper shared by the showcase runners.
2+
3+
Every showcase that renders a live Qt window (point cloud, camera image, plot) opens it lazily
4+
on the first viz tick and tolerates a missing pyqtgraph / display: the window is built once via
5+
a builder callback, and if that fails the runner silently continues without it (never retrying).
6+
"""
7+
8+
from collections.abc import Callable
9+
from typing import Any
10+
11+
12+
class LazyQtWindows:
13+
"""One-time pyqtgraph app + window setup with a best-effort fallback.
14+
15+
Usage::
16+
17+
self._qt = LazyQtWindows()
18+
...
19+
if self._qt.ensure(self._build, label="my showcase"):
20+
self._item.setData(...)
21+
self._qt.process()
22+
23+
:meth:`ensure` calls ``builder(app)`` exactly once (``app`` is the ``QApplication``). It
24+
returns ``True`` when pyqtgraph is importable and the build succeeds, ``False`` otherwise —
25+
and caches the verdict so a failed import is not retried on every step.
26+
"""
27+
28+
def __init__(self) -> None:
29+
self._app: Any = None
30+
self._tried = False
31+
self._ready = False
32+
33+
def ensure(self, builder: Callable[[Any], None], *, label: str) -> bool:
34+
if self._tried:
35+
return self._ready
36+
self._tried = True
37+
try:
38+
import pyqtgraph as pg
39+
40+
self._app = pg.mkQApp(label)
41+
builder(self._app)
42+
self._ready = True
43+
except Exception as exc: # noqa: BLE001 - viz is best-effort, never break the sim
44+
print(f"{label}: live window unavailable ({exc})")
45+
return self._ready
46+
47+
def process(self) -> None:
48+
"""Pump the Qt event loop so the window stays responsive."""
49+
if self._app is not None:
50+
self._app.processEvents()

0 commit comments

Comments
 (0)