Skip to content

Commit 33c80b5

Browse files
authored
Merge pull request #180 from KraHsu/feat/genesis-1.0-s5-xacro-imgui
Add InteractiveScene camera debug helpers (S5)
2 parents 1439bed + 780b56e commit 33c80b5

6 files changed

Lines changed: 268 additions & 0 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -54,6 +54,15 @@ All notable changes to GeneLab are recorded here.
5454
time, so robots whose actuator parameters live in Python config —
5555
rather than in an MJCF file — can use the same named knobs without
5656
hand-translating to PD gains.
57+
- **Camera debug helpers.** Added
58+
`InteractiveScene.draw_camera_frustums(camera_names=None, color=...)`
59+
and `InteractiveScene.draw_camera_trajectory(positions, radius, color)`
60+
wrapping Genesis 1.0's `scene.draw_debug_frustum` /
61+
`scene.draw_debug_trajectory`. The frustum helper resolves names
62+
against the scene's `CameraSensor` set; the trajectory helper passes
63+
positions through verbatim. Added a public `gs_camera` property on
64+
`CameraSensor` so the scene helper can reach the Genesis handle
65+
without poking private state.
5766

5867
### Changed
5968

‎docs/concepts/scene.en.md‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,23 @@ Genesis APIs and Isaac Lab-style task code use different vocabulary. Wrappers is
3131
difference: MDP terms read stable GeneLab properties while the backend integration handles Genesis
3232
details.
3333

34+
## Camera debug helpers
35+
36+
`InteractiveScene` exposes two thin wrappers around Genesis 1.0's debug-draw
37+
API for inspecting attached cameras:
38+
39+
- `draw_camera_frustums(camera_names=None, color=(1, 1, 1, 0.3))` draws the view
40+
frustum of every (or named) `CameraSensor` on the scene. Returns the number of
41+
frustums drawn. Raises if a name does not resolve to a `CameraSensor`.
42+
- `draw_camera_trajectory(positions, radius=0.002, color=(1, 0.5, 0, 0.8))`
43+
draws a polyline through `positions` (one row per world-frame point). The
44+
caller owns the buffer — typically a recorded camera path — so the scene
45+
keeps no history.
46+
47+
Both helpers must be called after `InteractiveScene.build()`; calling pre-build
48+
raises `RuntimeError`. Use them from a play-time hook to visualise where the
49+
RGB-D sensors are looking and how their pose evolved over a rollout.
50+
3451
## Where to continue
3552

3653
- [Module Map](../reference/module-map.md)

‎docs/concepts/scene.zh.md‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,22 @@ entity 表访问,例如用 `env.articulations["robot"].data` 读取 `RobotStat
2727

2828
Genesis API 和 Isaac Lab 风格任务代码使用的词汇不同。wrapper 隔离这种差异:MDP term 读取稳定的 GeneLab 属性,后端集成负责 Genesis 细节。
2929

30+
## 相机调试辅助
31+
32+
`InteractiveScene` 提供两个对 Genesis 1.0 debug-draw API 的薄封装,用于检查
33+
挂载在场景上的相机:
34+
35+
- `draw_camera_frustums(camera_names=None, color=(1, 1, 1, 0.3))` 绘制场景上
36+
每一个(或指定名字的)`CameraSensor` 的视锥。返回实际绘制的视锥数。指定
37+
的名字若没有对应 `CameraSensor` 会抛错。
38+
- `draw_camera_trajectory(positions, radius=0.002, color=(1, 0.5, 0, 0.8))`
39+
在 `positions`(每行一个世界系坐标点)上画一条折线。调用方持有 buffer ——
40+
通常是录制下来的相机路径 —— 所以 scene 不维护任何历史。
41+
42+
两个辅助方法都必须在 `InteractiveScene.build()` 之后调用;build 之前调用会抛
43+
`RuntimeError`。一般是从 play 阶段的 hook 里调用,用来可视化 RGB-D 传感器
44+
所看的视野以及一次 rollout 中位姿的变化。
45+
3046
## 继续阅读
3147

3248
- [模块地图](../reference/module-map.md)

‎src/genelab/scene/interactive_scene.py‎

Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,9 @@
2727
from genelab.sensor import Sensor
2828
from genelab.terrains import TerrainImporter
2929

30+
if TYPE_CHECKING:
31+
from genelab.sensor.camera import CameraSensor
32+
3033

3134
_pyrender_save_patch_applied = False
3235

@@ -340,6 +343,70 @@ def reset(self, env_ids: torch.Tensor) -> None:
340343
for art in self.articulations.values():
341344
art.reset(env_ids)
342345

346+
def draw_camera_frustums(
347+
self,
348+
*,
349+
camera_names: tuple[str, ...] | None = None,
350+
color: tuple[float, float, float, float] = (1.0, 1.0, 1.0, 0.3),
351+
) -> int:
352+
"""Draw the view frustum of one or more :class:`~genelab.sensor.CameraSensor` s.
353+
354+
Wraps Genesis 1.0's ``scene.draw_debug_frustum(camera, color=...)``. When
355+
``camera_names`` is ``None`` every ``CameraSensor`` on the scene is drawn;
356+
otherwise only the named ones (a name that doesn't resolve to a
357+
``CameraSensor`` raises). Returns the number of frustums drawn so callers
358+
can detect mis-named selections without inspecting the viewer state.
359+
360+
Cameras allocate their Genesis-side handle in
361+
:meth:`~genelab.sensor.CameraSensor.pre_build_genesis`; calling this before
362+
:meth:`build` raises a ``RuntimeError``.
363+
"""
364+
if not self._built:
365+
raise RuntimeError("draw_camera_frustums called before InteractiveScene.build()")
366+
selected = self._select_camera_sensors(camera_names)
367+
for sensor in selected:
368+
self._gs_scene.draw_debug_frustum(sensor.gs_camera, color=color)
369+
return len(selected)
370+
371+
def draw_camera_trajectory(
372+
self,
373+
positions: Any,
374+
*,
375+
radius: float = 0.002,
376+
color: tuple[float, float, float, float] = (1.0, 0.5, 0.0, 0.8),
377+
) -> None:
378+
"""Draw a polyline through ``positions`` (one row per world-frame point).
379+
380+
Thin wrapper over Genesis 1.0's
381+
``scene.draw_debug_trajectory(poss, radius=…, color=…)``. The caller owns
382+
the positions buffer (typically a recorded camera path or an inspection
383+
waypoint sequence) — no history is maintained on the scene side.
384+
"""
385+
if not self._built:
386+
raise RuntimeError("draw_camera_trajectory called before InteractiveScene.build()")
387+
self._gs_scene.draw_debug_trajectory(positions, radius=radius, color=color)
388+
389+
def _select_camera_sensors(self, names: tuple[str, ...] | None) -> "list[CameraSensor]":
390+
from genelab.sensor.camera import CameraSensor
391+
392+
if names is None:
393+
return [s for s in self._sensors.values() if isinstance(s, CameraSensor)]
394+
selected: list[CameraSensor] = []
395+
for n in names:
396+
sensor = self._sensors.get(n)
397+
if sensor is None:
398+
raise KeyError(
399+
f"draw_camera_frustums: no sensor named {n!r} on this scene "
400+
f"(have: {sorted(self._sensors)})"
401+
)
402+
if not isinstance(sensor, CameraSensor):
403+
raise TypeError(
404+
f"draw_camera_frustums: sensor {n!r} is not a CameraSensor "
405+
f"(got {type(sensor).__name__})"
406+
)
407+
selected.append(sensor)
408+
return selected
409+
343410
def close(self) -> None:
344411
scene = self._gs_scene
345412
if scene is None:

‎src/genelab/sensor/camera.py‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -151,6 +151,16 @@ def _validate_cfg(self) -> None:
151151
f"render_rgb / render_depth / render_segmentation must be True"
152152
)
153153

154+
@property
155+
def gs_camera(self) -> object | None:
156+
"""The underlying Genesis camera handle, or ``None`` before ``pre_build_genesis``.
157+
158+
Exposed for scene-level debug helpers (e.g.
159+
:meth:`~genelab.scene.InteractiveScene.draw_camera_frustums`) that need the
160+
handle to call Genesis's ``draw_debug_frustum`` API directly.
161+
"""
162+
return self._cam
163+
154164
def _allocate_camera(self, gs_scene: object, link: object) -> object:
155165
cam = gs_scene.add_camera( # type: ignore[attr-defined]
156166
res=(self._cfg_typed.width, self._cfg_typed.height),

‎tests/test_scene_camera_debug.py‎

Lines changed: 149 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,149 @@
1+
"""Unit tests for ``InteractiveScene.draw_camera_frustums`` / ``draw_camera_trajectory``.
2+
3+
The scene helpers wrap Genesis 1.0's ``scene.draw_debug_frustum(camera, color=...)``
4+
and ``scene.draw_debug_trajectory(poss, ...)``. The tests use a fake gs_scene that
5+
records every debug-draw call so we can assert (a) the right camera handle is
6+
forwarded for each selection and (b) selection errors are raised with useful
7+
messages — no real Genesis viewer needed.
8+
"""
9+
10+
from typing import Any
11+
12+
import pytest
13+
14+
torch = pytest.importorskip("torch")
15+
16+
17+
class _FakeCameraHandle:
18+
"""Stand-in for the gs.Camera handle attached by ``CameraSensor._allocate_camera``."""
19+
20+
def __init__(self, name: str) -> None:
21+
self.name = name
22+
23+
24+
class _FakeCameraSensor:
25+
"""Minimum subset of CameraSensor that ``_select_camera_sensors`` reads.
26+
27+
Subclasses the real ``CameraSensor`` class symbol so the ``isinstance`` check
28+
inside the helper still passes — the helper only ever touches ``gs_camera``.
29+
"""
30+
31+
def __init__(self, name: str) -> None:
32+
self._name = name
33+
self._cam = _FakeCameraHandle(name)
34+
35+
@property
36+
def gs_camera(self) -> Any:
37+
return self._cam
38+
39+
40+
class _FakeGsScene:
41+
"""Records every debug-draw call so tests can assert on them."""
42+
43+
def __init__(self) -> None:
44+
self.frustum_calls: list[tuple[Any, tuple[float, float, float, float]]] = []
45+
self.trajectory_calls: list[tuple[Any, float, tuple[float, float, float, float]]] = []
46+
47+
def draw_debug_frustum(self, camera: Any, color: tuple[float, float, float, float]) -> None:
48+
self.frustum_calls.append((camera, color))
49+
50+
def draw_debug_trajectory(
51+
self, poss: Any, radius: float, color: tuple[float, float, float, float]
52+
) -> None:
53+
self.trajectory_calls.append((poss, radius, color))
54+
55+
56+
def _build_scene(sensors: dict[str, Any], *, built: bool = True) -> Any:
57+
"""Construct just enough ``InteractiveScene`` surface for the debug helpers."""
58+
from genelab.scene.interactive_scene import InteractiveScene
59+
60+
scene = InteractiveScene.__new__(InteractiveScene)
61+
scene._gs_scene = _FakeGsScene()
62+
scene._sensors = sensors
63+
scene._built = built
64+
return scene
65+
66+
67+
def _patch_camera_sensor_isinstance(monkeypatch: pytest.MonkeyPatch) -> None:
68+
"""``_select_camera_sensors`` uses ``isinstance(sensor, CameraSensor)``; route
69+
the import to a class the fakes inherit from so the test fakes pass the check
70+
without needing a real Genesis env to construct a real ``CameraSensor``."""
71+
import genelab.sensor.camera as camera_mod
72+
73+
monkeypatch.setattr(camera_mod, "CameraSensor", _FakeCameraSensor)
74+
75+
76+
def test_draw_camera_frustums_default_draws_every_camera(
77+
monkeypatch: pytest.MonkeyPatch,
78+
) -> None:
79+
_patch_camera_sensor_isinstance(monkeypatch)
80+
sensors = {"front": _FakeCameraSensor("front"), "side": _FakeCameraSensor("side")}
81+
scene = _build_scene(sensors)
82+
83+
n = scene.draw_camera_frustums()
84+
85+
assert n == 2
86+
drawn = [call[0].name for call in scene._gs_scene.frustum_calls]
87+
assert sorted(drawn) == ["front", "side"]
88+
89+
90+
def test_draw_camera_frustums_named_selection(
91+
monkeypatch: pytest.MonkeyPatch,
92+
) -> None:
93+
_patch_camera_sensor_isinstance(monkeypatch)
94+
sensors = {"front": _FakeCameraSensor("front"), "side": _FakeCameraSensor("side")}
95+
scene = _build_scene(sensors)
96+
97+
n = scene.draw_camera_frustums(camera_names=("side",), color=(0.1, 0.2, 0.3, 0.4))
98+
99+
assert n == 1
100+
cam, color = scene._gs_scene.frustum_calls[0]
101+
assert cam.name == "side"
102+
assert color == (0.1, 0.2, 0.3, 0.4)
103+
104+
105+
def test_draw_camera_frustums_unknown_name_raises(
106+
monkeypatch: pytest.MonkeyPatch,
107+
) -> None:
108+
_patch_camera_sensor_isinstance(monkeypatch)
109+
sensors = {"front": _FakeCameraSensor("front")}
110+
scene = _build_scene(sensors)
111+
112+
with pytest.raises(KeyError, match="no sensor named 'ghost'"):
113+
scene.draw_camera_frustums(camera_names=("ghost",))
114+
115+
116+
def test_draw_camera_frustums_non_camera_sensor_raises(
117+
monkeypatch: pytest.MonkeyPatch,
118+
) -> None:
119+
_patch_camera_sensor_isinstance(monkeypatch)
120+
sensors = {"front": _FakeCameraSensor("front"), "imu": object()}
121+
scene = _build_scene(sensors)
122+
123+
with pytest.raises(TypeError, match="not a CameraSensor"):
124+
scene.draw_camera_frustums(camera_names=("imu",))
125+
126+
127+
def test_draw_camera_frustums_pre_build_raises() -> None:
128+
scene = _build_scene({}, built=False)
129+
with pytest.raises(RuntimeError, match="before InteractiveScene.build"):
130+
scene.draw_camera_frustums()
131+
132+
133+
def test_draw_camera_trajectory_forwards_args() -> None:
134+
scene = _build_scene({})
135+
poss = torch.tensor([[0.0, 0.0, 0.0], [1.0, 0.0, 0.0], [1.0, 1.0, 0.0]])
136+
137+
scene.draw_camera_trajectory(poss, radius=0.005, color=(0.0, 1.0, 0.0, 1.0))
138+
139+
assert len(scene._gs_scene.trajectory_calls) == 1
140+
forwarded, radius, color = scene._gs_scene.trajectory_calls[0]
141+
assert torch.equal(forwarded, poss)
142+
assert radius == pytest.approx(0.005)
143+
assert color == (0.0, 1.0, 0.0, 1.0)
144+
145+
146+
def test_draw_camera_trajectory_pre_build_raises() -> None:
147+
scene = _build_scene({}, built=False)
148+
with pytest.raises(RuntimeError, match="before InteractiveScene.build"):
149+
scene.draw_camera_trajectory([[0.0, 0.0, 0.0]])

0 commit comments

Comments
 (0)