diff --git a/docs/api/reference.md b/docs/api/reference.md index c6b15133..6938bd48 100644 --- a/docs/api/reference.md +++ b/docs/api/reference.md @@ -1,43 +1,102 @@ +--- +search: + exclude: true +--- + # API Reference -Auto-generated from the source docstrings via -[`mkdocstrings`](https://mkdocstrings.github.io/). The API surface below covers GeneLab's public -core. Extension packages document their own APIs in their own projects. +This page is generated from source docstrings via +[`mkdocstrings`](https://mkdocstrings.github.io/). It is shared by both language versions of the +site. Concept and how-to pages explain how to use the APIs; this page is for signatures, members, +and defaults. -!!! note "Single-source" - The API reference is generated from English docstrings and is shared by both language - versions of this site. +## Public facade ---- +::: genelab.lab + options: + show_if_no_docstring: true -## `genelab.lab` +## Registries and configs -The public facade. Most user code should import from here rather than from `genelab.registry` -directly. +::: genelab.registry + options: + show_if_no_docstring: true -::: genelab.lab +::: genelab.configs options: show_if_no_docstring: true ---- +## Environment -## `genelab.registry` +::: genelab.envs.manager_based_rl_env + options: + show_if_no_docstring: true -::: genelab.registry +## Managers + +::: genelab.managers options: show_if_no_docstring: true ---- +## MDP library -## `genelab.configs` +::: genelab.mdp + options: + show_if_no_docstring: true -::: genelab.configs +## Scene and entities + +::: genelab.scene options: show_if_no_docstring: true ---- +::: genelab.entity + options: + show_if_no_docstring: true + +## Actuators + +::: genelab.actuator + options: + show_if_no_docstring: true + +## Sensors + +::: genelab.sensor + options: + show_if_no_docstring: true + +## Terrains + +::: genelab.terrains + options: + show_if_no_docstring: true + +## RL + +::: genelab.rl + options: + show_if_no_docstring: true + +## Recording + +::: genelab.recording + options: + show_if_no_docstring: true + +## Bridges + +::: genelab.bridges + options: + show_if_no_docstring: true + +## Asset zoo + +::: genelab.asset_zoo + options: + show_if_no_docstring: true -## `genelab.cache` +## Utilities ::: genelab.cache options: diff --git a/docs/best-practices/debugging.en.md b/docs/best-practices/debugging.en.md new file mode 100644 index 00000000..aee40f01 --- /dev/null +++ b/docs/best-practices/debugging.en.md @@ -0,0 +1,88 @@ +# How to Debug Common Failures + +This guide lists the shortest checks for common GeneLab failures. + +## Unknown task + +```bash +uv run genelab list tasks +uv run genelab --import my_project.tasks list tasks +``` + +If the second command works, the package is importable only when you pass `--import`. Install an +entry point or keep using an explicit import. If neither works, confirm the package is installed or +its `src/` directory is on `PYTHONPATH`. + +## Unknown override path + +```bash +uv run genelab info TASK_ID +``` + +Copy the path from `Overridable cfg paths`. Remember that `play` shortcuts may retarget +`env.simulation.*` to `play_env.simulation.*` when a task has `play_env`. + +## Torch or Genesis import errors + +```bash +uv run python -c "import torch; print(torch.__version__, torch.version.cuda)" +uv run python -c "import genesis; print(genesis.__version__)" +``` + +Pick exactly one `torch-*` extra and resync if needed: + +```bash +uv sync --reinstall-package torch --extra torch-cu128 +``` + +## Cache or write-location errors + +```bash +uv run genelab cache +``` + +This creates project-local cache directories and points `XDG_CACHE_HOME` and `MPLCONFIGDIR` at +`.cache/`. + +## Viewer failures + +First run headless: + +```bash +uv run genelab play TASK_ID --steps 32 +``` + +Then try viewer mode: + +```bash +uv run genelab play TASK_ID --vis --steps 128 +``` + +Viewer failures are usually graphics-driver, display, or OpenGL platform problems rather than +registry problems. + +## Observation or reward shape errors + +Manager terms should return batch-shaped tensors: + +| Term type | Expected shape | +|---|---| +| observation term | `(num_envs, d)` or `(num_envs,)` | +| reward term | `(num_envs,)` | +| termination term | `(num_envs,)` bool | + +Avoid scalar `()` tensors. They can broadcast incorrectly or fail later in the runner. + +## Distributed env-count errors + +`--num_envs` is total across ranks and must divide evenly by `--gpus`. + +```bash +uv run genelab train TASK_ID --gpus 4 --num_envs 4096 +``` + +Use `--num_envs_per_gpu` to avoid division: + +```bash +uv run genelab train TASK_ID --gpus 4 --num_envs_per_gpu 1024 +``` diff --git a/docs/best-practices/debugging.zh.md b/docs/best-practices/debugging.zh.md new file mode 100644 index 00000000..fba5693a --- /dev/null +++ b/docs/best-practices/debugging.zh.md @@ -0,0 +1,85 @@ +# 如何调试常见问题 + +这份指南列出 GeneLab 常见失败的最短排查路径。 + +## Unknown task + +```bash +uv run genelab list tasks +uv run genelab --import my_project.tasks list tasks +``` + +如果第二条命令能看到任务，说明包只有在显式 `--import` 时才被导入。安装 entry point， +或继续使用显式导入。如果两条都不行，确认包已安装，或它的 `src/` 目录在 `PYTHONPATH` 上。 + +## Unknown override path + +```bash +uv run genelab info TASK_ID +``` + +从 `Overridable cfg paths` 复制路径。注意 task 有 `play_env` 时，`play` 的短标志可能把 +`env.simulation.*` 重定向到 `play_env.simulation.*`。 + +## Torch 或 Genesis 导入错误 + +```bash +uv run python -c "import torch; print(torch.__version__, torch.version.cuda)" +uv run python -c "import genesis; print(genesis.__version__)" +``` + +只选择一个 `torch-*` extra。需要时重新同步： + +```bash +uv sync --reinstall-package torch --extra torch-cu128 +``` + +## 缓存或写入目录错误 + +```bash +uv run genelab cache +``` + +这会创建项目本地缓存目录，并把 `XDG_CACHE_HOME` 和 `MPLCONFIGDIR` 指向 `.cache/`。 + +## Viewer 失败 + +先无界面运行： + +```bash +uv run genelab play TASK_ID --steps 32 +``` + +再尝试 viewer： + +```bash +uv run genelab play TASK_ID --vis --steps 128 +``` + +viewer 失败通常是图形驱动、display 或 OpenGL platform 问题，而不是注册表问题。 + +## Observation 或 reward shape 错误 + +Manager term 应返回带 batch 维度的 tensor： + +| Term 类型 | 期望 shape | +|---|---| +| observation term | `(num_envs, d)` 或 `(num_envs,)` | +| reward term | `(num_envs,)` | +| termination term | `(num_envs,)` bool | + +避免返回标量 `()` tensor。它可能错误广播，或在 runner 中延迟失败。 + +## 分布式 env 数错误 + +`--num_envs` 表示所有 rank 的总数，必须能被 `--gpus` 整除。 + +```bash +uv run genelab train TASK_ID --gpus 4 --num_envs 4096 +``` + +想避免除法语义时使用 `--num_envs_per_gpu`： + +```bash +uv run genelab train TASK_ID --gpus 4 --num_envs_per_gpu 1024 +``` diff --git a/docs/best-practices/extension-projects.en.md b/docs/best-practices/extension-projects.en.md new file mode 100644 index 00000000..f2eb2147 --- /dev/null +++ b/docs/best-practices/extension-projects.en.md @@ -0,0 +1,91 @@ +# How to Build an Extension Project + +Use an extension project when your robot, task, or experiment should live outside `src/genelab/`. +This is the normal shape for downstream research code. + +## 1. Generate a scaffold + +```bash +uv run genelab project new my_robot_project +``` + +The generated package has: + +```text +my_robot_project/ +├── pyproject.toml +└── src/my_robot_project/ + ├── config.py + ├── envs.py + ├── robots.py + └── tasks.py +``` + +## 2. Keep responsibilities separate + +| File | Responsibility | +|---|---| +| `config.py` | Dataclasses owned by the project. | +| `robots.py` | Robot asset/config factories. | +| `envs.py` | Runtime env classes or env factory helpers. | +| `tasks.py` | `register()` hook and task classes. | + +Avoid importing Genesis at module import time unless the object truly needs it. Registry discovery +should stay cheap. + +## 3. Register through one hook + +Expose a no-argument `register()` function: + +```python +def register() -> None: + register_robot(...) + register_env(...) + register_task(...) +``` + +Guard repeated registration when the hook may run more than once: + +```python +if "my-robot" not in ROBOTS: + register_robot(...) +``` + +## 4. Prefer entry points for daily use + +In `pyproject.toml`: + +```toml +[project.entry-points."genelab.extensions"] +my_robot_project = "my_robot_project.tasks:register" +``` + +Then install as editable: + +```bash +uv pip install -e my_robot_project +uv run genelab list tasks +``` + +Use explicit imports for temporary experiments: + +```bash +PYTHONPATH=my_robot_project/src \ + uv run genelab --import my_robot_project.tasks list tasks +``` + +## 5. Verify the package boundary + +Run these from outside the package directory: + +```bash +uv run python -c "import my_robot_project" +uv run genelab list tasks +uv run genelab info MyProject-Example-v0 +uv run genelab play MyProject-Example-v0 --steps 3 +``` + +## Expected result + +The project can be imported as a normal Python package, GeneLab discovers its entry point, and all +task commands work without editing GeneLab core. diff --git a/docs/best-practices/extension-projects.zh.md b/docs/best-practices/extension-projects.zh.md new file mode 100644 index 00000000..51d066d8 --- /dev/null +++ b/docs/best-practices/extension-projects.zh.md @@ -0,0 +1,88 @@ +# 如何构建扩展项目 + +当你的机器人、任务或实验不应该放进 `src/genelab/` 时，使用扩展项目。这是下游研究代码的常规形态。 + +## 1. 生成骨架 + +```bash +uv run genelab project new my_robot_project +``` + +生成的包结构： + +```text +my_robot_project/ +├── pyproject.toml +└── src/my_robot_project/ + ├── config.py + ├── envs.py + ├── robots.py + └── tasks.py +``` + +## 2. 分清文件职责 + +| 文件 | 职责 | +|---|---| +| `config.py` | 项目自己的 dataclass。 | +| `robots.py` | 机器人资产/配置 factory。 | +| `envs.py` | 运行时 env 类或 env factory helper。 | +| `tasks.py` | `register()` hook 和 task 类。 | + +除非对象确实需要 Genesis，否则不要在模块导入时启动 Genesis。注册表发现应保持轻量。 + +## 3. 通过单个 hook 注册 + +暴露无参 `register()` 函数： + +```python +def register() -> None: + register_robot(...) + register_env(...) + register_task(...) +``` + +hook 可能运行多次时，加幂等保护： + +```python +if "my-robot" not in ROBOTS: + register_robot(...) +``` + +## 4. 日常使用优先 entry point + +在 `pyproject.toml` 中： + +```toml +[project.entry-points."genelab.extensions"] +my_robot_project = "my_robot_project.tasks:register" +``` + +然后 editable 安装： + +```bash +uv pip install -e my_robot_project +uv run genelab list tasks +``` + +临时实验使用显式导入： + +```bash +PYTHONPATH=my_robot_project/src \ + uv run genelab --import my_robot_project.tasks list tasks +``` + +## 5. 验证包边界 + +从包目录外运行： + +```bash +uv run python -c "import my_robot_project" +uv run genelab list tasks +uv run genelab info MyProject-Example-v0 +uv run genelab play MyProject-Example-v0 --steps 3 +``` + +## 预期结果 + +项目能作为普通 Python 包导入，GeneLab 能发现 entry point，所有 task 命令都能运行，且不需要修改 GeneLab core。 diff --git a/docs/best-practices/rl-experiments.en.md b/docs/best-practices/rl-experiments.en.md new file mode 100644 index 00000000..bef217a6 --- /dev/null +++ b/docs/best-practices/rl-experiments.en.md @@ -0,0 +1,85 @@ +# How to Run RL Experiments + +This guide describes the practical flow for training and replaying GeneLab tasks with `genelab.rl`. + +## 1. Smoke-test before training + +Run these before any long job: + +```bash +uv run genelab info TASK_ID +uv run genelab play TASK_ID --agent zero --steps 32 +uv run genelab play TASK_ID --agent random --steps 64 +uv run genelab train TASK_ID --num_envs 64 --max_iterations 2 +``` + +This checks registry loading, env construction, action dimensions, observation groups, rewards, +terminations, and runner wiring. + +## 2. Pick the policy source for playback + +| Agent | Behavior | Use case | +|---|---|---| +| `zero` | Sends zero actions. | Passive physics and reset health checks. | +| `random` | Sends uniform random actions in `[-1, 1]`. | Action bounds and stability checks. | +| `trained` | Loads a checkpoint through the task runner. | Inspect trained behavior. | + +Checkpoint playback: + +```bash +uv run genelab play TASK_ID \ + --agent trained \ + --checkpoint logs/rsl_rl///model_300.pt +``` + +Passing `--checkpoint` defaults `--agent` to `trained`. + +## 3. Control env count deliberately + +Single process: + +```bash +uv run genelab train TASK_ID --num_envs 4096 +``` + +Distributed: + +```bash +uv run genelab train TASK_ID --gpus 4 --num_envs 4096 +``` + +With `--gpus 4`, `--num_envs 4096` means 4096 total, 1024 per rank. Use +`--num_envs_per_gpu` when you want to specify the per-rank value directly. + +## 4. Keep logs reproducible + +Use meaningful `experiment_name` and `run_name` in `RslRlOnPolicyRunnerCfg`. GeneLab writes: + +```text +logs/rsl_rl/// +├── params/env.json +├── params/agent.json +└── model_*.pt +``` + +Use `--log_dir PATH` only when you need an exact output directory, such as a distributed relaunch or +scripted comparison. + +## 5. Profile short runs first + +```bash +uv run genelab train TASK_ID \ + --prof \ + --prof-active 3 \ + --prof-repeat 1 \ + --max_iterations 10 +uv run genelab prof open logs/torch_profile +``` + +Profiler traces grow quickly. Start with a small active window and increase only after confirming +the trace size is manageable. + +## Expected result + +Every long run should have a passing smoke test, explicit env count semantics, inspectable +`params/*.json`, and a known checkpoint replay command. diff --git a/docs/best-practices/rl-experiments.zh.md b/docs/best-practices/rl-experiments.zh.md new file mode 100644 index 00000000..1e191005 --- /dev/null +++ b/docs/best-practices/rl-experiments.zh.md @@ -0,0 +1,81 @@ +# 如何运行 RL 实验 + +这份指南说明用 `genelab.rl` 训练和回放 GeneLab task 的实用流程。 + +## 1. 训练前先做 smoke test + +任何长任务前先跑： + +```bash +uv run genelab info TASK_ID +uv run genelab play TASK_ID --agent zero --steps 32 +uv run genelab play TASK_ID --agent random --steps 64 +uv run genelab train TASK_ID --num_envs 64 --max_iterations 2 +``` + +这会检查注册表加载、env 构造、action 维度、observation group、reward、termination 和 runner 接线。 + +## 2. 为回放选择策略来源 + +| Agent | 行为 | 用途 | +|---|---|---| +| `zero` | 发送零动作。 | 被动物理和 reset 健康检查。 | +| `random` | 发送 `[-1, 1]` 均匀随机动作。 | action 边界与稳定性检查。 | +| `trained` | 通过 task runner 加载 checkpoint。 | 检查训练后行为。 | + +checkpoint 回放： + +```bash +uv run genelab play TASK_ID \ + --agent trained \ + --checkpoint logs/rsl_rl///model_300.pt +``` + +传 `--checkpoint` 时，`--agent` 默认就是 `trained`。 + +## 3. 明确控制 env 数量 + +单进程： + +```bash +uv run genelab train TASK_ID --num_envs 4096 +``` + +分布式： + +```bash +uv run genelab train TASK_ID --gpus 4 --num_envs 4096 +``` + +`--gpus 4` 时，`--num_envs 4096` 表示总数 4096，每个 rank 1024。想直接指定每个 rank +数量时用 `--num_envs_per_gpu`。 + +## 4. 保持日志可复现 + +在 `RslRlOnPolicyRunnerCfg` 中使用有意义的 `experiment_name` 和 `run_name`。GeneLab 写入： + +```text +logs/rsl_rl/// +├── params/env.json +├── params/agent.json +└── model_*.pt +``` + +只有需要精确输出目录时才使用 `--log_dir PATH`，例如分布式重启或脚本化对比。 + +## 5. 先 profile 短 run + +```bash +uv run genelab train TASK_ID \ + --prof \ + --prof-active 3 \ + --prof-repeat 1 \ + --max_iterations 10 +uv run genelab prof open logs/torch_profile +``` + +profiler trace 增长很快。先用小 active 窗口确认体量，再扩大。 + +## 预期结果 + +每个长训练都应该有通过的 smoke test、明确的 env 数语义、可检查的 `params/*.json`，以及已知的 checkpoint 回放命令。 diff --git a/docs/best-practices/task-design.en.md b/docs/best-practices/task-design.en.md new file mode 100644 index 00000000..f652d385 --- /dev/null +++ b/docs/best-practices/task-design.en.md @@ -0,0 +1,94 @@ +# How to Design a GeneLab Task + +This guide shows how to structure a task that is easy to run, inspect, train, and extend. + +## 1. Start from the task id + +Pick a stable id before writing code: + +```text +---v0 +``` + +Examples: + +- `GeneLab-Inverted-Pendulum-v0` +- `Genelab-Velocity-Flat-Unitree-G1-v0` +- `MyProject-PickPlace-v0` + +Use the same id in `TaskCfg.name`, the `TASKS` registration, logs, and README commands. + +## 2. Keep `env` and `play_env` separate + +Use `env` for training defaults and `play_env` for human inspection. + +| Config | Typical defaults | +|---|---| +| `env` | Many envs, viewer off, curriculum and disturbances on, training sensors only. | +| `play_env` | One env, viewer on, mouse/bridge controls on, reduced randomization, optional plots. | + +Do not make users remember long overrides just to open a viewer. `genelab play TASK --vis` should +work with the task's play config. + +## 3. Put robot construction behind a factory + +Register a robot factory instead of a prebuilt object: + +```python +register_robot( + "my-robot", + get_my_robot_cfg, + description="My robot.", + cfg_type=MyRobotCfg, +) +``` + +Factories keep imports light and allow the CLI to list metadata without booting Genesis. + +## 4. Organize manager terms by intent + +Use descriptive names in every manager dict. These names become CLI override paths and log keys. + +```python +rewards_cfg = { + "track_lin_vel": RewardTermCfg(...), + "action_rate": RewardTermCfg(...), + "feet_slip": RewardTermCfg(...), +} +``` + +Avoid names like `r1`, `penalty2`, or `tmp`. A good name is stable enough to appear in a paper, +run config, or dashboard. + +## 5. Make observations explicit + +Use at least a `policy` group. Add a `critic` group when the runner needs privileged information. + +```python +observations_cfg = { + "policy": ObservationGroupCfg(terms=policy_terms, enable_corruption=True), + "critic": ObservationGroupCfg(terms=critic_terms, enable_corruption=False), +} +``` + +Keep corruption, scale, and clip settings in the term cfg so they are visible through +`genelab info TASK`. + +## 6. Validate with small rollouts + +Before long training, run: + +```bash +uv run genelab info TASK_ID +uv run genelab play TASK_ID --steps 32 +uv run genelab play TASK_ID --agent random --steps 64 +uv run genelab train TASK_ID --num_envs 64 --max_iterations 2 +``` + +This catches registration, scene construction, action dimension, observation shape, reward, and +runner wiring issues before a long job. + +## Expected result + +A well-designed task has a stable task id, a clear `TaskCfg`, a viewer-friendly `play_env`, readable +manager term names, documented overrides, and a short smoke command that exercises the full path. diff --git a/docs/best-practices/task-design.zh.md b/docs/best-practices/task-design.zh.md new file mode 100644 index 00000000..d089bdee --- /dev/null +++ b/docs/best-practices/task-design.zh.md @@ -0,0 +1,92 @@ +# 如何设计 GeneLab 任务 + +这份指南说明如何组织一个容易运行、检查、训练和扩展的 GeneLab task。 + +## 1. 先定 task id + +写代码前先选一个稳定 id： + +```text +---v0 +``` + +示例： + +- `GeneLab-Inverted-Pendulum-v0` +- `Genelab-Velocity-Flat-Unitree-G1-v0` +- `MyProject-PickPlace-v0` + +`TaskCfg.name`、`TASKS` 注册、日志和 README 命令里都使用同一个 id。 + +## 2. 分开 `env` 与 `play_env` + +`env` 放训练默认值，`play_env` 放人工检查默认值。 + +| 配置 | 典型默认值 | +|---|---| +| `env` | 多 env、viewer 关闭、课程和扰动开启、只保留训练传感器。 | +| `play_env` | 单 env、viewer 开启、鼠标/bridge 控制开启、随机化较少、可选实时曲线。 | + +不要让用户为了打开 viewer 记一串 override。`genelab play TASK --vis` 应该能直接用任务的 +play 配置运行。 + +## 3. 用 factory 构造机器人 + +注册 robot factory，而不是注册已经构造好的对象： + +```python +register_robot( + "my-robot", + get_my_robot_cfg, + description="My robot.", + cfg_type=MyRobotCfg, +) +``` + +factory 能保持导入轻量，也让 CLI 在不启动 Genesis 的情况下列出元信息。 + +## 4. 按意图命名 manager term + +每个 manager 字典都使用描述性名称。这些名称会成为 CLI override 路径和日志 key。 + +```python +rewards_cfg = { + "track_lin_vel": RewardTermCfg(...), + "action_rate": RewardTermCfg(...), + "feet_slip": RewardTermCfg(...), +} +``` + +避免 `r1`、`penalty2`、`tmp` 这类名字。好名字应该稳定到可以出现在论文、run config 或 +dashboard 中。 + +## 5. 显式组织 observation + +至少提供 `policy` 组。runner 需要特权信息时再加 `critic` 组。 + +```python +observations_cfg = { + "policy": ObservationGroupCfg(terms=policy_terms, enable_corruption=True), + "critic": ObservationGroupCfg(terms=critic_terms, enable_corruption=False), +} +``` + +corruption、scale、clip 设置放在 term cfg 中，这样 `genelab info TASK` 能直接展示。 + +## 6. 用小 rollout 验证 + +长训练前先跑： + +```bash +uv run genelab info TASK_ID +uv run genelab play TASK_ID --steps 32 +uv run genelab play TASK_ID --agent random --steps 64 +uv run genelab train TASK_ID --num_envs 64 --max_iterations 2 +``` + +这能在长任务前捕获注册、scene 构造、action 维度、observation shape、reward 和 runner 接线问题。 + +## 预期结果 + +一个设计良好的 task 应该有稳定 task id、清晰的 `TaskCfg`、适合 viewer 的 `play_env`、 +可读的 manager term 名、可查的 override，以及能跑通完整路径的短 smoke 命令。 diff --git a/docs/cli/list-info.en.md b/docs/cli/list-info.en.md index ba35bd19..477e4951 100644 --- a/docs/cli/list-info.en.md +++ b/docs/cli/list-info.en.md @@ -1,14 +1,9 @@ # Discovery: list and info -The `list` and `info` subcommands are the discovery surface — together they -turn an installed extension into a printable map of available tasks, envs, -robots, and the dotted-path keys that `play` / `train` can override. +Use `list` to see what extensions registered. Use `info` to inspect one registered object and copy +override paths. -## Listing the registry - -`list KIND` enumerates one of the three global registries (`robots`, `envs`, -`tasks`). The command imports every discovered extension first, so the listing -reflects the current install set. +## List registries ```bash uv run genelab list robots @@ -16,81 +11,37 @@ uv run genelab list envs uv run genelab list tasks ``` -Each row prints the registered name, description, and a per-entry detail line -(for tasks: `env=..., robot=..., trainable=...`; for envs / robots: the first -few scalar cfg fields). Tasks that registered `examples=[...]` strings via -`register_task(..., examples=[...])` show an `(N example(s))` suffix. - -## Inspecting a task - -`info NAME` resolves a single registered name and prints a detail panel. The -lookup falls through tasks → envs → robots in that order. On a TTY an unknown -name opens a `questionary` picker over the combined task / env / robot list; -in non-interactive contexts the command exits with the available-name -listing. +If an extension is not installed, import it explicitly: ```bash -uv run genelab info GeneLab-Sensors-Showcase-v0 +PYTHONPATH=examples/inverted_pendulum/src \ + uv run genelab --import genelab_inverted_pendulum.tasks list tasks ``` -The task panel contains three sections: - -1. **Description and meta** — the one-line description plus the `env=..., - robot=..., trainable=...` strip pulled from the task cfg. -2. **Overridable cfg paths** — a three-column table (`Path`, `Type`, - `Default`) generated by walking the cfg dataclass tree depth-first. Field - names beginning with `_` are skipped. Sub-dataclasses are expanded inline - under their parent's dotted path. -3. **Examples** — the verbatim `examples=[...]` lines the extension supplied at - registration time. They are display-only — `info` does not run them. +## Inspect a task -The same panel renders for envs and robots, but without the `env / robot / -trainable` meta strip. - -## Override paths +```bash +uv run genelab info GeneLab-Inverted-Pendulum-v0 +``` -Every row in the "Overridable cfg paths" table is the exact dotted path that -`play` and `train` accept on the command line. The CLI forwards each -`-- VALUE` flag into `genelab.configs.apply_overrides`, which coerces -the value against the target field's type hint at config build time. +The task view includes: -| `info` row | Equivalent override flag | +| Section | Meaning | |---|---| -| `env.simulation.vis` (bool, false) | `--vis` (shortcut) or `--env.simulation.vis true` | -| `env.simulation.gpu` (bool, false) | `--gpu` (shortcut) or `--env.simulation.gpu true` | -| `env.simulation.steps` (int, 240) | `--steps 500` (shortcut) or `--env.simulation.steps 500` | -| `env.simulation.dt` (float, 0.01) | `--dt 0.005` (shortcut) or `--env.simulation.dt 0.005` | -| `env.rewards_cfg.track_lin_vel.weight` (float, 1.0) | `--env.rewards_cfg.track_lin_vel.weight 2.0` | -| `env.rewards_cfg.track_lin_vel.params.std` (float, 0.25) | `--env.rewards_cfg.track_lin_vel.params.std 0.5` | -| `env.scene.env_spacing` (tuple, `(2.0, 2.0)`) | `--env.scene.env_spacing 3.0,3.0` | -| `env.scene.batch_render` (bool, false) | `--env.scene.batch_render true` | - -The four simulation shortcuts (`--vis`, `--gpu`, `--steps`, `--dt`) rewrite to -the `env.simulation.{vis,gpu,steps,dt}` paths before forwarding, so they show -up in profiler-trace argv exactly like an explicit override would. - -## Examples +| Metadata | Registered name, description, cfg type, examples. | +| Task config | `TaskCfg` summary: env, robot, trainable flag, agent. | +| Overridable paths | Dotted paths accepted by `play` and `train`. | -A short discovery loop for the inverted-pendulum task: +## Use copied override paths ```bash -# Confirm the extension is installed and pick a task id. -uv run genelab list tasks | grep Pendulum - -# Drill into one task — copy a dotted path from the "Overridable cfg paths" table. -uv run genelab info GeneLab-Inverted-Pendulum-v0 - -# Run the task with overrides applied. -uv run genelab play GeneLab-Inverted-Pendulum-v0 --vis --steps 240 \ - --env.rewards_cfg.pole_upright.weight 5.0 +uv run genelab play GeneLab-Inverted-Pendulum-v0 \ + --env.rewards_cfg.pole_upright.weight 4.0 ``` -The same flow works against envs and robots: `genelab info FrankaPandaCfg` (a -robot) reveals every `ArticulationCfg` field including the per-group -`actuators.*.stiffness` and `default_joint_pos.*` keys. +When `play_env` exists, play-mode shortcut flags target `play_env`; explicit paths remain explicit. ## See also -- [Play and Train](play-train.md) -- [Configs](../concepts/configs.md) -- [Managers and MDP terms](../concepts/managers.md) +- [Configuration Reference](../reference/configuration.md) +- [CLI Reference](../reference/cli.md) diff --git a/docs/cli/list-info.zh.md b/docs/cli/list-info.zh.md index 4601fa6c..2003eeac 100644 --- a/docs/cli/list-info.zh.md +++ b/docs/cli/list-info.zh.md @@ -1,13 +1,8 @@ # 发现：list 与 info -`list` 与 `info` 子命令是 GeneLab 的发现入口 —— 它们把已安装的扩展打印成一张 -"可用 task / env / robot + 可覆盖 dotted path" 的地图，供 `play` / `train` 的 -`-- VALUE` 旗标使用。 +用 `list` 查看扩展注册了什么。用 `info` 检查单个注册对象，并复制 override 路径。 -## 列举注册表 - -`list KIND` 枚举三个全局注册表之一（`robots` / `envs` / `tasks`）。命令会先导入所 -有已发现的扩展，因此列表反映当前安装集。 +## 列出注册表 ```bash uv run genelab list robots @@ -15,76 +10,37 @@ uv run genelab list envs uv run genelab list tasks ``` -每行打印注册名、说明，以及一条逐条 detail：task 显示 `env=..., robot=..., -trainable=...`；env / robot 显示首批几个标量 cfg 字段。通过 -`register_task(..., examples=[...])` 注册了 `examples=[...]` 字符串的 task 会附带 -`(N example(s))` 后缀。 - -## 查看任务 - -`info NAME` 解析单个注册名并打印 detail 面板。查找顺序是 task → env → robot。 -TTY 下名字找不到时会唤起 `questionary` 选择器，从 task / env / robot 名的并集 -中挑一个；非交互环境下沿用原行为，直接退出并列出可用名。 +如果扩展尚未安装，显式导入： ```bash -uv run genelab info GeneLab-Sensors-Showcase-v0 +PYTHONPATH=examples/inverted_pendulum/src \ + uv run genelab --import genelab_inverted_pendulum.tasks list tasks ``` -Task 面板分三块： - -1. **Description 与 meta** —— 一行 description，加 task cfg 上拉出的 - `env=..., robot=..., trainable=...` 元信息条。 -2. **Overridable cfg paths** —— 三列表格（`Path` / `Type` / `Default`）， - 通过深度优先遍历 cfg dataclass 树生成。以 `_` 开头的字段名被跳过；嵌套 - dataclass 会以父节点 dotted path 为前缀就地展开。 -3. **Examples** —— 扩展在注册时提供的原文 `examples=[...]` 行。仅展示， - `info` 不会执行它们。 - -env 与 robot 渲染同样的面板，但没有 `env / robot / trainable` 元信息条。 +## 查看任务 -## Override 路径 +```bash +uv run genelab info GeneLab-Inverted-Pendulum-v0 +``` -"Overridable cfg paths" 表里的每一行就是 `play` / `train` 能直接接受的 dotted -path。CLI 把每个 `-- VALUE` 旗标转发给 -`genelab.configs.apply_overrides`，后者在 cfg 构建期按目标字段类型 hint 做 coerce。 +任务视图包括： -| `info` 输出行 | 等价的 override 旗标 | +| 区域 | 含义 | |---|---| -| `env.simulation.vis` (bool, false) | `--vis`（捷径）或 `--env.simulation.vis true` | -| `env.simulation.gpu` (bool, false) | `--gpu`（捷径）或 `--env.simulation.gpu true` | -| `env.simulation.steps` (int, 240) | `--steps 500`（捷径）或 `--env.simulation.steps 500` | -| `env.simulation.dt` (float, 0.01) | `--dt 0.005`（捷径）或 `--env.simulation.dt 0.005` | -| `env.rewards_cfg.track_lin_vel.weight` (float, 1.0) | `--env.rewards_cfg.track_lin_vel.weight 2.0` | -| `env.rewards_cfg.track_lin_vel.params.std` (float, 0.25) | `--env.rewards_cfg.track_lin_vel.params.std 0.5` | -| `env.scene.env_spacing` (tuple, `(2.0, 2.0)`) | `--env.scene.env_spacing 3.0,3.0` | -| `env.scene.batch_render` (bool, false) | `--env.scene.batch_render true` | - -四个 simulation 捷径（`--vis` / `--gpu` / `--steps` / `--dt`）在转发前会被改写成 -`env.simulation.{vis,gpu,steps,dt}` 形式，因此 profiler trace argv 中显示的形态 -与显式 override 完全一致。 - -## 示例命令清单 +| 元信息 | 注册名、描述、cfg 类型、示例命令。 | +| 任务配置 | `TaskCfg` 摘要：env、robot、trainable、agent。 | +| 可覆盖路径 | `play` 和 `train` 接受的 dotted path。 | -针对 inverted-pendulum 任务的发现流程： +## 使用复制出的 override 路径 ```bash -# 确认扩展已安装并挑一个 task id。 -uv run genelab list tasks | grep Pendulum - -# 钻入单 task —— 从 "Overridable cfg paths" 表里复制一个 dotted path。 -uv run genelab info GeneLab-Inverted-Pendulum-v0 - -# 带 override 跑 task。 -uv run genelab play GeneLab-Inverted-Pendulum-v0 --vis --steps 240 \ - --env.rewards_cfg.pole_upright.weight 5.0 +uv run genelab play GeneLab-Inverted-Pendulum-v0 \ + --env.rewards_cfg.pole_upright.weight 4.0 ``` -同样的流程对 env 与 robot 也成立：`genelab info FrankaPandaCfg`（robot）会列出 -所有 `ArticulationCfg` 字段，包括 per-group 的 `actuators.*.stiffness` 与 -`default_joint_pos.*` 键。 +当 task 有 `play_env` 时，play 模式短标志会作用到 `play_env`；显式路径仍保持显式语义。 -## See also +## 另见 -- [Play and Train](play-train.md) -- [Configs](../concepts/configs.md) -- [Managers and MDP terms](../concepts/managers.md) +- [配置参考](../reference/configuration.md) +- [CLI 参考](../reference/cli.md) diff --git a/docs/cli/overview.en.md b/docs/cli/overview.en.md index 284b8862..7db8dcff 100644 --- a/docs/cli/overview.en.md +++ b/docs/cli/overview.en.md @@ -1,75 +1,50 @@ -# CLI overview +# CLI Overview -`genelab` is exposed as a console script via the `genelab = "genelab.cli:main"` entry point. -With `uv` the canonical invocation is: +The `genelab` CLI is a thin dispatcher over registries and task configs. It does not own task logic; +it discovers extensions, resolves a registered object, applies overrides, and calls the task or +runner. + +## Command model ```bash -uv run genelab [GLOBAL OPTIONS] [ARGS] +uv run genelab [global options] [arguments] ``` -## Subcommands - -| Subcommand | Purpose | -|------------|---------| -| `cache` | Create project-local simulation cache directories (`.cache/`) and set `XDG_CACHE_HOME` / `MPLCONFIGDIR`. | -| `list robots` | List registered robots from the `ROBOTS` registry. | -| `list envs` | List registered environments from the `ENVS` registry. | -| `list tasks` | List registered tasks from the `TASKS` registry. | -| `play` | Run a registered task in the configured Genesis backend; supports config overrides. | -| `train` | Train a registered task when an RL runner exists; supports multi-GPU via `torchrun`. | -| `prof open` | Launch TensorBoard against a `torch.profiler` trace directory. | -| `project new` | Scaffold a new external extension package with all three registries wired up. | - -## Global options - -The following flags accept any position in front of the subcommand: +| Area | Commands | +|---|---| +| Registry discovery | `list robots`, `list envs`, `list tasks`, `info NAME` | +| Runtime | `play TASK`, `train TASK` | +| Utilities | `cache`, `prof open` | +| Project scaffolding | `project new NAME` | -| Flag | Effect | -|------|--------| -| `--version` | Print the GeneLab version and exit. | -| `--import MODULE` | Eagerly import an extension module before dispatching. Repeatable. Useful for extensions that do not (yet) ship a `genelab.extensions` entry point. | -| `--no-entry-points` | Skip auto-discovery via the `genelab.extensions` entry-point group. Combined with `--import`, produces a fully explicit, reproducible loading order. | +## Extension loading order -## Extension discovery order +Every command that needs registry data loads extensions in this order: -On startup the CLI discovers extensions through three pathways, in order: entry-point -auto-discovery, explicit `--import MODULE` flags, then programmatic -`genelab.registry.load_extension_module(...)` calls. +1. Bundled asset zoo robots through `load_bundled_asset_zoo()`. +2. Installed `genelab.extensions` entry points, unless `--no-entry-points` is set. +3. Repeated explicit `--import MODULE` values. -## Shell completion +Use entry points for daily work and `--import` for local experiments. -`--install-completion` writes a completion script into the current shell's rc file -(bash, zsh, fish, PowerShell); `--show-completion` prints it to stdout for manual -installation. After installation, tab-completing `genelab info ` cycles through -every registered task, env, and robot name; `genelab play ` and -`genelab train ` cycle through task names; `genelab list ` offers -`robots / envs / tasks`. +## Overrides -!!! note "Entry-point extensions only" +Runtime commands accept unknown `--a.b.c VALUE` options after the task id. The CLI forwards them to +`apply_overrides`. - Completion callbacks load extensions through the `genelab.extensions` - entry-point group. Ad-hoc `--import MODULE` registrations do not appear - in the completion list — the shell strips global flags from the argv it - hands the callback, so the imported module list is invisible at that - point. - -## Interactive recovery +```bash +uv run genelab play TASK_ID --env.simulation.dt 0.005 +``` -When stdin is a TTY, four user-input mistakes fall back to a `questionary` -picker instead of exiting with a one-line error: +Use `genelab info TASK_ID` to list valid paths. -- `play` / `train` invoked with no task id, or with one that does not match - any registered task. -- `info NAME` with an unknown name. -- `--agent KIND` with a value other than `zero` / `random` / `trained`. -- `--` override paths that do not exist on the resolved task's cfg. +## Interactive mode -Outside a TTY — CI, pipes, scripts, pytest — the picker no-ops and the -original error surfaces unchanged, so non-interactive callers observe -identical behavior to a build without the recovery layer. +When stdin is a TTY, the CLI can prompt for a missing task id, unknown registry name, invalid +`--agent`, or unknown override path. In CI and scripts it raises the same errors directly. ## See also +- [CLI Reference](../reference/cli.md) +- [Discovery: list and info](list-info.md) - [Play and Train](play-train.md) -- [Project new](project-new.md) -- [Extensions](../concepts/extensions.md) diff --git a/docs/cli/overview.zh.md b/docs/cli/overview.zh.md index 2857241b..225ea095 100644 --- a/docs/cli/overview.zh.md +++ b/docs/cli/overview.zh.md @@ -1,68 +1,47 @@ # CLI 总览 -`genelab` 通过 `genelab = "genelab.cli:main"` entry point 暴露为命令行脚本。配合 `uv` 时 -通常这样调用： +`genelab` CLI 是注册表和任务配置上的薄调度层。它不拥有任务逻辑；它负责发现扩展、解析已注册对象、应用 override，然后调用 task 或 runner。 + +## 命令模型 ```bash -uv run genelab [全局选项] <子命令> [参数] +uv run genelab [全局选项] <命令> [参数] ``` -## 子命令 - -| 子命令 | 作用 | -|--------|------| -| `cache` | 创建项目本地仿真缓存目录（`.cache/`），并设置 `XDG_CACHE_HOME` / `MPLCONFIGDIR`。 | -| `list robots` | 列出 `ROBOTS` 注册表里已注册的机器人。 | -| `list envs` | 列出 `ENVS` 注册表里已注册的环境。 | -| `list tasks` | 列出 `TASKS` 注册表里已注册的任务。 | -| `play` | 在 Genesis 后端运行已注册任务，支持配置 override。 | -| `train` | 任务带 RL runner 时启动训练，通过 `torchrun` 支持多 GPU。 | -| `prof open` | 针对 `torch.profiler` 跟踪目录启动 TensorBoard。 | -| `project new` | 生成一个新的下游扩展包骨架，三种注册表全部接通。 | - -## 全局选项 - -下列标志放在任意子命令之前： +| 区域 | 命令 | +|---|---| +| 注册表发现 | `list robots`、`list envs`、`list tasks`、`info NAME` | +| 运行时 | `play TASK`、`train TASK` | +| 工具 | `cache`、`prof open` | +| 项目骨架 | `project new NAME` | -| 标志 | 作用 | -|------|------| -| `--version` | 打印 GeneLab 版本并退出。 | -| `--import MODULE` | 在派发子命令前显式导入扩展模块。可重复。适合扩展尚未提供 `genelab.extensions` entry point 时使用。 | -| `--no-entry-points` | 跳过通过 `genelab.extensions` entry-point 组的自动发现。与 `--import` 搭配可实现完全显式、可复现的扩展加载。 | +## 扩展加载顺序 -## 扩展发现顺序 +任何需要注册表数据的命令都会按顺序加载： -CLI 启动时按三条路径依次发现扩展：entry-point 自动发现、显式 `--import MODULE`、程序内的 -`genelab.registry.load_extension_module(...)`。 +1. 通过 `load_bundled_asset_zoo()` 加载随包 asset zoo robot。 +2. 已安装的 `genelab.extensions` entry point，除非设置 `--no-entry-points`。 +3. 重复传入的显式 `--import MODULE`。 -## Tab 补全 { #tab-completion } +日常工作用 entry point，本地实验用 `--import`。 -`--install-completion` 会把补全脚本写入当前 shell（bash / zsh / fish / PowerShell）的 -rc 文件；`--show-completion` 则把脚本打印到 stdout，便于手动安装。安装完成后，tab 补全 -`genelab info ` 会列出每一个已注册的 task / env / robot 名；`genelab play ` -与 `genelab train ` 列出已注册 task；`genelab list ` 列出 -`robots / envs / tasks`。 +## Overrides -!!! note "仅覆盖 entry-point 扩展" +运行时命令在 task id 后接受未知的 `--a.b.c VALUE` 选项。CLI 会把它们转发给 +`apply_overrides`。 - 补全回调通过 `genelab.extensions` entry-point 组加载扩展。临时 - `--import MODULE` 注册的扩展不会出现在补全列表里 —— shell 在调用补全回调时 - 已经把全局 `--import` 标志从 argv 里剥离，回调看不到这部分模块。 - -## 交互式回退 { #interactive-recovery } +```bash +uv run genelab play TASK_ID --env.simulation.dt 0.005 +``` -stdin 为 TTY 时，下列四种输入错误会回退到 `questionary` 选择器，而不是直接报错退出： +用 `genelab info TASK_ID` 查看有效路径。 -- `play` / `train` 没传 task id，或 task id 没注册。 -- `info NAME` 名字找不到。 -- `--agent KIND` 不是 `zero` / `random` / `trained`。 -- `--` override 路径在已解析任务的 cfg 上不存在。 +## 交互模式 -非 TTY 环境（CI、管道、脚本、pytest）下选择器自动 no-op，错误以原始形态抛出， -非交互调用者观察到的行为与未加回退层时完全一致。 +stdin 是 TTY 时，CLI 可以为缺失 task id、未知注册表名字、非法 `--agent`、未知 override 路径弹出选择器。CI 和脚本中会直接抛出相同错误。 -## See also +## 另见 +- [CLI 参考](../reference/cli.md) +- [发现：list 与 info](list-info.md) - [play 与 train](play-train.md) -- [新建项目](project-new.md) -- [扩展加载](../concepts/extensions.md) diff --git a/docs/cli/play-train.en.md b/docs/cli/play-train.en.md index e01e74c1..2e5d2b88 100644 --- a/docs/cli/play-train.en.md +++ b/docs/cli/play-train.en.md @@ -1,96 +1,58 @@ # Play and Train -`play` and `train` share a common dispatch path: resolve `` against the `TASKS` -registry, construct its `TaskCfg`, apply any command-line overrides, and hand the configured -task to either a single rollout (`play`) or the task's RL runner (`train`). +`play` runs a registered task. `train` runs a registered task through a supported runner when the +task provides an agent config. ## Play ```bash -uv run genelab play [SHORT FLAGS] [-- OVERRIDES] +uv run genelab play TASK_ID --steps 128 +uv run genelab play TASK_ID --vis --steps 500 +uv run genelab play TASK_ID --agent random --steps 128 ``` -Short flags may appear before or after ``; the CLI normalises the order internally. +Policy sources: -!!! tip "Interactive recovery" +| Agent | Behavior | +|---|---| +| `zero` | Zero actions. Default when no checkpoint is given. | +| `random` | Uniform random actions in `[-1, 1]`. | +| `trained` | Load a checkpoint and use the runner inference policy. | - Omitting `` on a TTY opens a `questionary` picker over the - registered tasks. Unknown task ids, malformed `--agent` values, and - `--` override paths that do not exist on the resolved cfg fall - back to the same picker. Non-TTY callers — CI, pipes, scripts — see the - original error messages unchanged. - -### Simulation shortcuts - -The following short flags rewrite to the corresponding `env.simulation.*` overrides: - -| Flag | Equivalent override | Effect | -|------|--------------------|--------| -| `--vis` | `env.simulation.vis=true` | Enable Genesis visualization. | -| `--gpu N` | `env.simulation.gpu=N` | Pin the rollout to a single GPU index. | -| `--steps N` | `env.simulation.steps=N` | Cap episode length to `N` steps. | - -### Runner flags - -When the task carries an RL agent config, `play` also accepts the runner-side flags: - -| Flag | Effect | -|------|--------| -| `--checkpoint PATH` | Load a trained policy and run inference rollouts. Forces `--agent` to default to `trained`. | -| `--num-envs N` | Override the registered env count (parallel environments). | -| `--agent {zero,random,trained}` | Pick the policy source. Defaults to `trained` when `--checkpoint` is set, otherwise `zero`. | - -### Override grammar - -After the short flags, any `-- VALUE` argument is parsed as a config override: +Checkpoint replay: ```bash -uv run genelab play \ - --env.simulation.dt 0.005 \ - --env.actions.scale 0.5 \ - --env.observations.include_velocity true +uv run genelab play TASK_ID \ + --checkpoint logs/rsl_rl///model_300.pt ``` -The dotted path walks the `TaskCfg` dataclass tree (typically rooted at `env.*`). Values are -coerced from string using the field's type hint — `int`, `float`, `bool`, `Path`, `list[...]`, -`tuple[...]` are all supported. - ## Train ```bash -uv run genelab train [SHORT FLAGS] [-- OVERRIDES] +uv run genelab train TASK_ID --num_envs 4096 --max_iterations 300 ``` -Train requires the registered task to expose an RL runner (commonly `rsl_rl_lib`). The -override syntax is identical to `play`. +For distributed training: -| Flag | Effect | -|------|--------| -| `--gpus N` | Dispatch via `torchrun --standalone --nproc_per_node=N` for distributed training. The task's runner must be `torchrun`-compatible. | -| `--checkpoint PATH` | Resume training from a checkpoint file. | -| `--num-envs N` | Override the registered env count (parallel environments). | -| `--max-iterations N` | Cap the number of PPO learning iterations. | -| `--seed N` | Override the RNG seed used by the runner. | -| `--log-dir PATH` | Override the log root; defaults to `logs///`. | -| `--agent {zero,random,trained}` | Forwarded to the runner; mostly useful for diagnostic runs. | - -When `--gpus N > 1`, the CLI masks `CUDA_VISIBLE_DEVICES` to the requested devices so each rank -sees a distinct GPU. +```bash +uv run genelab train TASK_ID --gpus 4 --num_envs 4096 +``` -## Examples +`--num_envs` is total across ranks and must divide evenly by `--gpus`. Use +`--num_envs_per_gpu` for per-rank semantics. -```bash -# Quick local rollout with visualization. -uv run genelab play wuji_hand --vis --steps 200 +## Config overrides -# Train on 4 GPUs with a custom action scale. -uv run genelab train wuji_hand --gpus 4 --env.actions.scale 0.3 +Any unknown option after the task id is treated as a dotted config override: -# Resume from a checkpoint. -uv run genelab train wuji_hand --checkpoint logs/wuji_hand/run_42/model_100.pt +```bash +uv run genelab play TASK_ID \ + --env.simulation.dt 0.005 \ + --env.rewards_cfg.action_rate.weight -0.01 ``` ## See also -- [Configs](../concepts/configs.md) -- [Registry](../concepts/registry.md) +- [Run RL Experiments](../best-practices/rl-experiments.md) +- [CLI Reference](../reference/cli.md) +- [RL runner](../concepts/rl-runner.md) diff --git a/docs/cli/play-train.zh.md b/docs/cli/play-train.zh.md index d0a82247..1da655ad 100644 --- a/docs/cli/play-train.zh.md +++ b/docs/cli/play-train.zh.md @@ -1,92 +1,57 @@ # play 与 train -`play` 与 `train` 共享同一条派发路径：在 `TASKS` 注册表查找 ``，构造其 `TaskCfg`， -应用命令行 override，然后交给单次 rollout（`play`）或任务自带的 RL runner（`train`）。 +`play` 运行已注册任务。`train` 在 task 提供 agent 配置时，通过支持的 runner 训练任务。 -## play +## Play ```bash -uv run genelab play [短标志] [-- 覆盖项] +uv run genelab play TASK_ID --steps 128 +uv run genelab play TASK_ID --vis --steps 500 +uv run genelab play TASK_ID --agent random --steps 128 ``` -短标志可放在 `` 前或后；CLI 内部会自动整理顺序。 +策略来源： -!!! tip "交互式回退" +| Agent | 行为 | +|---|---| +| `zero` | 零动作。未提供 checkpoint 时默认。 | +| `random` | `[-1, 1]` 均匀随机动作。 | +| `trained` | 加载 checkpoint 并使用 runner inference policy。 | - 在 TTY 下省略 `` 会唤起 `questionary` 选择器，从已注册 task 中 - 挑一个。task id 拼错、`--agent` 值不合法、`--` override 路径在 - 已解析 cfg 上不存在，都会回退到同一个选择器。非 TTY 调用者（CI、管道、 - 脚本）看到的报错与未启用回退层时一致。 +checkpoint 回放： -### 仿真短标志 - -下列短标志会被改写为对应的 `env.simulation.*` override： - -| 标志 | 等价 override | 作用 | -|------|--------------|------| -| `--vis` | `env.simulation.vis=true` | 开启 Genesis 可视化。 | -| `--gpu N` | `env.simulation.gpu=N` | 把 rollout 锁定到指定 GPU。 | -| `--steps N` | `env.simulation.steps=N` | 限制 episode 步数。 | - -### 运行时标志 - -任务携带 RL agent 配置时，`play` 还接受下列 runner 侧标志： - -| 标志 | 作用 | -|------|------| -| `--checkpoint PATH` | 加载已训练策略并做推理 rollout；会让 `--agent` 默认为 `trained`。 | -| `--num-envs N` | 覆盖任务注册时的 env 数量（并行环境数）。 | -| `--agent {zero,random,trained}` | 选择策略来源。设置 `--checkpoint` 时默认 `trained`，否则默认 `zero`。 | - -### Override 语法 +```bash +uv run genelab play TASK_ID \ + --checkpoint logs/rsl_rl///model_300.pt +``` -短标志之后的任意 `-- VALUE` 都会被当作配置 override： +## Train ```bash -uv run genelab play \ - --env.simulation.dt 0.005 \ - --env.actions.scale 0.5 \ - --env.observations.include_velocity true +uv run genelab train TASK_ID --num_envs 4096 --max_iterations 300 ``` -点路径沿 `TaskCfg` 的 dataclass 树（通常以 `env.*` 为根）下钻。字符串值按目标字段类型注解 -自动转换 —— 支持 `int`、`float`、`bool`、`Path`、`list[...]`、`tuple[...]`。 - -## train +分布式训练： ```bash -uv run genelab train [短标志] [-- 覆盖项] +uv run genelab train TASK_ID --gpus 4 --num_envs 4096 ``` -`train` 需要任务暴露一个 RL runner（通常是 `rsl_rl_lib`）。Override 语法与 `play` 一致。 - -| 标志 | 作用 | -|------|------| -| `--gpus N` | 通过 `torchrun --standalone --nproc_per_node=N` 启动分布式训练。任务自身的 runner 必须支持 `torchrun`。 | -| `--checkpoint PATH` | 从 checkpoint 文件继续训练。 | -| `--num-envs N` | 覆盖任务注册时的 env 数量（并行环境数）。 | -| `--max-iterations N` | 限制 PPO 学习迭代次数。 | -| `--seed N` | 覆盖 runner 使用的随机种子。 | -| `--log-dir PATH` | 覆盖日志根目录，默认 `logs///`。 | -| `--agent {zero,random,trained}` | 转发给 runner；多用于诊断式运行。 | +`--num_envs` 表示所有 rank 的总数，必须能被 `--gpus` 整除。每 rank 语义用 +`--num_envs_per_gpu`。 -当 `--gpus N > 1` 时，CLI 还会按所选设备掩码 `CUDA_VISIBLE_DEVICES`，让每个 rank 看到独立 -GPU。 +## 配置 override -## 示例 +task id 后的任何未知选项都会被当作 dotted config override： ```bash -# 本地可视化跑一遍。 -uv run genelab play wuji_hand --vis --steps 200 - -# 4 GPU 训练，并修改 action scale。 -uv run genelab train wuji_hand --gpus 4 --env.actions.scale 0.3 - -# 从 checkpoint 继续。 -uv run genelab train wuji_hand --checkpoint logs/wuji_hand/run_42/model_100.pt +uv run genelab play TASK_ID \ + --env.simulation.dt 0.005 \ + --env.rewards_cfg.action_rate.weight -0.01 ``` -## See also +## 另见 -- [配置系统](../concepts/configs.md) -- [注册表](../concepts/registry.md) +- [运行 RL 实验](../best-practices/rl-experiments.md) +- [CLI 参考](../reference/cli.md) +- [RL runner](../concepts/rl-runner.md) diff --git a/docs/cli/profiling.en.md b/docs/cli/profiling.en.md index 975683c2..348e9c31 100644 --- a/docs/cli/profiling.en.md +++ b/docs/cli/profiling.en.md @@ -1,64 +1,47 @@ # Profiling -GeneLab ships a thin wrapper around `torch.profiler` for measuring per-step cost on both -training and inference. The profiler is opt-in: `play` and `train` accept `--prof` flags, -and the matching `GENELAB_PROFILE` environment variables serve as a fallback. Traces are -written in TensorBoard format. +GeneLab forwards profiler flags to `torch.profiler` through `genelab.rl.maybe_profile`. -## Enabling the profiler - -Both commands accept the same flags. A CLI flag wins over the env var when both are set. +## Enable profiling ```bash -uv run genelab train --prof --prof-out logs/myprof -GENELAB_PROFILE=1 GENELAB_PROFILE_OUT=logs/myprof uv run genelab train +uv run genelab train TASK_ID \ + --prof \ + --prof-active 3 \ + --prof-repeat 1 \ + --max_iterations 10 ``` -## Flags - -| Flag | Env-var equivalent | Effect | -|------|--------------------|--------| -| `--prof` | `GENELAB_PROFILE=1` | Turn the profiler on. | -| `--prof-out PATH` | `GENELAB_PROFILE_OUT` | Trace output directory (default `logs/torch_profile`). | -| `--prof-wait N` | `GENELAB_PROFILE_WAIT` | Steps skipped before each cycle (default `10`). | -| `--prof-warmup N` | `GENELAB_PROFILE_WARMUP` | Warmup steps inside the cycle (default `5`). | -| `--prof-active N` | `GENELAB_PROFILE_ACTIVE` | Steps actively recorded per cycle (default `10`). | -| `--prof-repeat N` | `GENELAB_PROFILE_REPEAT` | Cycle count (default `2`). | -| `--prof-record-shapes` | `GENELAB_PROFILE_RECORD_SHAPES=1` | Record tensor shapes for op-input attribution. | -| `--prof-with-stack` | `GENELAB_PROFILE_WITH_STACK=1` | Capture Python stack traces. High overhead. | - -## Schedule defaults - -A cycle covers `wait + warmup + active` steps, and `repeat` is the number of cycles. With -the defaults each cycle covers 25 steps and two cycles run before capture stops, so a -single `--prof` run records 20 active steps — enough to surface hot ops without producing -oversized traces. +Open traces: -For `train`, one step is one rollout call on `RslRlVecEnvWrapper.step`. For `play`, one -step is one policy/env iteration. To express the schedule in PPO iterations instead of -env steps, multiply `--prof-wait`, `--prof-warmup`, and `--prof-active` by -`num_steps_per_env`. +```bash +uv run genelab prof open logs/torch_profile +``` -## Viewing traces +## Flags -`genelab prof open` launches TensorBoard against a trace directory: +| Flag | Meaning | +|---|---| +| `--prof` | Enable profiling. | +| `--prof-out PATH` | Trace output directory. | +| `--prof-wait N` | Initial wait steps. | +| `--prof-warmup N` | Warmup steps before recording. | +| `--prof-active N` | Recorded steps per cycle. | +| `--prof-repeat N` | Number of cycles. | +| `--prof-record-shapes` | Record tensor shapes. | +| `--prof-with-stack` | Capture Python stacks. Higher overhead. | -```bash -uv run genelab prof open logs/torch_profile --port 6006 -``` +## Practical defaults -The command refuses to start when the directory does not exist, and prints an install -hint when `tensorboard` is not on `PATH`. +Start short. A profiler step is advanced once per env step in GeneLab's RL loop, so traces can grow +quickly on large vectorized runs. ## Distributed runs -!!! note "Rank-0 only" - Under torchrun, only rank 0 writes traces. Other ranks see `maybe_profile()` as a - no-op so workers do not overwrite each other's files. CLI flags propagate through the - torchrun relaunch unchanged. +Only the main process writes profiler traces. Keep `--prof-active` small when profiling distributed +training. ## See also -- [Play and Train](play-train.md) -- [Configs](../concepts/configs.md) -- [torch.profiler reference](https://pytorch.org/docs/stable/profiler.html) +- [Run RL Experiments](../best-practices/rl-experiments.md) +- [CLI Reference](../reference/cli.md) diff --git a/docs/cli/profiling.zh.md b/docs/cli/profiling.zh.md index 20f2f655..f03860de 100644 --- a/docs/cli/profiling.zh.md +++ b/docs/cli/profiling.zh.md @@ -1,59 +1,45 @@ # 性能剖析 -GeneLab 在 `torch.profiler` 之上提供了一层薄壳，用于测量训练与推理阶段每一步的开销。剖析 -默认关闭：`play` 与 `train` 通过 `--prof` 系列标志启用，对应的 `GENELAB_PROFILE` 环境变量 -作为后备。所有跟踪结果以 TensorBoard 格式写出。 +GeneLab 通过 `genelab.rl.maybe_profile` 把 profiler 标志转发给 `torch.profiler`。 -## 启用剖析器 - -两个命令接受同一组标志。当 CLI 标志与环境变量同时设置时，CLI 标志优先。 +## 启用 profiling ```bash -uv run genelab train --prof --prof-out logs/myprof -GENELAB_PROFILE=1 GENELAB_PROFILE_OUT=logs/myprof uv run genelab train +uv run genelab train TASK_ID \ + --prof \ + --prof-active 3 \ + --prof-repeat 1 \ + --max_iterations 10 ``` -## 参数 - -| 标志 | 等价环境变量 | 作用 | -|------|--------------|------| -| `--prof` | `GENELAB_PROFILE=1` | 启用剖析器。 | -| `--prof-out PATH` | `GENELAB_PROFILE_OUT` | 跟踪输出目录（默认 `logs/torch_profile`）。 | -| `--prof-wait N` | `GENELAB_PROFILE_WAIT` | 每个周期开始前跳过的步数（默认 `10`）。 | -| `--prof-warmup N` | `GENELAB_PROFILE_WARMUP` | 周期内的预热步数（默认 `5`）。 | -| `--prof-active N` | `GENELAB_PROFILE_ACTIVE` | 每周期实际记录的步数（默认 `10`）。 | -| `--prof-repeat N` | `GENELAB_PROFILE_REPEAT` | 周期数（默认 `2`）。 | -| `--prof-record-shapes` | `GENELAB_PROFILE_RECORD_SHAPES=1` | 记录张量形状，便于按 op 输入归因。 | -| `--prof-with-stack` | `GENELAB_PROFILE_WITH_STACK=1` | 抓取 Python 调用栈，开销较高。 | - -## 调度默认值 - -一个周期覆盖 `wait + warmup + active` 步，`repeat` 决定周期数。默认参数下每个周期覆盖 25 -步、共采集 2 个周期，单次 `--prof` 总计记录 20 个 active 步 —— 足以暴露热点 op，又不会 -生成过大的跟踪文件。 +打开 trace： -对 `train`，一步指 `RslRlVecEnvWrapper.step` 的一次 rollout 调用；对 `play`，一步指一次 -策略/环境迭代。若希望以 PPO 迭代而非环境步表达调度，将 `--prof-wait`、`--prof-warmup`、 -`--prof-active` 乘以 `num_steps_per_env` 即可。 +```bash +uv run genelab prof open logs/torch_profile +``` -## 查看跟踪结果 +## 参数 -`genelab prof open` 会针对跟踪目录启动 TensorBoard： +| 标志 | 含义 | +|---|---| +| `--prof` | 启用 profiling。 | +| `--prof-out PATH` | trace 输出目录。 | +| `--prof-wait N` | 初始等待步数。 | +| `--prof-warmup N` | 记录前 warmup 步数。 | +| `--prof-active N` | 每个周期记录的步数。 | +| `--prof-repeat N` | 周期数。 | +| `--prof-record-shapes` | 记录 tensor shape。 | +| `--prof-with-stack` | 捕获 Python stack，开销更高。 | -```bash -uv run genelab prof open logs/torch_profile --port 6006 -``` +## 实用默认值 -目录不存在时命令直接拒绝运行；若 `PATH` 中找不到 `tensorboard`，会给出安装提示。 +先跑短 profile。GeneLab 的 RL loop 每个 env step 推进一次 profiler step，大规模 vectorized run 的 trace 会增长很快。 ## 分布式运行 -!!! note "仅 rank 0 写出跟踪" - `torchrun` 下只有 rank 0 写出跟踪文件，其余 rank 上 `maybe_profile()` 退化为 no-op， - 避免多个 worker 互相覆盖。CLI 标志会原样穿过 torchrun 的重启流程。 +只有 main process 写 profiler trace。profile 分布式训练时保持 `--prof-active` 较小。 -## See also +## 另见 -- [play 与 train](play-train.md) -- [配置系统](../concepts/configs.md) -- [torch.profiler 参考](https://pytorch.org/docs/stable/profiler.html) +- [运行 RL 实验](../best-practices/rl-experiments.md) +- [CLI 参考](../reference/cli.md) diff --git a/docs/cli/project-new.en.md b/docs/cli/project-new.en.md index 63b43fee..25959001 100644 --- a/docs/cli/project-new.en.md +++ b/docs/cli/project-new.en.md @@ -1,7 +1,7 @@ -# Project new +# Project New -`genelab project new` scaffolds a downstream extension package — a self-contained Python -project that registers robots, environments, and tasks into GeneLab's global registries. +`genelab project new` creates a standalone extension package. Use it for downstream projects rather +than editing `src/genelab/`. ## Usage @@ -9,50 +9,38 @@ project that registers robots, environments, and tasks into GeneLab's global reg uv run genelab project new my_robot_project ``` -### Options +Options: -| Option | Default | Effect | -|--------|---------|--------| -| `--path PATH` | `./` | Output directory. Created if missing. | -| `--package NAME` | derived from project name | Python import name (e.g. `my_robot_project`). | -| `--task-id ID` | `/-v0` | The first task ID registered by the scaffold. | -| `--force` | off | Overwrite an existing target directory. Use with care. | +| Option | Description | +|---|---| +| `--path PATH`, `-p PATH` | Parent directory for the generated project. | +| `--package NAME` | Python package name. Defaults to normalized `NAME`. | +| `--task-id ID` | Initial task id. Defaults to `-Example-v0`. | +| `--force` | Overwrite scaffold files if the target exists. | -## Scaffold output +## Generated structure -``` +```text my_robot_project/ -├── pyproject.toml # with [project.entry-points."genelab.extensions"] ├── README.md -└── src/ - └── my_robot_project/ - ├── __init__.py # exposes register() entry-point callable - ├── config.py # task-specific dataclass plugged into TaskCfg.env - ├── robots.py # robot registrations - ├── envs.py # environment registrations - └── tasks.py # task registration (using --task-id) +├── pyproject.toml +└── src/my_robot_project/ + ├── __init__.py + ├── config.py + ├── envs.py + ├── robots.py + └── tasks.py ``` -The generated `pyproject.toml` declares: - -```toml -[project.entry-points."genelab.extensions"] -my_robot_project = "my_robot_project:register" -``` - -Once installed (`uv pip install -e ./my_robot_project`), the extension is discovered on the -next CLI invocation without any `--import` flag. - -## Post-scaffold workflow +## After scaffolding ```bash -cd my_robot_project -uv pip install -e . # install the extension into the GeneLab venv -uv run genelab list tasks # confirm the new task ID shows up -uv run genelab play --vis +uv pip install -e my_robot_project +uv run genelab list tasks +uv run genelab play MyRobotProject-Example-v0 --steps 3 ``` ## See also +- [Build an Extension Project](../best-practices/extension-projects.md) - [Extensions](../concepts/extensions.md) -- [Configs](../concepts/configs.md) diff --git a/docs/cli/project-new.zh.md b/docs/cli/project-new.zh.md index 4e0c3ec6..ad523964 100644 --- a/docs/cli/project-new.zh.md +++ b/docs/cli/project-new.zh.md @@ -1,7 +1,6 @@ # 新建项目 -`genelab project new` 生成一个下游扩展包骨架 —— 一个独立 Python 项目，把机器人、环境、 -任务注册进 GeneLab 的全局注册表。 +`genelab project new` 创建独立扩展包。下游项目应使用它，而不是直接修改 `src/genelab/`。 ## 用法 @@ -9,50 +8,38 @@ uv run genelab project new my_robot_project ``` -### 选项 +选项： -| 选项 | 默认 | 作用 | -|------|------|------| -| `--path PATH` | `./` | 输出目录，不存在时自动创建。 | -| `--package NAME` | 由项目名推导 | Python 导入名（如 `my_robot_project`）。 | -| `--task-id ID` | `/-v0` | 骨架注册的首个任务 ID。 | -| `--force` | 关闭 | 已存在目标目录时覆盖。慎用。 | +| 选项 | 说明 | +|---|---| +| `--path PATH`、`-p PATH` | 生成项目的父目录。 | +| `--package NAME` | Python 包名，默认由 `NAME` 规范化得到。 | +| `--task-id ID` | 初始 task id，默认 `-Example-v0`。 | +| `--force` | 目标已存在时覆盖骨架文件。 | -## 生成的目录结构 +## 生成结构 -``` +```text my_robot_project/ -├── pyproject.toml # 含 [project.entry-points."genelab.extensions"] ├── README.md -└── src/ - └── my_robot_project/ - ├── __init__.py # 暴露 register() entry-point 调用入口 - ├── config.py # 接入 TaskCfg.env 的任务专属 dataclass - ├── robots.py # 机器人注册 - ├── envs.py # 环境注册 - └── tasks.py # 任务注册（使用 --task-id） +├── pyproject.toml +└── src/my_robot_project/ + ├── __init__.py + ├── config.py + ├── envs.py + ├── robots.py + └── tasks.py ``` -生成的 `pyproject.toml` 声明： - -```toml -[project.entry-points."genelab.extensions"] -my_robot_project = "my_robot_project:register" -``` - -安装后（`uv pip install -e ./my_robot_project`），扩展在下次启动 CLI 时被自动发现，无需任何 -`--import` 标志。 - -## 后续步骤 +## 生成后 ```bash -cd my_robot_project -uv pip install -e . # 安装扩展到 GeneLab 的 venv -uv run genelab list tasks # 确认新任务 ID 出现 -uv run genelab play --vis +uv pip install -e my_robot_project +uv run genelab list tasks +uv run genelab play MyRobotProject-Example-v0 --steps 3 ``` -## See also +## 另见 +- [构建扩展项目](../best-practices/extension-projects.md) - [扩展加载](../concepts/extensions.md) -- [配置系统](../concepts/configs.md) diff --git a/docs/concepts/actuators.en.md b/docs/concepts/actuators.en.md index 58bd6700..b69275b8 100644 --- a/docs/concepts/actuators.en.md +++ b/docs/concepts/actuators.en.md @@ -1,92 +1,31 @@ # Actuators -`genelab.actuator` is the electromechanical layer between a manager-based RL policy and -Genesis. Each `ActuatorBase` owns a regex-matched slice of an articulation's actuated -joints and decides how the per-step joint position target reaches the simulator: either -through Genesis's implicit PD law, or through a Python-side torque computation pushed via -`control_dofs_force`. +Actuators are the layer between policy actions and Genesis joint control. They let a robot config +choose how each joint group is driven without changing the action term or task logic. -## Why a dedicated layer +## Why a dedicated actuator layer -Real robots are not monolithic — a single Unitree G1 carries six distinct motor classes -spread across knees, hips, shoulders, and ankles. Each class has different stiffness, -damping, effort, velocity, armature, and friction. Folding all of that into flat -`joint_kp` / `joint_kv` dicts on `ArticulationCfg` loses the per-motor identity and -prevents modelling velocity-dependent saturation. The actuator namespace restores the -group abstraction and adds three torque models. +Different robots need different control assumptions. A simple cart can use implicit PD targets; a +legged robot may need torque limits and DC motor saturation. GeneLab keeps those mechanics in +actuator configs attached to an `ArticulationCfg`. -## Three shipped models +## Built-in models -| Class | Channel | Torque computation | -|---|---|---| -| `ImplicitPDActuator` | `implicit_pd` | None — Genesis solves `tau = kp*(q* - q) - kv*q_dot` internally | -| `IdealPDActuator` | `force` | `tau = clip(kp*(q* - q) - kv*q_dot, ±effort_limit)` | -| `DCMotorActuator` | `force` | `IdealPD` plus linear de-rating in the driving direction | +| Model | Behavior | +|---|---| +| `ImplicitPDActuator` | Uses Genesis/simulator implicit PD control. | +| `IdealPDActuator` | Computes PD torque in Python and writes force targets. | +| `DCMotorActuator` | Extends ideal PD with motor limits and saturation behavior. | -`ImplicitPDActuator` is the numerical equivalent of the pre-M2 baseline — Genesis is -called with `set_dofs_kp` / `set_dofs_kv` and the per-step target reaches the engine -through `control_dofs_position`. `IdealPDActuator` zeroes the simulator-side PD and -becomes the canonical torque-control path for explicit effort limits. `DCMotorActuator` -adds a torque-speed curve: +Actuators match joint groups by configured names or expressions, then expose dimensions and control +logic to action terms. -`tau_max(q_dot) = saturation_effort * clip(1 - |q_dot| / velocity_limit, 0, 1)` +## Design guidance -The de-rating only applies to the driving direction (the sign of `tau_pd` matches the -sign of `q_dot`); reverse-braking torque retains the full `effort_limit`. This matches -the Isaac Lab `DCMotor` semantics — the back-EMF saturation does not penalise -regenerative braking. +Keep actuator grouping aligned with robot mechanics. Avoid one giant actuator if arm, hand, and base +joints need different gains, limits, or action scales. -## Composing onto an articulation +## Where to continue -`ArticulationCfg.actuators` is a `dict[str, ActuatorBaseCfg]`. Every actuated joint must -be covered by exactly one group; both unmatched and conflicting regex matches raise -`ValueError` at `Articulation.bind` time. Passive joints get explicit zero-gain -`ImplicitPDActuatorCfg` entries so the topology is visible in the config. - -```python -from genelab.actuator import DCMotorActuatorCfg, ImplicitPDActuatorCfg -from genelab.entity import ArticulationCfg - -cfg = ArticulationCfg( - mjcf_path="/path/to/robot.xml", - default_joint_pos={"cart_slide": 0.0, "pole_hinge": 0.0}, - actuators={ - "cart": ImplicitPDActuatorCfg( - target_names_expr=("cart_slide",), stiffness=80.0, damping=8.0, - action_scale=1.0, - ), - "pole": ImplicitPDActuatorCfg( - target_names_expr=("pole_hinge",), stiffness=0.0, damping=0.0, - ), - }, -) -``` - -`target_names_expr` is a tuple of regex patterns matched against the articulation's -joint names. Each group's `action_scale` is published through -`Articulation.action_scale_tensor`, which `JointPositionAction` uses by default when -`scale` is `None` on the action config. - -## Failure modes worth knowing - -* **Empty `actuators` dict** — the articulation rejects an articulation with no declared - actuators. Even a passive pendulum needs a zero-gain group covering its hinge. -* **Unmatched joint** — `ValueError` lists the joints that no regex covered. -* **Conflicting groups** — `ValueError` lists the joints that more than one group matched - along with the group names involved. -* **`DCMotorActuatorCfg.velocity_limit is None`** — the de-rating curve has no - breakpoint; `__post_init__` raises before the actuator gets a chance to run. - -## Switching G1 between models - -The bundled `examples/unitree` extension declares six `DCMotorActuatorCfg` groups (5020, -7520_14, 7520_22, 4010, waist, ankle). To run the same robot under the simulator's -implicit PD law instead, swap each `DCMotorActuatorCfg(...)` for an -`ImplicitPDActuatorCfg(...)` with the same `stiffness` / `damping` / `effort_limit`. The -articulation makes no global assumption about which channel is active — every group -chooses individually. - -## See also - -- [Configs](configs.md) -- [Sensors](sensors.md) +- [Task design](../best-practices/task-design.md) +- [API Reference](../api/reference.md) diff --git a/docs/concepts/actuators.zh.md b/docs/concepts/actuators.zh.md index 72d476f2..d5d553e1 100644 --- a/docs/concepts/actuators.zh.md +++ b/docs/concepts/actuators.zh.md @@ -1,62 +1,27 @@ # 执行器 -`genelab.actuator` 是 manager-based RL 策略与 Genesis 之间的机电层。每个 `ActuatorBase` 通过正则表达式占有一段 articulation 的可驱动关节，并决定单步关节位置目标如何抵达仿真器：要么走 Genesis 的隐式 PD 通道，要么在 Python 端算出力矩并通过 `control_dofs_force` 推下去。 +执行器是策略 action 与 Genesis joint control 之间的一层。它让机器人配置决定每组关节如何驱动， +而不需要修改 action term 或任务逻辑。 -## 为什么需要专门的一层 +## 为什么需要专门的执行器层 -真实机器人不是单一规格 —— 一台 Unitree G1 同时挂着 6 种不同电机，分布在膝、髋、肩、踝。各类电机的 stiffness、damping、effort、velocity、armature、friction 都不一样。把这些信息全压进 `ArticulationCfg` 上扁平的 `joint_kp` / `joint_kv` 字典会丢掉电机身份，也无法表达速度相关的力矩饱和。actuator 命名空间把分组抽象重新拿回来，并加上 3 种力矩模型。 +不同机器人需要不同控制假设。简单小车可以用隐式 PD 目标；腿式机器人可能需要力矩限制和 DC motor 饱和。GeneLab 把这些机制放在挂到 `ArticulationCfg` 的 actuator 配置里。 -## 自带 3 种模型 +## 内置模型 -| 类 | 通道 | 力矩计算 | -|---|---|---| -| `ImplicitPDActuator` | `implicit_pd` | 不算 —— Genesis 内部求解 `tau = kp*(q* - q) - kv*q_dot` | -| `IdealPDActuator` | `force` | `tau = clip(kp*(q* - q) - kv*q_dot, ±effort_limit)` | -| `DCMotorActuator` | `force` | `IdealPD` 加上仅驱动方向生效的线性退磁 | +| 模型 | 行为 | +|---|---| +| `ImplicitPDActuator` | 使用 Genesis/仿真器隐式 PD 控制。 | +| `IdealPDActuator` | 在 Python 中计算 PD torque，并写入 force target。 | +| `DCMotorActuator` | 在 ideal PD 基础上加入电机限制和饱和行为。 | -`ImplicitPDActuator` 与 M2 之前的实现数值等价 —— 对 Genesis 调 `set_dofs_kp` / `set_dofs_kv`，每步目标通过 `control_dofs_position` 进入仿真。`IdealPDActuator` 把仿真器内部 PD 增益置零，成为带显式 effort 上限的力矩控制规范路径。`DCMotorActuator` 在上面追加一条力矩-速度退磁曲线： +执行器按配置的关节名或表达式匹配关节组，并向 action term 暴露维度和控制逻辑。 -`tau_max(q_dot) = saturation_effort * clip(1 - |q_dot| / velocity_limit, 0, 1)` +## 设计建议 -退磁仅作用于驱动方向（`tau_pd` 与 `q_dot` 同号时）；反向制动力矩保持完整的 `effort_limit`。这是 Isaac Lab `DCMotor` 的语义 —— 反电动势饱和不惩罚回馈制动。 +执行器分组应贴合机器人机构。若手臂、手、底座关节需要不同增益、限制或 action scale，不要放进一个巨大的 actuator。 -## 装配到 articulation 上 +## 继续阅读 -`ArticulationCfg.actuators` 是 `dict[str, ActuatorBaseCfg]`。每一个可驱动关节都必须被恰好一个组覆盖；未匹配和正则冲突都会在 `Articulation.bind` 时抛 `ValueError`。被动关节也要写一条零增益 `ImplicitPDActuatorCfg`，让拓扑结构在 config 中可见。 - -```python -from genelab.actuator import DCMotorActuatorCfg, ImplicitPDActuatorCfg -from genelab.entity import ArticulationCfg - -cfg = ArticulationCfg( - mjcf_path="/path/to/robot.xml", - default_joint_pos={"cart_slide": 0.0, "pole_hinge": 0.0}, - actuators={ - "cart": ImplicitPDActuatorCfg( - target_names_expr=("cart_slide",), stiffness=80.0, damping=8.0, - action_scale=1.0, - ), - "pole": ImplicitPDActuatorCfg( - target_names_expr=("pole_hinge",), stiffness=0.0, damping=0.0, - ), - }, -) -``` - -`target_names_expr` 是一组正则模式，对 articulation 的关节名做匹配。每组的 `action_scale` 通过 `Articulation.action_scale_tensor` 暴露 —— 当 `JointPositionAction.scale` 留 `None` 时（推荐默认）就会从这里取值。 - -## 需要知道的失败模式 - -* **`actuators` 字典为空** —— articulation 拒绝完全没有声明 actuator 的配置。哪怕是被动单摆，也要为它的 hinge 写一条零增益组。 -* **未匹配关节** —— `ValueError` 列出未被任何正则覆盖的关节。 -* **冲突分组** —— `ValueError` 列出被多个组同时命中的关节及涉及的组名。 -* **`DCMotorActuatorCfg.velocity_limit` 为 `None`** —— 退磁曲线无断点；`__post_init__` 在 actuator 实际运行前先抛错。 - -## 在 G1 上切换模型 - -仓库自带的 `examples/unitree` 扩展声明了 6 个 `DCMotorActuatorCfg` 组（5020、7520_14、7520_22、4010、waist、ankle）。如果想让同一台机器人走仿真器隐式 PD，把每条 `DCMotorActuatorCfg(...)` 替换为同样 `stiffness` / `damping` / `effort_limit` 的 `ImplicitPDActuatorCfg(...)` 即可。articulation 不对哪个通道被激活做全局假设 —— 每个组各自决定。 - -## See also - -- [Configs](configs.md) -- [Sensors](sensors.md) +- [设计任务](../best-practices/task-design.md) +- [API 参考](../api/reference.md) diff --git a/docs/concepts/asset_zoo.en.md b/docs/concepts/asset_zoo.en.md index 0c76852d..56405279 100644 --- a/docs/concepts/asset_zoo.en.md +++ b/docs/concepts/asset_zoo.en.md @@ -1,137 +1,30 @@ -# Asset zoo +# Asset Zoo -`genelab.asset_zoo` ships curated robot configurations as a bundled extension of the -core package (not part of the `genelab.lab` facade), so `from genelab.asset_zoo import -CartpoleCfg` works out of the box. Each entry pairs a -declarative `AssetSpec` (URL + md5 + filename, optionally + `archive_member`) with a -lazy factory that returns a fresh `ArticulationCfg`. The factories trigger an -md5-verified download only when invoked, so read-only commands like -`genelab list robots` never touch the network. +The asset zoo is a curated bundle of robot configs and downloadable assets shipped with GeneLab. +It is useful for examples and smoke tests, but downstream projects should still own their robot +configs in their own packages. -## Why a curated zoo +## Bundled assets -Isaac Lab ships 40+ ready-to-use robot configurations and MjLab ships three; both burn a -significant fraction of their distribution size on bundled MJCF / URDF files. GeneLab -keeps the runtime install lean by separating the **configuration code** (in-tree, in -this package) from the **asset blobs** (off-tree, in the -[`KraHsu/genelab-assets`](https://github.com/KraHsu/genelab-assets) repository). The -first call to a robot factory downloads its MJCF once into the project-local cache and -every subsequent call resolves to the cached path. +| Name | Factory | +|---|---| +| Unitree G1 | `UnitreeG1Cfg` | +| Unitree Go1 | `UnitreeGo1Cfg` | +| ANYmal C | `AnymalCCfg` | +| Franka Panda | `FrankaPandaCfg` | +| Cartpole | `CartpoleCfg` | +| G1 LAFAN1 motion | `g1_lafan1_dance1_subject2()` | -## Lifecycle of an asset +Importing `genelab.asset_zoo` registers bundled robots into `ROBOTS`. Asset downloads stay lazy and +only happen when a factory needs the actual file. -1. A robot module (e.g. `asset_zoo/cartpole.py`) declares a module-level constant - `AssetSpec(name=..., url=..., md5=..., filename=...)`. -2. Import of `genelab.asset_zoo` calls `register_robot(name, factory, ...)` as a side - effect. The factory is the function `CartpoleCfg`, not the eager result. -3. `genelab list robots` enumerates registered names without invoking factories. -4. `ROBOTS.get("cartpole")` invokes the factory, which calls `fetch_asset(spec)`. On a - cold cache the file lands in `/.cache/assets/cartpole//cartpole.xml` - after md5 verification. On warm cache the path resolves immediately. -5. The factory returns a fresh `ArticulationCfg` so downstream callers can mutate - `init_pos`, `default_joint_pos`, etc. without affecting siblings. +## Cache and verification -## Five built-in robots +Downloaded assets live under the project cache. Asset helpers can verify checksums before returning +paths, so tasks can depend on stable local files after the first fetch. -| Name | Cfg factory | DoF | Actuator groups | -|---|---|---|---| -| `cartpole` | `CartpoleCfg()` | 2 | `cart` (active PD), `pole` (passive zero-gain) | -| `franka` | `FrankaPandaCfg()` | 9 | `panda_arm` (high-PD, k=400), `panda_hand` (stiff grasp, k=1e4) | -| `g1` | `UnitreeG1Cfg()` | 29 | `5020` / `7520_14` / `7520_22` / `4010` / `waist` / `ankle` — six DCMotor families | -| `go1` | `UnitreeGo1Cfg()` | 12 | `hip` / `thigh` / `calf` (ImplicitPD, k=25) | -| `anymal-c` | `AnymalCCfg()` | 12 | `legs` (single ImplicitPD group, k=80) | +## Where to continue -Use `genelab info ` to print the full override-path tree once registered. -Cartpole mirrors `examples/inverted_pendulum`'s gains; G1 mirrors -`examples/unitree/.../g1/constants.py`; Franka, Go1, and Anymal C follow Isaac Lab's -published defaults so cross-stack experiments stay comparable. The four Menagerie-sourced -entries (`franka`, `g1`, `go1`, `anymal-c`) ship as `.tar.gz` archives so meshes and -textures travel with the MJCF. - -## Cache layout and md5 verification - -Single-file mode (cartpole): - -``` -.cache/assets/// -``` - -Archive mode (franka, g1, go1, anymal-c): - -``` -.cache/assets///extracted/ -``` - -`fetch_asset` writes the download to a `..part` sibling and renames atomically -once the digest matches. Archive blobs go through a second atomic step: the tar is -expanded into a `.extracting/` sibling using `tarfile`'s `data` filter (rejects symlinks -and parent-directory escapes) and renamed to `extracted/` only on success. A mismatch -raises `AssetDownloadError` carrying both expected and actual digests; cached files -keyed by a stale md5 are auto-pruned at the next call with the corrected hash, since -the path itself moves. - -!!! warning "Update the md5 every time the asset moves" - The cache key is `(name, md5)`; bumping the md5 invalidates old copies cleanly. - Forgetting to bump it after re-uploading a fixed MJCF can leave clients reading a - stale local file forever. - -## Adding a new robot config - -Single-file MJCF (no external mesh dependencies): - -```python -_MJCF = AssetSpec( - name="my-robot", - url="https://raw.githubusercontent.com/KraHsu/genelab-assets/main/my-robot/my-robot.xml", - md5="<32 hex chars>", - filename="my-robot.xml", -) -``` - -Menagerie-style folder with meshes / textures (pack as `.tar.gz`, name the entry MJCF): - -```python -_MJCF = AssetSpec( - name="my-robot", - url="https://raw.githubusercontent.com/KraHsu/genelab-assets/main/my_robot/my_robot.tar.gz", - md5="<32 hex chars>", - filename="my_robot.tar.gz", - archive_member="my_robot/my_robot.xml", -) -``` - -Either way the factory is the same: - -```python -def MyRobotCfg() -> ArticulationCfg: - return ArticulationCfg( - mjcf_path=str(fetch_asset(_MJCF)), - actuators={"all": ImplicitPDActuatorCfg(target_names_expr=(r".*",), stiffness=100.0, damping=5.0)}, - ) - -register_robot("my-robot", MyRobotCfg, description="...", cfg_type=ArticulationCfg) -``` - -Then add the module to `asset_zoo/__init__.py` so import side-effects run during -`load_bundled_asset_zoo()`. Downstream projects that prefer to keep robots out-of-tree -should use the same `AssetSpec` + `register_robot` pattern from their own extension -package — the helpers are public. - -## Failure modes worth knowing - -* **Network unavailable** — `AssetDownloadError` wraps `URLError`; the staging file is - removed so a retry starts clean. -* **md5 mismatch** — raises `AssetDownloadError` with both digests; the staging file is - removed before the exception propagates. -* **Stale cache after upstream re-upload** — the path is keyed by md5, so updating the - spec naturally points at a fresh subdirectory; old digests linger on disk until the - user deletes `.cache/assets//` manually. -* **Factory called before `load_bundled_asset_zoo()`** — `ROBOTS.get("cartpole")` - raises `KeyError`; the CLI calls the loader at startup so this only bites direct API - use. Import `genelab.asset_zoo` explicitly to register without going through the CLI. - -## See also - -- [Registry](registry.md) -- [Actuators](actuators.md) -- [Extensions](extensions.md) +- [Module Map](../reference/module-map.md) +- [Unitree G1](../examples/unitree-g1.md) +- [API Reference](../api/reference.md) diff --git a/docs/concepts/asset_zoo.zh.md b/docs/concepts/asset_zoo.zh.md index 235accd4..a71e5a9e 100644 --- a/docs/concepts/asset_zoo.zh.md +++ b/docs/concepts/asset_zoo.zh.md @@ -1,98 +1,26 @@ # 资产库 -`genelab.asset_zoo` 把精选的机器人配置作为核心包的预置扩展一起发布（不属于 `genelab.lab` 公开门面），所以 `from genelab.asset_zoo import CartpoleCfg` 开箱即用。每个条目把一个声明式的 `AssetSpec`（URL + md5 + 文件名，可选 `archive_member`）和一个返回新 `ArticulationCfg` 的 lazy factory 配对。factory 仅在被调用时才触发带 md5 校验的下载，所以像 `genelab list robots` 这种只读命令不会触网。 +asset zoo 是 GeneLab 随包提供的一组机器人配置和可下载资产。它适合示例和 smoke test，但下游项目仍应在自己的包里维护机器人配置。 -## 为什么需要预置库 +## 随包资产 -Isaac Lab 自带 40+ 现成机器人配置，MjLab 自带 3 个；两者都把可观比例的发行体积花在内置 MJCF / URDF 文件上。GeneLab 把**配置代码**（在树内、本包内）和**资产二进制**（在树外，放在 [`KraHsu/genelab-assets`](https://github.com/KraHsu/genelab-assets) 仓库）分开，让运行时安装保持精简。第一次调用机器人 factory 会把 MJCF 下载一次到项目本地缓存，后续每次调用都解析到缓存路径。 +| 名称 | Factory | +|---|---| +| Unitree G1 | `UnitreeG1Cfg` | +| Unitree Go1 | `UnitreeGo1Cfg` | +| ANYmal C | `AnymalCCfg` | +| Franka Panda | `FrankaPandaCfg` | +| Cartpole | `CartpoleCfg` | +| G1 LAFAN1 motion | `g1_lafan1_dance1_subject2()` | -## 资产生命周期 +导入 `genelab.asset_zoo` 会把随包机器人注册进 `ROBOTS`。资产下载保持惰性，只有 factory 真正需要文件时才发生。 -1. 机器人模块（例如 `asset_zoo/cartpole.py`）在模块级别声明一个常量 `AssetSpec(name=..., url=..., md5=..., filename=...)`。 -2. 导入 `genelab.asset_zoo` 时副作用调用 `register_robot(name, factory, ...)`。factory 是函数 `CartpoleCfg`，不是立即求值的结果。 -3. `genelab list robots` 枚举已注册名称而不调用 factory。 -4. `ROBOTS.get("cartpole")` 调用 factory，factory 内调 `fetch_asset(spec)`。冷缓存时文件落到 `/.cache/assets/cartpole//cartpole.xml`，md5 校验通过后才生效。热缓存时路径立刻解析。 -5. factory 返回一个新的 `ArticulationCfg`，下游可以随意改 `init_pos`、`default_joint_pos` 等字段而不影响兄弟实例。 +## 缓存与校验 -## 自带 5 种机器人 +下载资产位于项目缓存目录。资产 helper 可以在返回路径前校验 checksum，因此首次获取后，任务能依赖稳定的本地文件。 -| 名称 | Cfg factory | DoF | actuator 分组 | -|---|---|---|---| -| `cartpole` | `CartpoleCfg()` | 2 | `cart`（带 PD）、`pole`（被动零增益） | -| `franka` | `FrankaPandaCfg()` | 9 | `panda_arm`（高 PD，k=400）、`panda_hand`（强抓握，k=1e4） | -| `g1` | `UnitreeG1Cfg()` | 29 | `5020` / `7520_14` / `7520_22` / `4010` / `waist` / `ankle` —— 6 个 DCMotor 家族 | -| `go1` | `UnitreeGo1Cfg()` | 12 | `hip` / `thigh` / `calf`（ImplicitPD，k=25） | -| `anymal-c` | `AnymalCCfg()` | 12 | `legs`（单 ImplicitPD 组，k=80） | +## 继续阅读 -注册完成后用 `genelab info ` 打印完整的 override 路径树。Cartpole 的增益与 `examples/inverted_pendulum` 保持一致；G1 与 `examples/unitree/.../g1/constants.py` 对齐；Franka、Go1 与 Anymal C 沿用 Isaac Lab 公开的默认值，方便跨栈对比实验。4 个 Menagerie 来源条目（`franka`、`g1`、`go1`、`anymal-c`）以 `.tar.gz` archive 形式发布，mesh 与纹理随 MJCF 一并发送。 - -## 缓存目录与 md5 校验 - -单文件模式（cartpole）： - -``` -.cache/assets/// -``` - -archive 模式（franka、g1、go1、anymal-c）： - -``` -.cache/assets///extracted/ -``` - -`fetch_asset` 先把下载内容写到 `..part` 兄弟文件，digest 校验通过后再原子 rename。archive 模式多一步原子语义：tar 先解到 `.extracting/` 临时目录（使用 `tarfile` 的 `data` filter，拒绝软链接与父目录逃逸），成功后再 rename 为 `extracted/`。不匹配会抛 `AssetDownloadError`，错误信息携带期望与实际两份 digest；旧 md5 对应的缓存目录会在下次用新 hash 调用时被自然绕过，因为路径本身改变了。 - -!!! warning "资产每次替换都要更新 md5" - 缓存 key 是 `(name, md5)`；更新 md5 可以干净失效旧副本。如果重新上传了修复版 MJCF 但忘了改 md5，客户端会一直读到陈旧的本地文件。 - -## 增加新的 robot 配置 - -单文件 MJCF（无外部 mesh 依赖）： - -```python -_MJCF = AssetSpec( - name="my-robot", - url="https://raw.githubusercontent.com/KraHsu/genelab-assets/main/my-robot/my-robot.xml", - md5="<32 hex chars>", - filename="my-robot.xml", -) -``` - -Menagerie 风格带 mesh / texture 的目录（打成 `.tar.gz`，声明入口 MJCF）： - -```python -_MJCF = AssetSpec( - name="my-robot", - url="https://raw.githubusercontent.com/KraHsu/genelab-assets/main/my_robot/my_robot.tar.gz", - md5="<32 hex chars>", - filename="my_robot.tar.gz", - archive_member="my_robot/my_robot.xml", -) -``` - -两种模式共用同一个 factory： - -```python -def MyRobotCfg() -> ArticulationCfg: - return ArticulationCfg( - mjcf_path=str(fetch_asset(_MJCF)), - actuators={"all": ImplicitPDActuatorCfg(target_names_expr=(r".*",), stiffness=100.0, damping=5.0)}, - ) - -register_robot("my-robot", MyRobotCfg, description="...", cfg_type=ArticulationCfg) -``` - -随后把模块加进 `asset_zoo/__init__.py`，让 `load_bundled_asset_zoo()` 触发导入副作用。下游项目若想把机器人放在树外，可以在自己的扩展包里用同样的 `AssetSpec` + `register_robot` 模式 —— 这两个 helper 都是公开 API。 - -## 需要知道的失败模式 - -* **网络不可达** —— `AssetDownloadError` 包住 `URLError`；staging 文件已删除，重试从干净状态开始。 -* **md5 不匹配** —— 抛 `AssetDownloadError`，错误带两份 digest；staging 文件在异常抛出前已删除。 -* **上游重新上传后缓存陈旧** —— 路径按 md5 键控，更新 spec 自然指到新子目录；旧 digest 对应的目录留在磁盘上直到用户手动删除 `.cache/assets//`。 -* **factory 在 `load_bundled_asset_zoo()` 之前被调** —— `ROBOTS.get("cartpole")` 抛 `KeyError`；CLI 启动时会调用 loader，所以这只有在直接用 API 时才会出现。需要绕过 CLI 时显式 `import genelab.asset_zoo` 即可注册。 - -## See also - -- [Registry](registry.md) -- [Actuators](actuators.md) -- [Extensions](extensions.md) +- [模块地图](../reference/module-map.md) +- [Unitree G1](../examples/unitree-g1.md) +- [API 参考](../api/reference.md) diff --git a/docs/concepts/configs.en.md b/docs/concepts/configs.en.md index 07805366..aeb59455 100644 --- a/docs/concepts/configs.en.md +++ b/docs/concepts/configs.en.md @@ -1,65 +1,42 @@ # Configs -The config system is a small dataclass hierarchy under `genelab.configs`: +GeneLab configuration is a dataclass tree. The tree is intentionally ordinary Python rather than a +custom schema language: task authors can construct it with normal code, and the CLI can still expose +stable override paths. -``` -TaskCfg -└── env: object # downstream extensions plug in their own env dataclass - ManagerBasedEnvCfg - ├── simulation - ├── scene - ├── actions - ├── observations - ├── rewards - ├── terminations - └── events -``` - -`TaskCfg.env` is typed `object` on purpose — downstream extensions plug their own env dataclass -into it without touching core. Most users subclass `ManagerBasedEnvCfg` for their `env` field. +## The core idea -## apply_overrides +`TaskCfg` is the boundary between discovery and runtime. It records the task id, the registered env +and robot names, the default env config, an optional play-mode env config, and an optional agent +config. -The headline function is `apply_overrides(cfg, dict)`, which parses dotted paths and applies -them to the config tree: - -```python -from genelab.configs import apply_overrides - -apply_overrides(cfg, { - "env.simulation.dt": "0.005", - "env.simulation.steps": "500", - "env.robot.actuators.cart.stiffness": "100.0", - "env.observations.include_velocity": "true", -}) +```text +TaskCfg +├── env +├── play_env +└── agent ``` -### Type coercion +`env` is typed as `object` on purpose. Downstream projects can use a subclass such as +`ManagerBasedRlEnvCfg` without changing GeneLab core. -String values are coerced using the field's type hint on the target dataclass. Supported -targets: +## Why dotted overrides work -| Hint | Accepted strings | -|------|------------------| -| `bool` | `true`/`false`/`1`/`0`/`yes`/`no` (case-insensitive) | -| `int` | any base-10 integer literal | -| `float` | any Python float literal | -| `Path` | passed through `pathlib.Path(...)` | -| `list[T]` | comma-separated, each element coerced as `T` | -| `tuple[T, ...]` | comma-separated, each element coerced as `T` | -| `str` | identity | +The CLI treats `--a.b.c VALUE` as a path through this dataclass tree. `apply_overrides` resolves the +path, reads the current value and annotation, coerces the string value, and mutates the config before +runtime starts. -When a path does not resolve to a known field or a value cannot be coerced, `apply_overrides` -raises a descriptive error at config build time, not at simulation runtime. +That keeps experiments reproducible: the final `TaskCfg` can be dumped, and invalid paths fail early +instead of silently being ignored. -### Forwarding from the CLI +## Play configs are first-class -The `play` / `train` subcommands forward every `-- VALUE` flag into `apply_overrides`. -The three simulation shortcuts (`--vis`, `--gpu`, `--steps`) rewrite to -`env.simulation.{vis,gpu,steps}` overrides before forwarding. +Training and inspection often need different defaults. `play_env` lets a task keep viewer-friendly +settings next to training settings: fewer envs, viewer on, optional mouse interaction, and optional +recording outputs. -## See also +## Where to continue -- [Actuators](actuators.md) -- [Play and Train](../cli/play-train.md) -- [API Reference](../api/reference.md) +- [Configuration Reference](../reference/configuration.md) +- [Design a Task](../best-practices/task-design.md) +- [CLI Reference](../reference/cli.md) diff --git a/docs/concepts/configs.zh.md b/docs/concepts/configs.zh.md index 73c438a7..d19a44ba 100644 --- a/docs/concepts/configs.zh.md +++ b/docs/concepts/configs.zh.md @@ -1,62 +1,34 @@ # 配置系统 -GeneLab 的配置体系是 `genelab.configs` 下的一个小型 dataclass 层级： +GeneLab 配置是一棵 dataclass 树。它刻意使用普通 Python，而不是自定义 schema：任务作者可以用常规代码构造配置，CLI 也仍然能暴露稳定的 override 路径。 -``` -TaskCfg -└── env: object # 下游扩展把自己的 env dataclass 接入这里 - ManagerBasedEnvCfg - ├── simulation - ├── scene - ├── actions - ├── observations - ├── rewards - ├── terminations - └── events -``` - -`TaskCfg.env` 类型刻意写成 `object` —— 下游扩展无需触碰核心即可接入自己的 env dataclass。 -大多数情况下 `env` 字段会是 `ManagerBasedEnvCfg` 的子类。 +## 核心概念 -## apply_overrides +`TaskCfg` 是发现层与运行层之间的边界。它记录 task id、已注册 env 和 robot 名、默认 env 配置、 +可选 play-mode env 配置、可选 agent 配置。 -核心函数是 `apply_overrides(cfg, dict)`，解析点路径并把值应用到配置树： - -```python -from genelab.configs import apply_overrides - -apply_overrides(cfg, { - "env.simulation.dt": "0.005", - "env.simulation.steps": "500", - "env.robot.actuators.cart.stiffness": "100.0", - "env.observations.include_velocity": "true", -}) +```text +TaskCfg +├── env +├── play_env +└── agent ``` -### 类型转换 +`env` 有意标成 `object`。下游项目可以使用 `ManagerBasedRlEnvCfg` 这样的子类，而不需要修改 GeneLab core。 -字符串值按目标 dataclass 字段的类型注解自动转换： +## 为什么 dotted override 可行 -| 类型注解 | 接受的字符串 | -|---------|-------------| -| `bool` | `true`/`false`/`1`/`0`/`yes`/`no`（大小写不敏感） | -| `int` | 任意 10 进制整数字面量 | -| `float` | 任意 Python float 字面量 | -| `Path` | 经过 `pathlib.Path(...)` | -| `list[T]` | 逗号分隔，每个元素按 `T` 转换 | -| `tuple[T, ...]` | 逗号分隔，每个元素按 `T` 转换 | -| `str` | 原样 | +CLI 把 `--a.b.c VALUE` 当作 dataclass 树上的路径。`apply_overrides` 解析路径、读取当前值和类型注解、转换字符串值，并在运行前修改配置。 -当点路径无法解析到已知字段，或值无法转换时，`apply_overrides` 在配置构造期就抛出明确错误， -避免错误延后到仿真运行时才暴露。 +这样实验可复现：最终 `TaskCfg` 可以被 dump，非法路径会早失败，而不是被静默忽略。 -### CLI 转发路径 +## Play 配置是一等配置 -`play` / `train` 把每个 `-- VALUE` 标志转发给 `apply_overrides`。三个仿真短标志 -（`--vis`、`--gpu`、`--steps`）在转发前被改写为 `env.simulation.{vis,gpu,steps}`。 +训练和人工检查通常需要不同默认值。`play_env` 让 task 把 viewer 友好的设置和训练设置放在一起： +更少 env、打开 viewer、可选鼠标交互、可选 recording 输出。 -## See also +## 继续阅读 -- [执行器](actuators.md) -- [play 与 train](../cli/play-train.md) -- [API 参考](../api/reference.md) +- [配置参考](../reference/configuration.md) +- [设计任务](../best-practices/task-design.md) +- [CLI 参考](../reference/cli.md) diff --git a/docs/concepts/extensions.en.md b/docs/concepts/extensions.en.md index 13999d62..7732b9ea 100644 --- a/docs/concepts/extensions.en.md +++ b/docs/concepts/extensions.en.md @@ -1,65 +1,32 @@ # Extensions -GeneLab core ships **no** robots, environments, or tasks. All content lives in downstream -extension packages. The CLI discovers extensions through three pathways, in order of -preference. +Extensions are ordinary Python packages that register robots, environments, and tasks with GeneLab. +They are the expected way to build real projects. -## 1. Entry-point auto-discovery (recommended) +## Why extensions are separate -Declare in the extension's `pyproject.toml`: +Keeping downstream projects outside `src/genelab/` avoids turning the framework into a collection of +project-specific code. It also lets teams version, install, and publish their robot packages +independently. -```toml -[project.entry-points."genelab.extensions"] -my_robot_project = "my_robot_project:register" -``` +## Discovery mechanisms -The CLI auto-imports every entry point in the `genelab.extensions` group on startup. -`register()` is a no-argument callable at the package top level that performs the actual -`ROBOTS.register(...)` / `ENVS.register(...)` / `TASKS.register(...)` calls (or simply imports -modules that perform them as a side effect). +| Mechanism | Best use | +|---|---| +| Entry point in `genelab.extensions` | Installed packages and daily workflows. | +| CLI `--import MODULE` | Temporary local modules or debugging entry-point loading. | +| Programmatic loading | Embedded applications. | -This is the pathway `genelab project new` wires up automatically. +All mechanisms end with the same operation: a registration function calls `register_robot`, +`register_env`, and `register_task`. -## 2. Explicit `--import` +## Extension contract -```bash -uv run genelab --import my_pkg.module1 --import my_pkg.module2 list tasks -``` +An extension should be importable as a package, keep registry-time imports light, expose a +no-argument `register()` hook, and avoid duplicate registration in repeated loads. -Repeatable. Common uses: +## Where to continue -- The extension does not (yet) ship a `genelab.extensions` entry point. -- A fully explicit, reproducible loading order is required (combine with `--no-entry-points`). -- Iterating on a not-yet-installed package while ad-hoc-adding its source directory to - `sys.path` (the CLI also adds the current working directory to `sys.path`). - -## 3. Programmatic - -```python -from genelab.registry import load_extension_module - -load_extension_module("my_pkg.module") -``` - -Used by tests and embedding scripts. The same idempotency guard applies, so calling it after -the CLI has already auto-discovered the same module is a no-op. - -## Disabling auto-discovery - -```bash -uv run genelab --no-entry-points --import my_pkg list tasks -``` - -This is the most reproducible setup — only the modules explicitly named are loaded. - -## Reference extension - -`examples/genelab_examples/` is the reference shape: a `pyproject.toml` with the entry point, a -top-level `register()` callable, and `config.py` / `robots.py` / `envs.py` / `tasks.py` -modules. `tests/fake_extension.py` is the same shape stripped to bare minimum for the test -suite. - -## See also - -- [Project new](../cli/project-new.md) +- [Build an Extension Project](../best-practices/extension-projects.md) +- [Project New](../cli/project-new.md) - [Registry](registry.md) diff --git a/docs/concepts/extensions.zh.md b/docs/concepts/extensions.zh.md index beebfb35..e18c295f 100644 --- a/docs/concepts/extensions.zh.md +++ b/docs/concepts/extensions.zh.md @@ -1,61 +1,27 @@ # 扩展加载 -GeneLab 核心**不**自带机器人、环境、任务，所有内容由下游扩展包提供。CLI 通过三条路径发现 -扩展，按以下优先级。 +扩展是普通 Python 包，用来向 GeneLab 注册机器人、环境和任务。真实项目应使用这种方式构建。 -## 1. Entry-point 自动发现（推荐） +## 为什么扩展要分离 -在扩展的 `pyproject.toml` 中声明： +把下游项目放在 `src/genelab/` 外，可以避免框架变成项目专用代码集合。团队也能独立 version、install 和发布自己的机器人包。 -```toml -[project.entry-points."genelab.extensions"] -my_robot_project = "my_robot_project:register" -``` +## 发现机制 -CLI 启动时自动 import `genelab.extensions` 组下的每个 entry point。`register()` 是一个无参 -可调用，位于扩展包顶层，负责真正调用 `ROBOTS.register(...)` / `ENVS.register(...)` / -`TASKS.register(...)`（或仅 import 一些以副作用方式注册的模块）。 +| 机制 | 适用场景 | +|---|---| +| `genelab.extensions` entry point | 已安装包和日常工作流。 | +| CLI `--import MODULE` | 临时本地模块或调试 entry-point 加载。 | +| 程序内加载 | 嵌入式应用。 | -这条路径正是 `genelab project new` 默认接好的。 +所有机制最终做同一件事：注册函数调用 `register_robot`、`register_env`、`register_task`。 -## 2. 显式 `--import` +## 扩展契约 -```bash -uv run genelab --import my_pkg.module1 --import my_pkg.module2 list tasks -``` +扩展应能作为包导入，保持注册阶段导入轻量，暴露无参 `register()` hook，并避免重复加载时重复注册。 -可重复。常见场景： - -- 扩展尚未提供 `genelab.extensions` entry point。 -- 需要完全显式、可复现的加载顺序（搭配 `--no-entry-points`）。 -- 调试尚未安装的包，临时把源码目录塞进 `sys.path`（CLI 也会把当前工作目录加入 `sys.path`）。 - -## 3. 程序内调用 - -```python -from genelab.registry import load_extension_module - -load_extension_module("my_pkg.module") -``` - -测试与嵌入脚本使用。同样的幂等守卫生效，即使 CLI 已经自动发现过同一模块，再调用一次也是 -no-op。 - -## 关闭自动发现 - -```bash -uv run genelab --no-entry-points --import my_pkg list tasks -``` - -这是最可复现的组合 —— 只加载显式命名的模块。 - -## 参考扩展 - -`examples/genelab_examples/` 是参考形态：`pyproject.toml` 带 entry point、顶层有 -`register()` 可调用、并提供 `config.py` / `robots.py` / `envs.py` / `tasks.py`。 -`tests/fake_extension.py` 是同样形态但削减到最小，供测试套件使用。 - -## See also +## 继续阅读 +- [构建扩展项目](../best-practices/extension-projects.md) - [新建项目](../cli/project-new.md) - [注册表](registry.md) diff --git a/docs/concepts/managers.en.md b/docs/concepts/managers.en.md index 21cbecd7..55beea69 100644 --- a/docs/concepts/managers.en.md +++ b/docs/concepts/managers.en.md @@ -1,274 +1,54 @@ -# Managers and MDP terms +# Managers and MDP Terms -`genelab.managers` exposes the seven manager-style MDP hooks that -`ManagerBasedRlEnv` orchestrates: actions, commands, observations, rewards, -terminations, events, and curriculums. Each manager owns a dict of *term* cfgs -keyed by name; the env constructs them in a fixed order, then drives them once -per control step or once per reset. +GeneLab follows the Isaac Lab-style manager-based environment pattern: the environment owns the +simulation loop, while MDP behavior is split into named terms managed by specialized managers. -## Term and manager +## Why split the MDP -A *term* is a single MDP-side computation — one reward component, one -observation column group, one termination check. A *manager* is the container -that instantiates all terms of one kind, exposes their aggregate output, and -schedules their per-step / per-reset calls. +A robot learning environment usually contains many concerns: -``` -ManagerBasedRlEnvCfg -├── actions_cfg: dict[str, ActionTermCfg] -├── commands_cfg: dict[str, CommandTermCfg] -├── observations_cfg: dict[str, ObservationGroupCfg] # group → terms -├── rewards_cfg: dict[str, RewardTermCfg] -├── terminations_cfg: dict[str, TerminationTermCfg] -├── events_cfg: dict[str, EventTermCfg] -└── curriculum_cfg: dict[str, CurriculumTermCfg] -``` +- action decoding +- command sampling +- observations +- rewards +- terminations +- reset and interval events +- curricula +- metrics -Every term cfg inherits `ManagerTermBaseCfg` with two fields: +Putting all of that directly in `step()` makes tasks hard to inspect and override. Manager-based +configs keep each concern named, typed, and discoverable. -| Field | Type | Meaning | -|---|---|---| -| `func` | `Callable[..., Any]` | The per-step computation. For observation / reward / termination / event / curriculum terms this is a function taking `(env, **params)` and returning a tensor (or `None` for events). Action and command terms ignore `func` and use `class_type` instead. | -| `params` | `dict[str, Any]` | Keyword arguments injected on every call. `apply_overrides` can mutate individual entries via dotted paths. | - -## The seven managers - -| Manager | Term cfg | Term identifier | Call schedule | -|---|---|---|---| -| `ActionManager` | `ActionTermCfg(class_type=…)` | `class_type` | `process_action` once per env step; `apply_action` `decimation` times per env step | -| `CommandManager` | `CommandTermCfg(class_type=…)` | `class_type` | `compute(dt)` once per env step (resampling on countdown) | -| `ObservationManager` | `ObservationGroupCfg.terms[name] = ObservationTermCfg(func=…)` | `func` | `compute()` once per env step (after reset and after step) | -| `RewardManager` | `RewardTermCfg(func=…, weight=…)` | `func` | `compute(dt)` once per env step | -| `TerminationManager` | `TerminationTermCfg(func=…, time_out=…)` | `func` | `compute()` once per env step | -| `EventManager` | `EventTermCfg(func=…, mode=…)` | `func` | once per construct (`startup`), once per reset (`reset`), or on per-env countdown (`interval`) | -| `CurriculumManager` | `CurriculumTermCfg(func=…)` | `func` | `compute(env_ids)` once per reset, after the other managers reset | - -## Term lifecycle - -`ManagerBasedRlEnv.__init__` constructs the seven managers in the order above. -For each term: - -1. The manager deep-copies the cfg dict so per-instance mutation never bleeds - back into the user's source cfg. -2. If `func` is a class, `_base.instantiate_class_term` replaces it with - `func(cfg=term_cfg, env=env)` so the term can cache references. Plain - callables are kept as-is. -3. Action and command managers instead invoke `class_type(term_cfg, env)` and - keep the resulting instance under the term name. - -After all managers are constructed the env calls `event_manager.apply("startup")` -once, then `articulation.refresh()`, then `reset()` for every env. - -## ActionManager - -Each `ActionTermCfg` declares a `class_type` (an `ActionTerm` subclass) and an -optional `asset_name`. The manager concatenates per-term `action_dim` slots -into a single flat action vector. Multiple action terms can coexist, e.g. one -covering the arm joints and one covering the gripper: - -```python -from genelab.mdp.actions.joint_position import JointPositionActionCfg - -actions_cfg = { - "panda_arm": JointPositionActionCfg( - asset_name="robot", - joint_names=(r"^joint[1-7]$",), - use_default_offset=True, - ), - "panda_hand": JointPositionActionCfg( - asset_name="robot", - joint_names=(r"finger_joint.*",), - use_default_offset=True, - ), -} -``` - -`process_action(action)` runs once per env step; `apply_action()` is called -`decimation` times per env step before each sim sub-step, so the controller -sees a fixed setpoint for the whole decimation window. - -## ObservationManager - -Observations are organised into named *groups* (`policy`, `critic`, …); each -group is a dict of named terms. - -```python -from genelab.managers import ObservationGroupCfg, ObservationTermCfg -from genelab import mdp - -observations_cfg = { - "policy": ObservationGroupCfg( - terms={ - "base_lin_vel": ObservationTermCfg(func=mdp.base_lin_vel), - "joint_pos_rel": ObservationTermCfg(func=mdp.joint_pos_rel), - "last_action": ObservationTermCfg(func=mdp.last_action), - }, - concatenate_terms=True, - enable_corruption=False, - ), -} -``` - -For each term `compute()` calls `term_cfg.func(env, **term_cfg.params)`, -unsqueezes 1-D returns to `(num_envs, 1)`, then optionally applies `noise` -(only when `enable_corruption=True`), `scale`, and `clip`. When -`concatenate_terms=True` (default) the per-group output is a single -`(num_envs, total_dim)` tensor; otherwise the manager stacks along a new last -axis. - -## RewardManager - -`RewardTermCfg.weight` is the signed scalar multiplier. The manager skips -zero-weight terms. When `ManagerBasedRlEnvCfg.scale_rewards_by_dt=True` -(default) the per-term tensor is also multiplied by the env step `dt`, so total -episode return is comparable across simulation frequencies. - -```python -from genelab.managers import RewardTermCfg -from genelab import mdp +## The manager set -rewards_cfg = { - "track_lin_vel": RewardTermCfg( - func=mdp.track_linear_velocity_xy_exp, - weight=1.0, - params={"command_name": "base_velocity", "std": 0.25}, - ), - "action_rate": RewardTermCfg(func=mdp.action_rate_l2, weight=-0.005), - "joint_acc": RewardTermCfg(func=mdp.joint_acc_l2, weight=-2.5e-7), -} -``` - -Per-term `weight * func(...)` outputs are NaN-coerced, summed, and exposed as a -single `(num_envs,)` reward tensor. The manager also accumulates a per-term -episode sum; `reset()` returns the mean episode reward per term (logged as -`Episode_Reward/` in `extras["log"]`). - -## TerminationManager - -Each `TerminationTermCfg.func` must return a bool tensor of shape -`(num_envs,)`. Terms with `time_out=True` flow into the truncation buffer -(`info["time_out"]`); the rest accumulate into the terminated buffer. Both are -OR'd to produce the final `dones` mask. RSL-RL distinguishes the two so the -PPO update treats truncation correctly. - -```python -from genelab.managers import TerminationTermCfg -from genelab import mdp - -terminations_cfg = { - "time_out": TerminationTermCfg(func=mdp.time_out, time_out=True), - "fell_over": TerminationTermCfg( - func=mdp.bad_orientation, - params={"limit_angle": 1.0}, - ), -} -``` - -## EventManager - -`EventTermCfg.mode` is a `Literal["startup", "reset", "interval"]`: - -| Mode | Fires | -|---|---| -| `startup` | Once, right after all managers are constructed. Useful for one-shot randomisation of physical parameters (mass, friction). | -| `reset` | On every env reset, before `command_manager.reset` and `action_manager.reset`. Use this to randomise initial joint state, root pose, or env-specific buffers. | -| `interval` | Per-env countdown sampled uniformly from `interval_range_s`. Fires inside the env step when the countdown hits zero, then resamples. Use for periodic disturbances (random push). | - -```python -from genelab.managers import EventTermCfg -from genelab import mdp - -events_cfg = { - "reset_joints": EventTermCfg( - mode="reset", - func=mdp.reset_joints_to_default, - params={"pos_jitter": 0.05, "vel_jitter": 0.0}, - ), - "push_robot": EventTermCfg( - mode="interval", - func=mdp.push_by_setting_velocity, - interval_range_s=(8.0, 12.0), - params={"velocity_range": {"x": (-0.5, 0.5), "y": (-0.5, 0.5)}}, - ), -} -``` - -## CommandManager - -A *command* is a persistent per-env target (velocity command, motion -reference, goal pose) that survives across env steps until it is resampled. -Each `CommandTermCfg.class_type` is a `CommandTerm` subclass; the term holds -the live tensor and resamples it every `resampling_time_range` seconds. -Observation and reward terms read commands back via -`env.command_manager.get_command(name)`. - -```python -from genelab.mdp.commands.uniform_velocity import UniformVelocityCommandCfg - -commands_cfg = { - "base_velocity": UniformVelocityCommandCfg( - asset_name="robot", - resampling_time_range=(10.0, 10.0), - ranges=UniformVelocityCommandCfg.Ranges( - lin_vel_x=(-1.0, 1.0), lin_vel_y=(-0.5, 0.5), ang_vel_z=(-1.0, 1.0), - ), - ), -} -``` - -## CurriculumManager - -Curriculum terms run on every reset (after the other managers have reset) -and may mutate scene state — `TerrainImporter.terrain_levels`, env-specific -spawn poses, reward weights. The return value is logged as -`Curriculum/` in `extras["log"]`; tensor returns are reduced to their -mean across `env_ids`. - -```python -from genelab.managers import CurriculumTermCfg -from genelab.mdp.curriculums import terrain_levels_vel - -curriculum_cfg = { - "terrain_levels": CurriculumTermCfg( - func=terrain_levels_vel, - params={ - "command_name": "base_velocity", - "distance_threshold": 5.0, - "demote_ratio": 0.5, - }, - ), -} -``` +| Manager | Config field | Role | +|---|---|---| +| `ActionManager` | `actions_cfg` | Converts policy actions into simulator targets or torques. | +| `CommandManager` | `commands_cfg` | Maintains sampled goals such as target velocity or reference motion. | +| `ObservationManager` | `observations_cfg` | Computes named observation groups such as `policy` and `critic`. | +| `RewardManager` | `rewards_cfg` | Computes weighted reward terms and episode summaries. | +| `TerminationManager` | `terminations_cfg` | Separates terminated and time-out conditions. | +| `EventManager` | `events_cfg` | Runs startup, reset, and interval randomization or perturbation. | +| `CurriculumManager` | `curriculum_cfg` | Adjusts difficulty or task state at reset boundaries. | +| `MetricsManager` | `metrics_cfg` | Records non-reward diagnostics. | -## Failure modes worth knowing +## Runtime order -!!! warning "Wrong observation tensor shape" - `ObservationManager.compute` only auto-unsqueezes 1-D returns to - `(num_envs, 1)`. A scalar-per-env reward term that accidentally returns a - `()` scalar will broadcast in the wrong direction. Always return shape - `(num_envs,)` or `(num_envs, d)`. +`ManagerBasedRlEnv` builds the Genesis scene first, then creates managers, binds sensors, applies +startup events, and runs an initial reset. During each env step it processes actions, advances the +scene for `decimation` physics ticks, refreshes state, updates sensors, computes commands, events, +rewards, metrics, terminations, resets done envs, and finally computes observations. -!!! warning "Missing class_type / func" - `ActionManager` and `CommandManager` silently skip terms whose `class_type` - is `None`. `RewardManager` / `TerminationManager` / `EventManager` / - `CurriculumManager` keep the no-op default `func` from `ManagerTermBaseCfg`, - so a missing `func=` produces a manager that returns `None` and crashes - downstream when the result is used as a tensor. Always set the term - factory explicitly. +This order matters because terms read shared state. For example, reward terms see the state after +actions and sensor updates; reset events run before the next observation is emitted. -!!! warning "Interval events without `interval_range_s`" - `EventManager.__init__` asserts that every `mode="interval"` term provides - `interval_range_s=(low, high)`. The assertion fires at manager construction - so misconfigured tasks never reach the first reset. +## Term names are part of the interface -!!! tip "Override term params at the CLI" - Every term `params` entry is reachable via `apply_overrides`. The dotted - path mirrors the cfg tree printed by `genelab info `: - `--env.rewards_cfg.track_lin_vel.weight 2.0` rescales a reward term; - `--env.rewards_cfg.track_lin_vel.params.std 0.5` retargets a kwarg. +Term names appear in override paths and logs. A reward named `track_lin_vel` can be changed from the +CLI and appears in episode summaries. Stable names make experiments easier to compare. -## See also +## Where to continue -- [Configs](configs.md) -- [Sensors](sensors.md) -- [Discovery: list and info](../cli/list-info.md) +- [Design a Task](../best-practices/task-design.md) +- [MDP terms reference](mdp.md) +- [API Reference](../api/reference.md) diff --git a/docs/concepts/managers.zh.md b/docs/concepts/managers.zh.md index 8533bc09..07199b04 100644 --- a/docs/concepts/managers.zh.md +++ b/docs/concepts/managers.zh.md @@ -1,251 +1,51 @@ # Manager 与 MDP term -`genelab.managers` 暴露了 `ManagerBasedRlEnv` 编排的七个 manager 风格 MDP 钩子： -actions、commands、observations、rewards、terminations、events、curriculums。 -每个 manager 持有一个按名键入的 term cfg 字典；env 按固定顺序构造它们，再按每控制 -步或每次 reset 的节奏驱动。 +GeneLab 使用 Isaac Lab 风格的 manager-based 环境模式：环境负责仿真循环，MDP 行为拆成命名 term， +由专门 manager 管理。 -## term 与 manager +## 为什么拆分 MDP -*term* 是单条 MDP 侧的计算 —— 一个 reward 分量、一组 observation 列、一个终止判定。 -*manager* 是容器，负责实例化某一类 term、暴露聚合输出、并按每步 / 每次 reset 调度。 +机器人学习环境通常包含很多关注点： -``` -ManagerBasedRlEnvCfg -├── actions_cfg: dict[str, ActionTermCfg] -├── commands_cfg: dict[str, CommandTermCfg] -├── observations_cfg: dict[str, ObservationGroupCfg] # group → terms -├── rewards_cfg: dict[str, RewardTermCfg] -├── terminations_cfg: dict[str, TerminationTermCfg] -├── events_cfg: dict[str, EventTermCfg] -└── curriculum_cfg: dict[str, CurriculumTermCfg] -``` +- action 解码 +- command 采样 +- observation +- reward +- termination +- reset 和 interval event +- curriculum +- metric -所有 term cfg 都继承 `ManagerTermBaseCfg`，含两个字段： +如果全部塞进 `step()`，任务会很难检查和 override。manager-based 配置让每个关注点都有名字、类型和可发现路径。 -| 字段 | 类型 | 含义 | -|---|---|---| -| `func` | `Callable[..., Any]` | 每步执行的计算。observation / reward / termination / event / curriculum 类 term 的 `func` 签名为 `(env, **params)` 并返回 tensor（event 可返回 `None`）。action 与 command term 不使用 `func`，改用 `class_type`。 | -| `params` | `dict[str, Any]` | 每次调用注入的关键字参数。`apply_overrides` 可按 dotted path 改写其中任一条目。 | - -## 七个 manager - -| Manager | Term cfg | term 识别字段 | 调用节奏 | -|---|---|---|---| -| `ActionManager` | `ActionTermCfg(class_type=…)` | `class_type` | 每 env step 一次 `process_action`；每 env step 内 `apply_action` 触发 `decimation` 次 | -| `CommandManager` | `CommandTermCfg(class_type=…)` | `class_type` | 每 env step 一次 `compute(dt)`（按倒计时重采样） | -| `ObservationManager` | `ObservationGroupCfg.terms[name] = ObservationTermCfg(func=…)` | `func` | 每 env step 一次 `compute()`（reset 后与 step 后各一次） | -| `RewardManager` | `RewardTermCfg(func=…, weight=…)` | `func` | 每 env step 一次 `compute(dt)` | -| `TerminationManager` | `TerminationTermCfg(func=…, time_out=…)` | `func` | 每 env step 一次 `compute()` | -| `EventManager` | `EventTermCfg(func=…, mode=…)` | `func` | 构造时一次（`startup`）、每次 reset 一次（`reset`）、按每 env 倒计时触发（`interval`） | -| `CurriculumManager` | `CurriculumTermCfg(func=…)` | `func` | 每次 reset 一次 `compute(env_ids)`，在其他 manager reset 之后 | - -## term 生命周期 - -`ManagerBasedRlEnv.__init__` 按上表顺序构造七个 manager。每个 term 的处理： - -1. Manager 深拷贝 cfg 字典，避免逐实例 mutation 反向污染源 cfg。 -2. 当 `func` 是类时，`_base.instantiate_class_term` 把它替换为 - `func(cfg=term_cfg, env=env)`，让 term 在构造期缓存引用。普通可调用对象保持原样。 -3. action / command manager 改用 `class_type(term_cfg, env)`，把得到的实例按 term 名存好。 - -七个 manager 全部构造完毕后，env 调用一次 `event_manager.apply("startup")`， -再 `articulation.refresh()`，最后对所有 env 跑一次 `reset()`。 - -## ActionManager - -每个 `ActionTermCfg` 声明一个 `class_type`（`ActionTerm` 子类）与可选的 `asset_name`。 -Manager 把每个 term 的 `action_dim` 拼成单个扁平 action 向量。多个 action term -可以并存 —— 例如手臂一个、夹爪一个： - -```python -from genelab.mdp.actions.joint_position import JointPositionActionCfg - -actions_cfg = { - "panda_arm": JointPositionActionCfg( - asset_name="robot", - joint_names=(r"^joint[1-7]$",), - use_default_offset=True, - ), - "panda_hand": JointPositionActionCfg( - asset_name="robot", - joint_names=(r"finger_joint.*",), - use_default_offset=True, - ), -} -``` - -`process_action(action)` 在每 env step 调用一次；`apply_action()` 在每 env step 内 -触发 `decimation` 次（每个 sim 子步前一次），因此控制器在整个 decimation 窗口内 -看到的是同一个目标。 - -## ObservationManager - -Observation 按命名 *group*（`policy` / `critic` / …）组织；每个 group 是命名 term 字典。 - -```python -from genelab.managers import ObservationGroupCfg, ObservationTermCfg -from genelab import mdp - -observations_cfg = { - "policy": ObservationGroupCfg( - terms={ - "base_lin_vel": ObservationTermCfg(func=mdp.base_lin_vel), - "joint_pos_rel": ObservationTermCfg(func=mdp.joint_pos_rel), - "last_action": ObservationTermCfg(func=mdp.last_action), - }, - concatenate_terms=True, - enable_corruption=False, - ), -} -``` - -`compute()` 对每个 term 调用 `term_cfg.func(env, **term_cfg.params)`，1-D 返回会被 -自动 unsqueeze 成 `(num_envs, 1)`，随后按需 apply `noise`（仅当 -`enable_corruption=True`）、`scale`、`clip`。`concatenate_terms=True`（默认）时整组 -输出是单张 `(num_envs, total_dim)` tensor；否则 manager 沿新末轴 stack。 - -## RewardManager - -`RewardTermCfg.weight` 是带符号标量乘子。Manager 跳过 weight=0 的 term。当 -`ManagerBasedRlEnvCfg.scale_rewards_by_dt=True`（默认）时，逐 term tensor 还会再乘 -env step `dt`，使得 episode 总回报在不同仿真频率下可比。 - -```python -from genelab.managers import RewardTermCfg -from genelab import mdp +## manager 集合 -rewards_cfg = { - "track_lin_vel": RewardTermCfg( - func=mdp.track_linear_velocity_xy_exp, - weight=1.0, - params={"command_name": "base_velocity", "std": 0.25}, - ), - "action_rate": RewardTermCfg(func=mdp.action_rate_l2, weight=-0.005), - "joint_acc": RewardTermCfg(func=mdp.joint_acc_l2, weight=-2.5e-7), -} -``` - -逐 term 的 `weight * func(...)` 结果会 NaN coerce、求和，并作为 `(num_envs,)` 的 -reward tensor 输出。Manager 同时累计每 term 的 episode 求和；`reset()` 返回每 -term 的 episode 平均奖励（按 `Episode_Reward/` 写入 `extras["log"]`）。 - -## TerminationManager - -每个 `TerminationTermCfg.func` 必须返回 `(num_envs,)` 的 bool tensor。`time_out=True` -的 term 写入 truncation buffer（`info["time_out"]`），其余写入 terminated buffer。 -两者 OR 后得到最终 `dones`。RSL-RL 区分这两者以在 PPO 更新中正确处理截断。 - -```python -from genelab.managers import TerminationTermCfg -from genelab import mdp - -terminations_cfg = { - "time_out": TerminationTermCfg(func=mdp.time_out, time_out=True), - "fell_over": TerminationTermCfg( - func=mdp.bad_orientation, - params={"limit_angle": 1.0}, - ), -} -``` - -## EventManager - -`EventTermCfg.mode` 是 `Literal["startup", "reset", "interval"]`： - -| Mode | 触发时机 | -|---|---| -| `startup` | 全部 manager 构造完成后一次。适合一次性随机化物理参数（质量、摩擦）。 | -| `reset` | 每次 env reset，在 `command_manager.reset` 与 `action_manager.reset` 之前。适合随机化初始 joint state、root pose、env 缓存。 | -| `interval` | 按每 env 倒计时（从 `interval_range_s` 均匀采样）。在 env step 内倒计时归零时触发，触发后重新采样。适合周期性扰动（随机推力）。 | - -```python -from genelab.managers import EventTermCfg -from genelab import mdp - -events_cfg = { - "reset_joints": EventTermCfg( - mode="reset", - func=mdp.reset_joints_to_default, - params={"pos_jitter": 0.05, "vel_jitter": 0.0}, - ), - "push_robot": EventTermCfg( - mode="interval", - func=mdp.push_by_setting_velocity, - interval_range_s=(8.0, 12.0), - params={"velocity_range": {"x": (-0.5, 0.5), "y": (-0.5, 0.5)}}, - ), -} -``` - -## CommandManager - -*command* 是按 env 持久化的目标（速度命令、动作参考、目标位姿），跨 env step 保留， -直到被重采样。每个 `CommandTermCfg.class_type` 是 `CommandTerm` 子类；term 持有 -当前 tensor 并每 `resampling_time_range` 秒重采样一次。observation 与 reward term -通过 `env.command_manager.get_command(name)` 读回。 - -```python -from genelab.mdp.commands.uniform_velocity import UniformVelocityCommandCfg - -commands_cfg = { - "base_velocity": UniformVelocityCommandCfg( - asset_name="robot", - resampling_time_range=(10.0, 10.0), - ranges=UniformVelocityCommandCfg.Ranges( - lin_vel_x=(-1.0, 1.0), lin_vel_y=(-0.5, 0.5), ang_vel_z=(-1.0, 1.0), - ), - ), -} -``` - -## CurriculumManager - -Curriculum term 在每次 reset 触发（在其他 manager reset 之后），可改写 scene 状态 —— -`TerrainImporter.terrain_levels`、env 专属 spawn pose、reward weight。返回值按 -`Curriculum/` 写入 `extras["log"]`；tensor 返回会被 reduce 成在 `env_ids` 上的均值。 - -```python -from genelab.managers import CurriculumTermCfg -from genelab.mdp.curriculums import terrain_levels_vel - -curriculum_cfg = { - "terrain_levels": CurriculumTermCfg( - func=terrain_levels_vel, - params={ - "command_name": "base_velocity", - "distance_threshold": 5.0, - "demote_ratio": 0.5, - }, - ), -} -``` +| Manager | 配置字段 | 作用 | +|---|---|---| +| `ActionManager` | `actions_cfg` | 把策略 action 转成仿真目标或力矩。 | +| `CommandManager` | `commands_cfg` | 维护采样目标，例如目标速度或参考 motion。 | +| `ObservationManager` | `observations_cfg` | 计算 `policy`、`critic` 等 observation group。 | +| `RewardManager` | `rewards_cfg` | 计算带权 reward term 和 episode 摘要。 | +| `TerminationManager` | `terminations_cfg` | 区分 terminated 和 time-out 条件。 | +| `EventManager` | `events_cfg` | 执行 startup、reset、interval 随机化或扰动。 | +| `CurriculumManager` | `curriculum_cfg` | 在 reset 边界调整难度或任务状态。 | +| `MetricsManager` | `metrics_cfg` | 记录非 reward 诊断指标。 | -## 已知失效模式 +## 运行时顺序 -!!! warning "observation tensor 形状错误" - `ObservationManager.compute` 只会把 1-D 返回 auto-unsqueeze 成 `(num_envs, 1)`。 - 意外返回 `()` 标量的 reward term 会朝错误方向广播。term 必须返回 `(num_envs,)` - 或 `(num_envs, d)`。 +`ManagerBasedRlEnv` 先构建 Genesis scene，再创建 manager、绑定传感器、执行 startup event、 +跑初始 reset。每个 env step 中，它处理 action，推进 `decimation` 个 physics tick，刷新状态， +更新传感器，计算 command、event、reward、metric、termination，reset 已完成 env，最后计算 observation。 -!!! warning "缺失 class_type / func" - `ActionManager` 与 `CommandManager` 会静默跳过 `class_type=None` 的 term。 - `RewardManager` / `TerminationManager` / `EventManager` / `CurriculumManager` - 保留 `ManagerTermBaseCfg` 的 no-op 默认 `func`，缺失 `func=` 会构造出返回 `None` - 的 manager，并在下游把结果当 tensor 用时崩溃。term factory 必须显式设置。 +这个顺序很重要，因为 term 读取共享状态。例如 reward term 看到的是 action 和 sensor update 之后的状态； +reset event 会在下一次 observation 发出前运行。 -!!! warning "interval event 缺 `interval_range_s`" - `EventManager.__init__` 断言每个 `mode="interval"` term 必须给 `interval_range_s=(low, high)`。 - 断言在 manager 构造时触发，错配的 task 永远跑不到第一次 reset。 +## term 名称是接口的一部分 -!!! tip "在 CLI 覆盖 term 参数" - 每个 term 的 `params` 条目都可通过 `apply_overrides` 访问。dotted path 与 - `genelab info ` 打印的 cfg 树一一对应： - `--env.rewards_cfg.track_lin_vel.weight 2.0` 重新缩放一个 reward term； - `--env.rewards_cfg.track_lin_vel.params.std 0.5` 改写一个 kwarg。 +term 名会出现在 override 路径和日志中。名为 `track_lin_vel` 的 reward 可以从 CLI 修改，也会出现在 episode 摘要里。稳定命名让实验更容易比较。 -## See also +## 继续阅读 -- [Configs](configs.md) -- [Sensors](sensors.md) -- [Discovery: list and info](../cli/list-info.md) +- [设计任务](../best-practices/task-design.md) +- [MDP term 参考](mdp.md) +- [API 参考](../api/reference.md) diff --git a/docs/concepts/mdp.en.md b/docs/concepts/mdp.en.md index 9b6b7042..1d6cad68 100644 --- a/docs/concepts/mdp.en.md +++ b/docs/concepts/mdp.en.md @@ -1,281 +1,49 @@ -# MDP terms reference +# MDP Terms Reference -`genelab.mdp` is a library of plain Python callables that plug straight into -the seven managers. Every function follows the same signature shape — `(env, -**params) -> torch.Tensor` for observation / reward / termination, `(env, -env_ids, **params) -> None` for events, `(env, env_ids, **params) -> Tensor` -for curriculums — and is referenced by the corresponding `*TermCfg.func` (or -`class_type`) field. +`genelab.mdp` is a reusable term library. It does not define a task by itself; task configs select +functions and classes from this library and wire them into managers. -## Reusable building blocks +## Actions and commands -Every name listed below is re-exported by `genelab.mdp.__init__`, so -`from genelab import mdp` is enough to reach them. The full module map: +| Area | Public pieces | +|---|---| +| Actions | `JointPositionActionCfg`, `JointPositionAction` | +| Velocity commands | `UniformVelocityCommandCfg`, `UniformVelocityCommand` | +| Motion commands | `MotionCommandCfg`, `MotionCommand`, `MotionLoader` | -``` -genelab.mdp.actions → JointPositionAction(Cfg) -genelab.mdp.commands → UniformVelocityCommand(Cfg), MotionCommand(Cfg), MotionLoader -genelab.mdp.observations → 16 observation functions -genelab.mdp.rewards → 15 reward functions / classes -genelab.mdp.terminations → 6 termination functions -genelab.mdp.events → 3 event functions -genelab.mdp.curriculums → 1 curriculum function -genelab.mdp.noise → NoiseCfg base + Unoise + Gnoise -``` - -The output-shape column below uses `B = num_envs`, `D = action / joint / -sensor dimension`, `N = number of bodies or feet`. Reward and termination -terms always return shape `(B,)`; observation terms return `(B, D)` (or are -auto-unsqueezed from `(B,)` by `ObservationManager`). - -## Actions - -| Name | Signature (Cfg fields used) | Behavior | -|---|---|---| -| `JointPositionAction` | `JointPositionActionCfg(joint_names, scale, use_default_offset, asset_name)` | `target = default + scale * raw_action`. Targets are dispatched via `Articulation.write_joint_targets_partial`, which routes each joint to its actuator group's declared channel (position for implicit PD, force for `IdealPDActuator` / `DCMotorActuator`). `scale=None` (default) inherits per-joint scale from `Articulation.action_scale_tensor`; a `float` or `dict[str, float]` overrides it. Patterns are regex; unmatched patterns raise. | - -```python -from genelab.mdp.actions.joint_position import JointPositionActionCfg - -actions_cfg = { - "panda_arm": JointPositionActionCfg( - asset_name="robot", - joint_names=(r"^joint[1-7]$",), - use_default_offset=True, - ), -} -``` - -## Commands - -| Name | Signature (Cfg fields used) | Behavior | -|---|---|---| -| `UniformVelocityCommand` | `UniformVelocityCommandCfg(ranges, rel_standing_envs, heading_command, heading_control_stiffness, resampling_time_range, asset_name)` | Body-frame `[lin_vel_x, lin_vel_y, ang_vel_z]` per env, resampled uniformly from `ranges` every `resampling_time_range` seconds. With `heading_command=True` (default), `ang_vel_z` is rewritten each step to drive the body toward a sampled heading. `rel_standing_envs` zeroes a fraction of resampled commands so the policy also sees a "stand still" target. `command` shape `(B, 3)`. | -| `MotionCommand` | `MotionCommandCfg(motion_file, anchor_body_name, body_names, motion_body_order, motion_joint_order, pose_range, velocity_range, sampling_mode, ...)` | Drives the env toward a recorded motion clip frame-by-frame. Reads the NPZ schema produced by mjlab's `csv_to_npz`. `motion_body_order` / `motion_joint_order` describe the NPZ's native body / joint axis ordering (typically mjlab's MJCF DFS) so `MotionLoader` can permute the data into the runtime robot's order — required when the source uses a different traversal than Genesis. Exposes anchor + multi-body reference and current pose / velocity for the relative-pose reward functions in the next section. `sampling_mode="start"` always begins at frame 0; `"uniform"` (default) samples a random frame. | -| `MotionLoader` | `MotionLoader(motion_file, body_indexes, device, joint_perm=None)` | Helper class consumed by `MotionCommand`. Loads the NPZ into device tensors, applies the optional `joint_perm` to the joint axis, and slices `body_pos_w / body_quat_w / body_lin_vel_w / body_ang_vel_w` down to the bodies the command tracks. Direct use is rare — instantiate `MotionCommandCfg` instead. | - -```python -from genelab.mdp.commands.velocity_command import UniformVelocityCommandCfg - -commands_cfg = { - "base_velocity": UniformVelocityCommandCfg( - asset_name="robot", - resampling_time_range=(10.0, 10.0), - ranges=UniformVelocityCommandCfg.Ranges( - lin_vel_x=(-1.0, 1.0), lin_vel_y=(-0.5, 0.5), ang_vel_z=(-1.0, 1.0), - ), - heading_command=True, - ), -} -``` +Actions convert policy outputs to simulator control. Commands hold sampled goals that observations +and rewards can read. ## Observations -All listed functions take `env` plus the noted kwargs and return a tensor of -shape `(B, D)` or `(B,)` (auto-unsqueezed to `(B, 1)` by the manager). - -| Name | Params | Shape | Source | -|---|---|---|---| -| `base_lin_vel` | — | `(B, 3)` | Body-frame floating-base linear velocity. | -| `base_ang_vel` | — | `(B, 3)` | Body-frame floating-base angular velocity. | -| `projected_gravity` | — | `(B, 3)` | World gravity projected into the body frame (IMU-orientation proxy). | -| `joint_pos_rel` | — | `(B, num_dofs)` | Joint positions minus the default pose. | -| `joint_vel_rel` | — | `(B, num_dofs)` | Joint velocities (default is zero — same as raw `joint_vel`). | -| `last_action` | — | `(B, total_action_dim)` | The action tensor the manager processed last step. | -| `generated_commands` | `command_name` | `(B, command_dim)` | `env.command_manager.get_command(command_name)`. | -| `sensor_data` | `sensor_name` | varies | Per-step cached tensor on `env.sensors[sensor_name]`. Use this for IMU, FrameTransformer, ray-cast outputs that are already tensors. | -| `foot_air_time` | `sensor_name` (must be a `ContactSensor`) | `(B, N_feet)` | Current per-foot air time (zero while in contact). | -| `foot_contact` | `sensor_name` (must be a `ContactSensor`) | `(B, N_feet)` | Binary contact mask as float. | -| `foot_contact_forces` | `sensor_name` (must be a `ContactSensor`) | `(B, N_feet * 3)` | Per-foot contact force, compressed via `sign(f) * log1p(|f|)` and flattened. | -| `height_scan` | `sensor_name` (must be a `TerrainHeightSensor`) | `(B, num_rays)` | Per-ray heights above the terrain. | -| `motion_anchor_pos_b` | `command_name` (must be a `MotionCommand`) | `(B, 3)` | Reference anchor position expressed in the robot's anchor frame. | -| `motion_anchor_ori_b` | `command_name` | `(B, 6)` | Reference 6D orientation (first two columns of the rotation matrix). | -| `robot_body_pos_b` | `command_name` | `(B, N_bodies * 3)` | Per-body positions in the robot's anchor frame (privileged critic obs). | -| `robot_body_ori_b` | `command_name` | `(B, N_bodies * 6)` | Per-body 6D orientations in the robot's anchor frame. | - -```python -from genelab.managers import ObservationGroupCfg, ObservationTermCfg -from genelab import mdp -from genelab.mdp.noise import Unoise - -observations_cfg = { - "policy": ObservationGroupCfg( - terms={ - "base_lin_vel": ObservationTermCfg(func=mdp.base_lin_vel, - noise=Unoise(n_min=-0.1, n_max=0.1)), - "joint_pos_rel": ObservationTermCfg(func=mdp.joint_pos_rel), - "last_action": ObservationTermCfg(func=mdp.last_action), - "commands": ObservationTermCfg(func=mdp.generated_commands, - params={"command_name": "base_velocity"}), - }, - enable_corruption=True, - ), -} -``` - -## Rewards - -All reward functions return shape `(B,)`. `RewardManager` multiplies by -`RewardTermCfg.weight` and (when `scale_rewards_by_dt=True`) by `dt`. Negative -weights turn a "deviation" reward into a penalty. - -| Name | Params | Behavior | -|---|---|---| -| `track_linear_velocity_xy_exp` | `command_name`, `std=0.5` | `exp(-||cmd_xy − vel_xy||² / std²)`. | -| `track_angular_velocity_z_exp` | `command_name`, `std=0.5` | `exp(-(cmd_z − vel_z)² / std²)`. | -| `action_rate_l2` | — | `Σ_d (action_d − prev_action_d)²` — penalize action jitter. | -| `joint_acc_l2` | — | **Placeholder** (currently returns zeros; awaits accel buffer). | -| `flat_orientation_l2` | — | `Σ (projected_gravity_xy)²` — penalize tilt. | -| `upright_exp` | `std=0.45` | `exp(-||projected_gravity_xy||² / std²)` — positive reward for upright base. | -| `variable_posture` | `command_name`, `std_standing` / `std_walking` / `std_running` (regex→float dicts), `default_std`, `walking_threshold`, `running_threshold` | Speed-dependent posture reward: `exp(-mean((joint_pos − default)² / std²))`, with std chosen per env from the command magnitude. Class-style term — instantiated once at construct, then `__call__` per step. | -| `joint_pos_limits` | — | L2 of joint-position excursion past ±π. | -| `feet_air_time` | `threshold=0.4` | **Stub** — reward proportional to mean foot-link height above ground, clamped at `threshold`. | -| `motion_global_anchor_position_error_exp` | `command_name`, `std` | `exp(-||p_ref − p_robot||² / std²)` on the anchor body in world frame. | -| `motion_global_anchor_orientation_error_exp` | `command_name`, `std` | Geodesic rotation error on the anchor body, mapped through a Gaussian kernel. | -| `motion_relative_body_position_error_exp` | `command_name`, `std`, `body_names=None` | Multi-body L2 position error against anchor-aligned reference frames. `body_names=None` includes every tracked body. | -| `motion_relative_body_orientation_error_exp` | `command_name`, `std`, `body_names=None` | Multi-body geodesic rotation error against anchor-aligned references. | -| `motion_global_body_linear_velocity_error_exp` | `command_name`, `std`, `body_names=None` | L2 world-frame linear-velocity error across the tracked bodies. | -| `motion_global_body_angular_velocity_error_exp` | `command_name`, `std`, `body_names=None` | L2 world-frame angular-velocity error across the tracked bodies. | - -!!! warning "Placeholder rewards" - `joint_acc_l2` and `feet_air_time` are stubs awaiting proper accel / - contact buffers. They compile and emit a tensor of the correct shape but - the *value* is not the locomotion-grade signal one would normally expect - from these names. Treat them as `weight=0.0` placeholders until they are - wired against per-step velocity history (`joint_acc_l2`) and a - `ContactSensor` (`feet_air_time` — `mdp.foot_air_time` already provides - the real signal as an observation term). - -```python -from genelab.managers import RewardTermCfg -from genelab import mdp - -rewards_cfg = { - "track_lin_vel": RewardTermCfg( - func=mdp.track_linear_velocity_xy_exp, - weight=1.0, - params={"command_name": "base_velocity", "std": 0.25}, - ), - "track_ang_vel": RewardTermCfg( - func=mdp.track_angular_velocity_z_exp, - weight=0.5, - params={"command_name": "base_velocity", "std": 0.25}, - ), - "upright": RewardTermCfg(func=mdp.upright_exp, weight=0.2), - "action_rate": RewardTermCfg(func=mdp.action_rate_l2, weight=-0.005), - "flat_ori": RewardTermCfg(func=mdp.flat_orientation_l2, weight=-0.5), -} -``` - -## Terminations - -All termination functions return shape `(B,)` bool. `TerminationTermCfg.time_out` -routes the value to the truncation buffer (RSL-RL's "info['time_out']") instead -of the terminated buffer. - -| Name | Params | Behavior | -|---|---|---| -| `time_out` | — | `episode_length_buf >= max_episode_length`. Always paired with `time_out=True`. | -| `bad_orientation` | `limit_angle=math.radians(70.0)` | True when the body z-axis tilts more than `limit_angle` from world up. | -| `root_height_below` | `min_height` | True when `root_pos.z < min_height`. | -| `bad_anchor_pos_z_only` | `command_name`, `threshold` | True when the robot anchor z drifts further than `threshold` from the reference clip. | -| `bad_anchor_ori` | `command_name`, `threshold` | True when the tilt error between robot anchor and reference exceeds `threshold` (gravity-z proxy). | -| `bad_motion_body_pos_z_only` | `command_name`, `threshold`, `body_names=None` | True when any selected body's vertical position deviates past `threshold`. | - -```python -from genelab.managers import TerminationTermCfg -from genelab import mdp - -terminations_cfg = { - "time_out": TerminationTermCfg(func=mdp.time_out, time_out=True), - "fell_over": TerminationTermCfg(func=mdp.bad_orientation, - params={"limit_angle": 1.0}), -} -``` - -## Events - -Event functions take an extra `env_ids` argument and return `None`. The -`EventTermCfg.mode` decides when they fire — `startup` once at construct, -`reset` on every env reset, `interval` on per-env countdown (requires -`interval_range_s`). - -| Name | Params | Behavior | -|---|---|---| -| `reset_root_state_uniform` | `pose_range` (`x` / `y` / `z` / `roll` / `pitch` / `yaw` → `(low, high)` dict), `velocity_range` (same axes) | Randomize floating-base pose and velocity within the given ranges. Pose offsets are added to `cfg.robot.init_pos`; orientation offsets layer on top of `cfg.robot.init_quat`. | -| `reset_joints_to_default` | `pos_jitter=0.0`, `vel_jitter=0.0` | Write the default joint pose to the selected envs, optionally with uniform ±jitter on position and velocity. | -| `push_by_setting_velocity` | `velocity_range` (`x` / `y` / `z` / `roll` / `pitch` / `yaw` → `(low, high)`) | Overwrite the base linear and angular velocity. Combined with `mode="interval"`, this is the canonical "random push" disturbance. | - -```python -from genelab.managers import EventTermCfg -from genelab import mdp - -events_cfg = { - "reset_root": EventTermCfg( - mode="reset", - func=mdp.reset_root_state_uniform, - params={"pose_range": {"yaw": (-3.14, 3.14)}, - "velocity_range": {"x": (-0.1, 0.1)}}, - ), - "push_robot": EventTermCfg( - mode="interval", - func=mdp.push_by_setting_velocity, - interval_range_s=(8.0, 12.0), - params={"velocity_range": {"x": (-0.5, 0.5), "y": (-0.5, 0.5)}}, - ), -} -``` - -## Curriculums - -| Name | Params | Behavior | -|---|---|---| -| `terrain_levels_vel` | `distance_threshold`, `demote_ratio=0.5` | Promote / demote each env's `TerrainImporter.terrain_levels` index by how far it walked from spawn. Envs that travelled more than `distance_threshold` move up; envs that travelled less than `distance_threshold * demote_ratio` move down. Levels then drive a fresh spawn-origin lookup, and the curriculum writes the new root pose into the sim. No-op when `env.scene.terrain is None`. Returns the per-env mean level so the manager logs `Curriculum/`. | - -```python -from genelab.managers import CurriculumTermCfg -from genelab.mdp.curriculums import terrain_levels_vel +Common observation functions include base velocity, projected gravity, relative joint position and +velocity, last action, generated commands, sensor data, contact features, terrain height scans, and +motion-tracking state. -curriculum_cfg = { - "terrain_levels": CurriculumTermCfg( - func=terrain_levels_vel, - params={"distance_threshold": 5.0, "demote_ratio": 0.5}, - ), -} -``` +Observation functions should return `(num_envs, d)` or `(num_envs,)` tensors. The observation +manager handles optional noise, scaling, clipping, and group concatenation. -## Noise +## Rewards and terminations -`NoiseCfg` is the abstract base; concrete subclasses plug into -`ObservationTermCfg.noise` and only fire when the enclosing -`ObservationGroupCfg.enable_corruption=True`. +Reward functions cover velocity tracking, action smoothness, joint acceleration, orientation, +limits, foot clearance, slip, air time, self collision, angular momentum, and motion-tracking +errors. Termination functions cover time-out, orientation, root height, and motion-tracking failure. -| Name | Fields | Behavior | -|---|---|---| -| `NoiseCfg` | — | Abstract base; subclass and implement `apply(data) -> Tensor`. | -| `Unoise` | `n_min=-1.0`, `n_max=1.0` | Uniform additive noise sampled in `[n_min, n_max]`. | -| `Gnoise` | `mean=0.0`, `std=1.0` | Gaussian additive noise `N(mean, std²)`. | +Reward functions should return `(num_envs,)`; termination functions should return `(num_envs,)` +boolean tensors. -```python -from genelab.managers import ObservationGroupCfg, ObservationTermCfg -from genelab.mdp.noise import Gnoise, Unoise -from genelab import mdp +## Events, curricula, metrics, and noise -observations_cfg = { - "policy": ObservationGroupCfg( - enable_corruption=True, - terms={ - "base_lin_vel": ObservationTermCfg(func=mdp.base_lin_vel, - noise=Unoise(n_min=-0.1, n_max=0.1)), - "joint_pos_rel": ObservationTermCfg(func=mdp.joint_pos_rel, - noise=Gnoise(std=0.01)), - }, - ), -} -``` +| Area | Examples | +|---|---| +| Events | `reset_root_state_uniform`, `reset_joints_to_default`, `push_by_setting_velocity` | +| Curricula | `terrain_levels_vel`, `commands_vel` | +| Metrics | `mean_action_acc`, `angular_momentum_mean`, `air_time_mean`, `slip_velocity_mean` | +| Noise | `Unoise`, `Gnoise` | +| Domain randomization | `mdp.dr.body`, `mdp.dr.joint`, `mdp.dr.geom` | -## See also +## Where to continue - [Managers and MDP terms](managers.md) -- [Sensors](sensors.md) +- [Task design](../best-practices/task-design.md) - [API Reference](../api/reference.md) diff --git a/docs/concepts/mdp.zh.md b/docs/concepts/mdp.zh.md index 6fc7efc4..a5c261ad 100644 --- a/docs/concepts/mdp.zh.md +++ b/docs/concepts/mdp.zh.md @@ -1,273 +1,43 @@ # MDP term 参考 -`genelab.mdp` 是一组可直接插入七个 manager 的 Python 可调用对象。所有函数遵循相同 -签名形状 —— observation / reward / termination 是 `(env, **params) -> torch.Tensor`， -event 是 `(env, env_ids, **params) -> None`，curriculum 是 -`(env, env_ids, **params) -> Tensor` —— 并由相应 `*TermCfg.func`（或 `class_type`） -字段引用。 +`genelab.mdp` 是可复用 term 库。它本身不定义任务；任务配置从这个库里选择函数和类，并接到 manager 中。 -## 可复用 building block +## Actions 与 commands -下表列出的所有名字都已在 `genelab.mdp.__init__` 中 re-export，因此 -`from genelab import mdp` 就能拿到。完整模块结构： +| 区域 | 公开组件 | +|---|---| +| Actions | `JointPositionActionCfg`、`JointPositionAction` | +| 速度命令 | `UniformVelocityCommandCfg`、`UniformVelocityCommand` | +| Motion 命令 | `MotionCommandCfg`、`MotionCommand`、`MotionLoader` | -``` -genelab.mdp.actions → JointPositionAction(Cfg) -genelab.mdp.commands → UniformVelocityCommand(Cfg)、MotionCommand(Cfg)、MotionLoader -genelab.mdp.observations → 16 个 observation 函数 -genelab.mdp.rewards → 15 个 reward 函数 / 类 -genelab.mdp.terminations → 6 个 termination 函数 -genelab.mdp.events → 3 个 event 函数 -genelab.mdp.curriculums → 1 个 curriculum 函数 -genelab.mdp.noise → NoiseCfg 基类 + Unoise + Gnoise -``` - -下面 shape 列里 `B = num_envs`、`D = action / joint / sensor 维度`、 -`N = 身体段或脚的个数`。reward 与 termination term 总是返回 `(B,)`；observation -term 返回 `(B, D)`（或 `(B,)`，会被 `ObservationManager` 自动 unsqueeze 成 `(B, 1)`）。 - -## Actions - -| 名字 | 签名（用到的 Cfg 字段） | 行为 | -|---|---|---| -| `JointPositionAction` | `JointPositionActionCfg(joint_names, scale, use_default_offset, asset_name)` | `target = default + scale * raw_action`。目标通过 `Articulation.write_joint_targets_partial` 派发，按每个 joint 所在 actuator group 声明的通道执行（implicit PD 走 position，`IdealPDActuator` / `DCMotorActuator` 走 force）。`scale=None`（默认）从 `Articulation.action_scale_tensor` 继承逐 joint 比例；`float` 或 `dict[str, float]` 可覆盖。`joint_names` 为 regex，匹配不到任何 joint 会抛错。 | - -```python -from genelab.mdp.actions.joint_position import JointPositionActionCfg - -actions_cfg = { - "panda_arm": JointPositionActionCfg( - asset_name="robot", - joint_names=(r"^joint[1-7]$",), - use_default_offset=True, - ), -} -``` - -## Commands - -| 名字 | 签名（用到的 Cfg 字段） | 行为 | -|---|---|---| -| `UniformVelocityCommand` | `UniformVelocityCommandCfg(ranges, rel_standing_envs, heading_command, heading_control_stiffness, resampling_time_range, asset_name)` | 每 env 一组 body-frame `[lin_vel_x, lin_vel_y, ang_vel_z]`，每 `resampling_time_range` 秒从 `ranges` 均匀重采样。`heading_command=True`（默认）时，`ang_vel_z` 每步被重写以把 body 朝向目标 heading 拉。`rel_standing_envs` 把一定比例的重采样命令清零，让 policy 也见到 "stand still" 目标。`command` shape `(B, 3)`。 | -| `MotionCommand` | `MotionCommandCfg(motion_file, anchor_body_name, body_names, motion_body_order, motion_joint_order, pose_range, velocity_range, sampling_mode, ...)` | 按帧驱动 env 跟随预录动作 clip。读 mjlab `csv_to_npz` 产生的 NPZ schema。`motion_body_order` / `motion_joint_order` 描述 NPZ 内部 body / joint 轴的原生顺序（通常是 mjlab MJCF 的 DFS），`MotionLoader` 据此把数据重排到运行时机器人的顺序 —— 上游遍历顺序与 Genesis 不一致时必须传入。暴露 anchor 与多 body 的 reference / current 位姿 / 速度，供下节相对位姿 reward 函数使用。`sampling_mode="start"` 始终从第 0 帧开始；`"uniform"`（默认）随机抽帧。 | -| `MotionLoader` | `MotionLoader(motion_file, body_indexes, device, joint_perm=None)` | `MotionCommand` 内部用的辅助类。把 NPZ 读入设备 tensor，按可选的 `joint_perm` 重排 joint 轴，并把 `body_pos_w / body_quat_w / body_lin_vel_w / body_ang_vel_w` 切到要跟踪的几个 body。直接使用极少 —— 通常实例化 `MotionCommandCfg` 即可。 | - -```python -from genelab.mdp.commands.velocity_command import UniformVelocityCommandCfg - -commands_cfg = { - "base_velocity": UniformVelocityCommandCfg( - asset_name="robot", - resampling_time_range=(10.0, 10.0), - ranges=UniformVelocityCommandCfg.Ranges( - lin_vel_x=(-1.0, 1.0), lin_vel_y=(-0.5, 0.5), ang_vel_z=(-1.0, 1.0), - ), - heading_command=True, - ), -} -``` +action 把策略输出转换为仿真控制。command 持有 observation 和 reward 可读取的采样目标。 ## Observations -下列所有函数都接 `env` 加注明的 kwargs，并返回 shape `(B, D)` 或 `(B,)` 的 tensor -（后者会被 manager auto-unsqueeze 成 `(B, 1)`）。 - -| 名字 | Params | Shape | 来源 | -|---|---|---|---| -| `base_lin_vel` | — | `(B, 3)` | body-frame 浮动基线速度。 | -| `base_ang_vel` | — | `(B, 3)` | body-frame 浮动基角速度。 | -| `projected_gravity` | — | `(B, 3)` | 世界重力投影到 body frame（IMU 朝向 proxy）。 | -| `joint_pos_rel` | — | `(B, num_dofs)` | 关节位置减默认 pose。 | -| `joint_vel_rel` | — | `(B, num_dofs)` | 关节速度（默认为零，等同 raw `joint_vel`）。 | -| `last_action` | — | `(B, total_action_dim)` | 上一步 manager 处理的 action tensor。 | -| `generated_commands` | `command_name` | `(B, command_dim)` | `env.command_manager.get_command(command_name)`。 | -| `sensor_data` | `sensor_name` | 视 sensor 而定 | `env.sensors[sensor_name]` 上每步缓存的 tensor。适用于 IMU、FrameTransformer、ray-cast 等输出已是 tensor 的 sensor。 | -| `foot_air_time` | `sensor_name`（必须是 `ContactSensor`） | `(B, N_feet)` | 每脚的当前 air time（接触时为零）。 | -| `foot_contact` | `sensor_name`（必须是 `ContactSensor`） | `(B, N_feet)` | 接触二值 mask（float）。 | -| `foot_contact_forces` | `sensor_name`（必须是 `ContactSensor`） | `(B, N_feet * 3)` | 每脚接触力，按 `sign(f) * log1p(|f|)` 压缩并铺平。 | -| `height_scan` | `sensor_name`（必须是 `TerrainHeightSensor`） | `(B, num_rays)` | 每条射线相对地形的高度。 | -| `motion_anchor_pos_b` | `command_name`（必须是 `MotionCommand`） | `(B, 3)` | 参考 anchor 位置在 robot anchor frame 下的表示。 | -| `motion_anchor_ori_b` | `command_name` | `(B, 6)` | 参考 6D 朝向（rotation matrix 前两列）。 | -| `robot_body_pos_b` | `command_name` | `(B, N_bodies * 3)` | 每身体段在 robot anchor frame 下的位置（critic 用的特权观测）。 | -| `robot_body_ori_b` | `command_name` | `(B, N_bodies * 6)` | 每身体段在 robot anchor frame 下的 6D 朝向。 | - -```python -from genelab.managers import ObservationGroupCfg, ObservationTermCfg -from genelab import mdp -from genelab.mdp.noise import Unoise - -observations_cfg = { - "policy": ObservationGroupCfg( - terms={ - "base_lin_vel": ObservationTermCfg(func=mdp.base_lin_vel, - noise=Unoise(n_min=-0.1, n_max=0.1)), - "joint_pos_rel": ObservationTermCfg(func=mdp.joint_pos_rel), - "last_action": ObservationTermCfg(func=mdp.last_action), - "commands": ObservationTermCfg(func=mdp.generated_commands, - params={"command_name": "base_velocity"}), - }, - enable_corruption=True, - ), -} -``` - -## Rewards - -所有 reward 函数返回 shape `(B,)`。`RewardManager` 会乘 `RewardTermCfg.weight`，并在 -`scale_rewards_by_dt=True` 时再乘 `dt`。负 weight 把"偏差"型 reward 变成惩罚。 - -| 名字 | Params | 行为 | -|---|---|---| -| `track_linear_velocity_xy_exp` | `command_name`、`std=0.5` | `exp(-||cmd_xy − vel_xy||² / std²)`。 | -| `track_angular_velocity_z_exp` | `command_name`、`std=0.5` | `exp(-(cmd_z − vel_z)² / std²)`。 | -| `action_rate_l2` | — | `Σ_d (action_d − prev_action_d)²` —— 惩罚 action 抖动。 | -| `joint_acc_l2` | — | **占位**（当前返回零；待加 accel buffer）。 | -| `flat_orientation_l2` | — | `Σ (projected_gravity_xy)²` —— 惩罚倾斜。 | -| `upright_exp` | `std=0.45` | `exp(-||projected_gravity_xy||² / std²)` —— 站直时正 reward。 | -| `variable_posture` | `command_name`、`std_standing` / `std_walking` / `std_running`（regex → float 字典）、`default_std`、`walking_threshold`、`running_threshold` | 速度自适应姿态 reward：`exp(-mean((joint_pos − default)² / std²))`，每 env 按命令模长挑 std。class 风格 term —— 构造时一次性 `__init__`，每步走 `__call__`。 | -| `joint_pos_limits` | — | 关节位置超过 ±π 部分的 L2。 | -| `feet_air_time` | `threshold=0.4` | **stub** —— 用 foot link 平均高度近似 air-time，超过 `threshold` 饱和。 | -| `motion_global_anchor_position_error_exp` | `command_name`、`std` | 世界系 anchor 位置 `exp(-||p_ref − p_robot||² / std²)`。 | -| `motion_global_anchor_orientation_error_exp` | `command_name`、`std` | 世界系 anchor 朝向几何 rotation error 套 Gaussian 核。 | -| `motion_relative_body_position_error_exp` | `command_name`、`std`、`body_names=None` | anchor-aligned 多 body 位置 L2 误差。`body_names=None` 表示所有被跟踪 body。 | -| `motion_relative_body_orientation_error_exp` | `command_name`、`std`、`body_names=None` | anchor-aligned 多 body 几何 rotation 误差。 | -| `motion_global_body_linear_velocity_error_exp` | `command_name`、`std`、`body_names=None` | 世界系多 body 线速度 L2 误差。 | -| `motion_global_body_angular_velocity_error_exp` | `command_name`、`std`、`body_names=None` | 世界系多 body 角速度 L2 误差。 | - -!!! warning "占位 reward" - `joint_acc_l2` 与 `feet_air_time` 是 stub，等正经 accel / 接触缓冲落地。它们能正常 - 返回 shape 正确的 tensor，但**值本身不是**通常意义下的 locomotion 信号。除非已经 - 把它们接到逐步速度历史（`joint_acc_l2`）与 `ContactSensor`（`feet_air_time` —— - `mdp.foot_air_time` 作为 observation term 已经能给真实信号），否则按 `weight=0.0` - 占位处理。 - -```python -from genelab.managers import RewardTermCfg -from genelab import mdp - -rewards_cfg = { - "track_lin_vel": RewardTermCfg( - func=mdp.track_linear_velocity_xy_exp, - weight=1.0, - params={"command_name": "base_velocity", "std": 0.25}, - ), - "track_ang_vel": RewardTermCfg( - func=mdp.track_angular_velocity_z_exp, - weight=0.5, - params={"command_name": "base_velocity", "std": 0.25}, - ), - "upright": RewardTermCfg(func=mdp.upright_exp, weight=0.2), - "action_rate": RewardTermCfg(func=mdp.action_rate_l2, weight=-0.005), - "flat_ori": RewardTermCfg(func=mdp.flat_orientation_l2, weight=-0.5), -} -``` - -## Terminations - -所有 termination 函数返回 shape `(B,)` bool。`TerminationTermCfg.time_out` 把返回值 -送进 truncation buffer（RSL-RL 的 `info["time_out"]`），而不是 terminated buffer。 - -| 名字 | Params | 行为 | -|---|---|---| -| `time_out` | — | `episode_length_buf >= max_episode_length`。永远配 `time_out=True`。 | -| `bad_orientation` | `limit_angle=math.radians(70.0)` | body z 轴相对世界向上方向倾斜超过 `limit_angle` 时为 True。 | -| `root_height_below` | `min_height` | `root_pos.z < min_height` 时为 True。 | -| `bad_anchor_pos_z_only` | `command_name`、`threshold` | robot anchor z 相对 reference clip 漂移超过 `threshold` 时为 True。 | -| `bad_anchor_ori` | `command_name`、`threshold` | robot anchor 与 reference 倾斜误差（重力 z 投影 proxy）超过 `threshold` 时为 True。 | -| `bad_motion_body_pos_z_only` | `command_name`、`threshold`、`body_names=None` | 任一选中 body 的垂直位置偏离参考超过 `threshold` 时为 True。 | - -```python -from genelab.managers import TerminationTermCfg -from genelab import mdp - -terminations_cfg = { - "time_out": TerminationTermCfg(func=mdp.time_out, time_out=True), - "fell_over": TerminationTermCfg(func=mdp.bad_orientation, - params={"limit_angle": 1.0}), -} -``` - -## Events - -Event 函数多接一个 `env_ids` 参数，返回 `None`。`EventTermCfg.mode` 决定触发时机 —— -`startup` 构造期一次、`reset` 每次 env reset、`interval` 每 env 倒计时（需要 -`interval_range_s`）。 - -| 名字 | Params | 行为 | -|---|---|---| -| `reset_root_state_uniform` | `pose_range`（`x` / `y` / `z` / `roll` / `pitch` / `yaw` → `(low, high)` 字典）、`velocity_range`（同样的轴） | 在给定范围内随机化浮动基位姿与速度。位姿偏移叠加在 `cfg.robot.init_pos` 上；朝向偏移叠加在 `cfg.robot.init_quat` 上。 | -| `reset_joints_to_default` | `pos_jitter=0.0`、`vel_jitter=0.0` | 把默认关节 pose 写入选中 env，可选叠加位置与速度的 uniform ±jitter。 | -| `push_by_setting_velocity` | `velocity_range`（`x` / `y` / `z` / `roll` / `pitch` / `yaw` → `(low, high)`） | 覆盖 base 线速度与角速度。配合 `mode="interval"` 就是标准的 "随机推" 扰动。 | - -```python -from genelab.managers import EventTermCfg -from genelab import mdp - -events_cfg = { - "reset_root": EventTermCfg( - mode="reset", - func=mdp.reset_root_state_uniform, - params={"pose_range": {"yaw": (-3.14, 3.14)}, - "velocity_range": {"x": (-0.1, 0.1)}}, - ), - "push_robot": EventTermCfg( - mode="interval", - func=mdp.push_by_setting_velocity, - interval_range_s=(8.0, 12.0), - params={"velocity_range": {"x": (-0.5, 0.5), "y": (-0.5, 0.5)}}, - ), -} -``` - -## Curriculums - -| 名字 | Params | 行为 | -|---|---|---| -| `terrain_levels_vel` | `distance_threshold`、`demote_ratio=0.5` | 按每 env 离 spawn 走了多远，调整 `TerrainImporter.terrain_levels` 索引。走超过 `distance_threshold` 上一档；走少于 `distance_threshold * demote_ratio` 下一档。新 level 触发 spawn origin 重查，curriculum 把新 root pose 写回 sim。`env.scene.terrain is None` 时整体 no-op。返回每 env 平均 level，manager 自动按 `Curriculum/` 写日志。 | - -```python -from genelab.managers import CurriculumTermCfg -from genelab.mdp.curriculums import terrain_levels_vel +常见 observation 函数包括 base velocity、projected gravity、relative joint position/velocity、 +last action、generated commands、sensor data、contact 特征、terrain height scan 和 motion-tracking 状态。 -curriculum_cfg = { - "terrain_levels": CurriculumTermCfg( - func=terrain_levels_vel, - params={"distance_threshold": 5.0, "demote_ratio": 0.5}, - ), -} -``` +observation 函数应返回 `(num_envs, d)` 或 `(num_envs,)` tensor。Observation manager 负责可选噪声、缩放、裁剪和 group concat。 -## Noise +## Rewards 与 terminations -`NoiseCfg` 是抽象基类；具体子类挂在 `ObservationTermCfg.noise` 上，仅当所在 -`ObservationGroupCfg.enable_corruption=True` 时生效。 +reward 函数覆盖速度跟踪、action 平滑、关节加速度、姿态、关节限制、足端 clearance、slip、air time、 +self collision、角动量和 motion-tracking 误差。termination 函数覆盖 time-out、姿态、root height 和 motion-tracking 失败。 -| 名字 | 字段 | 行为 | -|---|---|---| -| `NoiseCfg` | — | 抽象基类；继承并实现 `apply(data) -> Tensor`。 | -| `Unoise` | `n_min=-1.0`、`n_max=1.0` | uniform 噪声 `[n_min, n_max]`，加性叠加。 | -| `Gnoise` | `mean=0.0`、`std=1.0` | Gaussian 噪声 `N(mean, std²)`，加性叠加。 | +reward 函数应返回 `(num_envs,)`；termination 函数应返回 `(num_envs,)` bool tensor。 -```python -from genelab.managers import ObservationGroupCfg, ObservationTermCfg -from genelab.mdp.noise import Gnoise, Unoise -from genelab import mdp +## Events、curricula、metrics、noise -observations_cfg = { - "policy": ObservationGroupCfg( - enable_corruption=True, - terms={ - "base_lin_vel": ObservationTermCfg(func=mdp.base_lin_vel, - noise=Unoise(n_min=-0.1, n_max=0.1)), - "joint_pos_rel": ObservationTermCfg(func=mdp.joint_pos_rel, - noise=Gnoise(std=0.01)), - }, - ), -} -``` +| 区域 | 示例 | +|---|---| +| Events | `reset_root_state_uniform`、`reset_joints_to_default`、`push_by_setting_velocity` | +| Curricula | `terrain_levels_vel`、`commands_vel` | +| Metrics | `mean_action_acc`、`angular_momentum_mean`、`air_time_mean`、`slip_velocity_mean` | +| Noise | `Unoise`、`Gnoise` | +| Domain randomization | `mdp.dr.body`、`mdp.dr.joint`、`mdp.dr.geom` | -## See also +## 继续阅读 -- [Managers and MDP terms](managers.md) -- [Sensors](sensors.md) -- [API Reference](../api/reference.md) +- [Manager 与 MDP term](managers.md) +- [设计任务](../best-practices/task-design.md) +- [API 参考](../api/reference.md) diff --git a/docs/concepts/recording.en.md b/docs/concepts/recording.en.md index a7d47eee..1798767d 100644 --- a/docs/concepts/recording.en.md +++ b/docs/concepts/recording.en.md @@ -1,185 +1,41 @@ -# Recording and live plotting +# Recording and Live Plotting -`genelab.recording` packages Genesis's recorder + live-plotter stack behind a -declarative `RecordingCfg`. A scene can declare one or more recordings, each pairing a -data source (sensor, articulation field, or custom callable) with one or more output -sinks (PyQt/Matplotlib plot, NPZ/CSV file, MP4/AVI video). The Genesis recorder pipeline -does the actual sampling on its own threads — Genelab just describes what to record and -wires the env lifecycle. - -## Quick start - -```python -from genelab.configs import InteractiveSceneCfg, SimulationCfg -from genelab.recording import NPZFileCfg, PyQtPlotCfg, RecordingCfg -from genelab.sensor import IMUSensorCfg - -scene = InteractiveSceneCfg( - sensors=(IMUSensorCfg(name="cart_imu", link_name="cart"),), - recordings=( - RecordingCfg( - name="cart_acc", - source="cart_imu", - field="lin_acc_b", - outputs=( - PyQtPlotCfg(title="cart linear acc", labels=("ax", "ay", "az")), - NPZFileCfg(filename="logs/cart_acc.npz"), - ), - ), - ), -) -``` - -Running `env.step(...)` then samples `cart_imu.data.lin_acc_b` once per control step, -streams it to a live PyQt window, and accumulates an `.npz` file written when the env -is closed. No manual data plumbing in your runner. +Recording turns selected runtime signals into live plots, files, or video without embedding output +code in the environment loop. ## Mental model -* **One `RecordingCfg` = one source × N sinks.** Outputs share the same data callable. -* **Registration is scene-build-time.** Genesis's `add_recorder` is `@assert_unbuilt`, - so recordings land in `InteractiveSceneCfg` (next to `sensors`), not on the env-level - `*_cfg` dicts. A small `RecorderBridge` carries the not-yet-bound env reference - through to the callables. -* **Sampling happens inside `gs_scene.step()`.** Recorders run on Genesis threads — - PyQt plots update without blocking the env loop. - -## Sources - -A `RecordingCfg.source` is one of: - -* **Sensor name (string).** Combined with `field=...` to pick a tensor from the sensor's - `data` payload (`"lin_acc_b"`, `"orientation"`, `"rgb"`, etc.). If `field` is omitted, - `sensor.data` must already be a plain tensor — dataclass payloads raise a clear error. -* **Callable.** Takes either zero arguments or a single `env` argument; arity is - detected once at registration. Useful for raw articulation state or computed signals - that don't have a sensor wrapper. - -```python -# sensor + dotted field -RecordingCfg(name="imu_ori", source="imu", field="orientation", outputs=(...,)) - -# nullary callable -RecordingCfg(name="wall_clock", source=time.monotonic, outputs=(...,)) - -# env-aware callable -RecordingCfg( - name="q0", - source=lambda env: env.robot_state.joint_pos[:, 0], - outputs=(...,), -) -``` - -`RecordingCfg.env_idx` (default 0) squeezes a multi-env leading dimension to a single -env before each sample reaches the sinks; pass `env_idx=None` to keep the full batch -(useful when dumping all envs to NPZ). - -## Output reference - -| Cfg | Backend | When to use | -|-----|---------|-------------| -| `PyQtPlotCfg` | `pyqtgraph` (threaded) | Default for live plots — works on every platform Genesis supports. | -| `MPLPlotCfg` | `matplotlib` | Fallback when PyQt is unavailable; runs on the main thread on macOS (may stutter). | -| `NPZFileCfg` | `numpy.savez_compressed` | Buffered; one file per recording, flushed on `env.close()` (or per-episode with `save_on_reset=True`). | -| `CSVFileCfg` | `csv.writer` | Row-streamed; pass `header=("a","b",...)` for column names, `save_every_write=True` to flush each row. | -| `VideoFileCfg` | `cv2.VideoWriter` | MP4/AVI from a `CameraSensor`; only pairs with camera sources. | - -All five cfgs accept the common `hz`, `buffer_size`, and `buffer_full_wait_time` knobs -from `RecorderOptions`. Setting `hz` explicitly always wins over the Genelab defaults. - -## Sampling cadence and the sensor cache - -`Sensor.data` is cached per control step. If a recorder sampled every physics tick it -would record `decimation` identical rows per control step. Genelab fixes this for you: - -* **Sensor-name sources** default to `hz = 1 / (sim.dt * decimation)` (the control rate). -* **Callable sources** default to `hz = None` (every physics tick) — the assumption is - that the callable reads raw articulation state that genuinely changes each tick. +`RecordingCfg` describes one data source and one or more output sinks. The scene converts these +configs into Genesis recorders during build, then binds them to the live env after sensors exist. -Override either by setting `hz` on the output cfg directly: - -```python -NPZFileCfg(filename="raw.npz", hz=200.0) # explicit 200 Hz regardless of sim.dt -``` - -## Episode boundaries and `save_on_reset` - -When at least one output has `save_on_reset=True`, the env's auto-reset path calls -`gs_scene._recorder_manager.reset()` to flush and rotate file outputs (NPZ counter, -new CSV header row). The very first reset inside `ManagerBasedRlEnv.__init__` is -skipped so file counters start at 0, matching user expectations -(`pole_0.npz`, `pole_1.npz`, …). The final partial buffer is flushed by -`env.close()`. - -Plotters are unaffected by reset (history is rolling). - -## Multi-env - -Plotters always read a single env. Use `env_idx` to pick which one — the default 0 is -almost always what you want when scaling envs for RL while debugging. - -File writers can record all envs at once: set `env_idx=None` and they store the full -`(num_envs, ...)` tensor. Plot outputs combined with `env_idx=None` will fail at the -first sample — split the recording. - -## Cameras and video - -A `CameraSensor` source paired with a single `VideoFileCfg` writes a one-env video: - -```python -RecordingCfg( - name="wrist_video", - source="wrist_cam", - outputs=(VideoFileCfg(filename="logs/wrist.mp4", env_idx=0, fps=30),), -) -``` - -Caveats: - -* The camera must have `render_rgb=True` — depth-only cameras are rejected at runtime. -* `BatchRenderer` is Linux x86-64 + CUDA only. On macOS the default Rasterizer still - produces a single-env `(1, H, W, 3)` tensor that `env_idx=0` slices correctly. -* Pairing a plotter with a camera source is rejected at registration — plotters can't - consume H×W×3 frames. - -## Custom callables and arbitrary signals - -Any quantity Genelab exposes on the env can become a recording. Examples: - -```python -# joint torque (post-actuator) -RecordingCfg( - name="joint_effort", - source=lambda env: env.robot_state.dof_force[:, env.joint_names.index("joint1")], - outputs=(NPZFileCfg(filename="logs/effort.npz"),), -) - -# reward term snapshot -RecordingCfg( - name="track_reward", - source=lambda env: env.reward_manager._term_sums["track_lin_vel"], - outputs=(PyQtPlotCfg(title="track reward"),), -) +```text +RecordingCfg(source, field, env_idx) +└── PyQtPlotCfg / MPLPlotCfg / NPZFileCfg / CSVFileCfg / VideoFileCfg ``` -The callable runs on every recorder tick (every physics step by default). Keep it -cheap — heavy compute belongs in a Genelab sensor. +## Sources -## Optional dependencies +| Source | Meaning | +|---|---| +| Sensor name | Read `env.sensors[name].data`, optionally walking `field`. | +| Callable with no args | Called directly. | +| Callable with `env` arg | Called with the live `ManagerBasedRlEnv`. | -Live plot backends live behind a `recording` extra on Genelab itself: +`env_idx` squeezes a batched tensor for per-env plotting. Use `env_idx=None` when the source already +returns a scalar or when a file should capture the full batch. -```bash -pip install "genelab[recording]" # pulls pyqtgraph + PyQt5 + matplotlib -``` +## Outputs -The showcase package depends on `genelab[recording]` transitively, so -`uv pip install -e examples/genelab_showcase` already pulls them in — no manual step. -File-only recordings (NPZ / CSV / VideoFile) need no extra dependencies beyond -Genelab's existing stack (numpy + PyAV, both Genesis transitive deps). +| Output | Use case | +|---|---| +| `PyQtPlotCfg` | Live PyQtGraph plots. | +| `MPLPlotCfg` | Live Matplotlib plots. | +| `NPZFileCfg` | Compressed array dumps at cleanup/reset. | +| `CSVFileCfg` | Row-oriented streaming output. | +| `VideoFileCfg` | Camera frames to `.mp4`. | -## See also +## Where to continue - [Sensors](sensors.md) -- [Scene and entities](scene.md) -- The `Showcase-Recording` task in [Examples → Showcase](../examples/showcase.md). +- [Debug Common Failures](../best-practices/debugging.md) +- [API Reference](../api/reference.md) diff --git a/docs/concepts/recording.zh.md b/docs/concepts/recording.zh.md index c3fc4d7f..1c4ae0c4 100644 --- a/docs/concepts/recording.zh.md +++ b/docs/concepts/recording.zh.md @@ -1,176 +1,38 @@ # 数据录制与实时绘图 -`genelab.recording` 将 Genesis 的录制器 + 实时绘图栈封装为声明式 `RecordingCfg`。 -一个场景可以声明多个录制项，每项把一个数据源（传感器、articulation 字段、或自定义 -回调）连接到一个或多个输出（PyQt/Matplotlib 实时图、NPZ/CSV 文件、MP4/AVI 视频）。 -真正的采样由 Genesis 的录制器线程完成 —— Genelab 只负责描述要录什么、把 env 的 -生命周期接好。 - -## 快速开始 - -```python -from genelab.configs import InteractiveSceneCfg, SimulationCfg -from genelab.recording import NPZFileCfg, PyQtPlotCfg, RecordingCfg -from genelab.sensor import IMUSensorCfg - -scene = InteractiveSceneCfg( - sensors=(IMUSensorCfg(name="cart_imu", link_name="cart"),), - recordings=( - RecordingCfg( - name="cart_acc", - source="cart_imu", - field="lin_acc_b", - outputs=( - PyQtPlotCfg(title="cart linear acc", labels=("ax", "ay", "az")), - NPZFileCfg(filename="logs/cart_acc.npz"), - ), - ), - ), -) -``` - -每次 `env.step(...)` 就会按控制步采样 `cart_imu.data.lin_acc_b`，实时推送到 PyQt -窗口，并累积一个 `.npz` 文件在 `env.close()` 时写出。runner 里不需要手写任何数据 -搬运代码。 +Recording 把选定运行时信号变成实时曲线、文件或视频，而不需要把输出代码嵌入环境循环。 ## 心智模型 -* **一个 `RecordingCfg` = 一个数据源 × N 个输出**。所有输出共享同一个数据 callable。 -* **注册发生在 scene-build 时刻**。Genesis 的 `add_recorder` 标了 `@assert_unbuilt`， - 所以录制配置放在 `InteractiveSceneCfg`（紧挨 `sensors`），不在 env 层的 - `*_cfg` 字典里。小小的 `RecorderBridge` 把还未构造的 env 引用透传给回调。 -* **采样在 `gs_scene.step()` 内部触发**。录制器跑在 Genesis 的线程上 —— PyQt 实时 - 图不会阻塞 env 主循环。 - -## 数据源 - -`RecordingCfg.source` 可以是： - -* **传感器名（字符串）**。配合 `field=...` 从该传感器的 `data` 里取一个张量 - （`"lin_acc_b"`、`"orientation"`、`"rgb"` 等）。如果省略 `field`，要求 - `sensor.data` 本身就是普通张量 —— 数据类负载会抛出清晰的错误。 -* **可调用对象**。零参或单 `env` 参数；arity 在注册时检测一次。适合裸的 - articulation 状态或没有专门 sensor 包装的派生信号。 - -```python -# 传感器 + dotted field -RecordingCfg(name="imu_ori", source="imu", field="orientation", outputs=(...,)) - -# 零参 callable -RecordingCfg(name="wall_clock", source=time.monotonic, outputs=(...,)) - -# 带 env 的 callable -RecordingCfg( - name="q0", - source=lambda env: env.robot_state.joint_pos[:, 0], - outputs=(...,), -) -``` - -`RecordingCfg.env_idx`（默认 0）会把多 env 的首维度切片到单 env 再交给下游 sink； -传 `env_idx=None` 可保留完整批次（适合把所有 env 一次性 dump 到 NPZ）。 - -## 输出参考 - -| Cfg | 后端 | 适用场景 | -|-----|------|---------| -| `PyQtPlotCfg` | `pyqtgraph`（线程化） | 实时绘图首选 —— Genesis 支持的所有平台都能用 | -| `MPLPlotCfg` | `matplotlib` | PyQt 不可用时的 fallback；macOS 下走主线程，可能卡顿 | -| `NPZFileCfg` | `numpy.savez_compressed` | 缓存到内存，`env.close()` 时一次写出（或开启 `save_on_reset=True` 按 episode 切文件）| -| `CSVFileCfg` | `csv.writer` | 行流式；`header=("a","b",...)` 设列名，`save_every_write=True` 每行 flush | -| `VideoFileCfg` | `cv2.VideoWriter` | 来自 `CameraSensor` 的 MP4/AVI；只能配相机源 | - -所有五个 cfg 都接受 `RecorderOptions` 的公共字段：`hz`、`buffer_size`、 -`buffer_full_wait_time`。显式设置 `hz` 总会覆盖 Genelab 的默认策略。 - -## 采样频率与传感器缓存 - -`Sensor.data` 在每个控制步缓存。如果录制器按物理 tick 采样，会得到 `decimation` 个 -重复行。Genelab 自动处理这一点： - -* **传感器源**默认 `hz = 1 / (sim.dt * decimation)`（即控制率）。 -* **Callable 源**默认 `hz = None`（每个物理 tick 都采）—— 假设 callable 读裸状态， - 每个 tick 都会变。 +`RecordingCfg` 描述一个数据源和一个或多个输出 sink。scene 在 build 时把这些配置转换成 Genesis recorder，并在传感器存在后绑定到 live env。 -显式重写时直接在输出 cfg 上设 `hz`： - -```python -NPZFileCfg(filename="raw.npz", hz=200.0) # 强制 200 Hz，无视 sim.dt -``` - -## Episode 边界与 `save_on_reset` - -当至少有一个输出开了 `save_on_reset=True`，env 的 auto-reset 会调用 -`gs_scene._recorder_manager.reset()`，flush 并轮换文件输出（NPZ 计数器自增、CSV -重写表头）。`ManagerBasedRlEnv.__init__` 里的首次 reset 会被跳过，所以文件计数从 0 -开始，符合直觉（`pole_0.npz`、`pole_1.npz`、…）。最后没写完的缓存由 `env.close()` -flush 出去。 - -绘图器不受 reset 影响（历史是滚动窗口）。 - -## 多 env - -绘图器永远只读一个 env。用 `env_idx` 选哪个 —— 默认 0，几乎总是 RL 调试时要的。 - -文件写出器可以一次记录所有 env：设 `env_idx=None`，就会存完整 `(num_envs, ...)` -张量。把绘图器和 `env_idx=None` 配在一起会在首次采样时报错 —— 拆成两个 recording。 - -## 相机与视频 - -一个 `CameraSensor` 源配单个 `VideoFileCfg` 就能写出单 env 的视频： - -```python -RecordingCfg( - name="wrist_video", - source="wrist_cam", - outputs=(VideoFileCfg(filename="logs/wrist.mp4", env_idx=0, fps=30),), -) -``` - -注意： - -* 相机必须开 `render_rgb=True` —— 仅深度相机会在运行时报错。 -* `BatchRenderer` 只支持 Linux x86-64 + CUDA。macOS 下默认 Rasterizer 仍会输出 - 单 env 的 `(1, H, W, 3)` 张量，`env_idx=0` 正确切片。 -* 绘图器配相机源会在注册阶段被拒绝 —— 绘图器吃不下 H×W×3 帧。 - -## 自定义 callable 和任意信号 - -任何 env 上能取到的量都能录。例子： - -```python -# 关节扭矩（actuator 之后） -RecordingCfg( - name="joint_effort", - source=lambda env: env.robot_state.dof_force[:, env.joint_names.index("joint1")], - outputs=(NPZFileCfg(filename="logs/effort.npz"),), -) - -# 奖励项快照 -RecordingCfg( - name="track_reward", - source=lambda env: env.reward_manager._term_sums["track_lin_vel"], - outputs=(PyQtPlotCfg(title="track reward"),), -) +```text +RecordingCfg(source, field, env_idx) +└── PyQtPlotCfg / MPLPlotCfg / NPZFileCfg / CSVFileCfg / VideoFileCfg ``` -Callable 在每次录制 tick 都会跑（默认每个物理步）。保持轻量 —— 重计算应该放进 -Genelab sensor。 +## 数据源 -## 可选依赖 +| Source | 含义 | +|---|---| +| 传感器名 | 读取 `env.sensors[name].data`，可选沿 `field` 取字段。 | +| 无参 callable | 直接调用。 | +| 带 `env` 参数的 callable | 用 live `ManagerBasedRlEnv` 调用。 | -实时绘图的后端都在 Genelab 的 `recording` extra 里： +`env_idx` 会压缩 batched tensor 的 env 维，用于单 env 绘图。源已经返回标量，或文件需要捕获完整 batch 时，使用 `env_idx=None`。 -```bash -pip install "genelab[recording]" # 拉 pyqtgraph + PyQt5 + matplotlib -``` +## 输出 -showcase 包通过 `genelab[recording]` 间接依赖，`uv pip install -e -examples/genelab_showcase` 已经会自动带上 —— 不需要手装。只写文件的录制 -（NPZ / CSV / VideoFile）不需要额外依赖（numpy + PyAV 都是 Genesis 自带）。 +| Output | 用途 | +|---|---| +| `PyQtPlotCfg` | 实时 PyQtGraph 曲线。 | +| `MPLPlotCfg` | 实时 Matplotlib 曲线。 | +| `NPZFileCfg` | cleanup/reset 时写压缩数组。 | +| `CSVFileCfg` | 行式流式输出。 | +| `VideoFileCfg` | 相机帧写 `.mp4`。 | -## 另见 +## 继续阅读 - [传感器](sensors.md) -- [Scene 与实体](scene.md) -- Examples → Showcase 中的 `Showcase-Recording` 任务。 +- [调试常见问题](../best-practices/debugging.md) +- [API 参考](../api/reference.md) diff --git a/docs/concepts/registry.en.md b/docs/concepts/registry.en.md index 4e057c80..1745fe1c 100644 --- a/docs/concepts/registry.en.md +++ b/docs/concepts/registry.en.md @@ -1,50 +1,46 @@ # Registry -GeneLab provides a generic `Registry[T]` plus three module-level singletons exported from -`genelab.lab`: +GeneLab uses small registries to keep the framework separate from downstream robot projects. +The core package does not need to know every robot, environment, or task at import time; extensions +register named factories when the CLI or user code asks for them. -| Singleton | Holds | -|-----------|-------| -| `ROBOTS` | Robot factories. | -| `ENVS` | Environment factories. | -| `TASKS` | Task factories (each task pairs an environment with optional runner / agent config). | +## Why registries exist -## Entry shape +Robotics projects often mix three concerns: asset definitions, environment construction, and +experiment execution. GeneLab separates them: -A registry entry is a 4-tuple: **`(name, description, factory, cfg_type)`**. The `factory` is a -callable invoked lazily on `get(name)`, so importing a registration site does not eagerly build -heavy objects. +| Registry | Boundary | +|---|---| +| `ROBOTS` | Named robot/asset config factories. | +| `ENVS` | Named environment factories. | +| `TASKS` | Named task factories that carry `TaskCfg` and runtime methods. | -## API surface +This gives the CLI a stable discovery model without forcing a central list of all downstream +projects into `genelab` itself. -```python -from genelab.lab import ROBOTS, ENVS, TASKS, TaskCfg +## Lazy factories -def make_my_robot(): - ... +Registry entries store factories rather than constructed objects. Listing names and descriptions is +cheap; expensive work such as importing Genesis, downloading assets, or building a scene happens +only when the factory is called. -ROBOTS.register( - name="my_robot", - description="A demo robot.", - factory=make_my_robot, - cfg_type=None, -) +This is why a command like `genelab list tasks` can be fast while `genelab play TASK` is allowed to +initialize the simulator. -# Later: -robot = ROBOTS.get("my_robot") -``` +## Extension loading -The `name` is the canonical lookup key; the `description` is what `genelab list robots` -displays. +Extensions can register in three ways: -## Idempotent extension loading +| Mechanism | Use case | +|---|---| +| `genelab.extensions` entry point | Daily use after installing a package. | +| `--import MODULE` | Local experiments, notebooks, and packages not installed as entry points. | +| `load_extension_module()` | Programmatic embedding. | -The registry module tracks `_loaded_extension_modules` and `_loaded_entrypoints` sets so that -repeated calls to load the same extension are no-ops. This matters when entry-point -auto-discovery (default) combines with explicit `--import` flags — both pathways end up calling -`load_extension_module`, but the second call is short-circuited. +Registration should be idempotent when a module may be loaded repeatedly in one process. -## See also +## Where to continue -- [Extensions](extensions.md) +- [Build an Extension Project](../best-practices/extension-projects.md) +- [Discovery: list and info](../cli/list-info.md) - [API Reference](../api/reference.md) diff --git a/docs/concepts/registry.zh.md b/docs/concepts/registry.zh.md index bb4ef964..fa647f1e 100644 --- a/docs/concepts/registry.zh.md +++ b/docs/concepts/registry.zh.md @@ -1,46 +1,41 @@ # 注册表 -GeneLab 提供泛型 `Registry[T]` 与三个由 `genelab.lab` 导出的模块级单例： +GeneLab 使用小型注册表把框架和下游机器人项目分开。核心包不需要在导入时知道所有机器人、 +环境和任务；扩展在 CLI 或用户代码需要时，把命名 factory 注册进来。 -| 单例 | 持有内容 | -|------|---------| -| `ROBOTS` | 机器人工厂。 | -| `ENVS` | 环境工厂。 | -| `TASKS` | 任务工厂（每个任务把环境与可选 runner / agent 配置打包）。 | +## 为什么需要注册表 -## 条目结构 +机器人项目常把三件事混在一起：资产定义、环境构造、实验执行。GeneLab 把它们拆开： -注册条目是一个四元组：**`(name, description, factory, cfg_type)`**。`factory` 是一个可调用， -只在 `get(name)` 时才会被调用，因此导入注册位点不会立即构造重型对象。 +| 注册表 | 边界 | +|---|---| +| `ROBOTS` | 命名机器人/资产配置 factory。 | +| `ENVS` | 命名环境 factory。 | +| `TASKS` | 命名任务 factory，持有 `TaskCfg` 与运行方法。 | -## 接口速览 +这样 CLI 可以有稳定的发现模型，而不用把所有下游项目硬编码进 `genelab`。 -```python -from genelab.lab import ROBOTS, ENVS, TASKS, TaskCfg +## 惰性 factory -def make_my_robot(): - ... +注册表条目保存 factory，而不是已经构造好的对象。列名字和描述很轻量；导入 Genesis、下载资产、 +构建 scene 这类昂贵工作只在 factory 被调用时发生。 -ROBOTS.register( - name="my_robot", - description="A demo robot.", - factory=make_my_robot, - cfg_type=None, -) +因此 `genelab list tasks` 可以很快，而 `genelab play TASK` 可以启动仿真器。 -# 之后： -robot = ROBOTS.get("my_robot") -``` +## 扩展加载 -`name` 是规范查找键；`description` 出现在 `genelab list robots` 输出里。 +扩展有三种注册方式： -## 幂等扩展加载 +| 机制 | 用途 | +|---|---| +| `genelab.extensions` entry point | 包安装后的日常使用。 | +| `--import MODULE` | 本地实验、notebook、未安装 entry point 的包。 | +| `load_extension_module()` | 程序内嵌入。 | -注册模块维护 `_loaded_extension_modules` 与 `_loaded_entrypoints` 两个集合，重复加载同一 -扩展会被短路。entry-point 自动发现（默认）与显式 `--import` 同时启用时，这一点尤其 -重要 —— 两条路径最终都会调用 `load_extension_module`，但第二次调用直接返回。 +当模块可能在同一进程中重复加载时，注册逻辑应保持幂等。 -## See also +## 继续阅读 -- [扩展加载](extensions.md) +- [构建扩展项目](../best-practices/extension-projects.md) +- [发现：list 与 info](../cli/list-info.md) - [API 参考](../api/reference.md) diff --git a/docs/concepts/rl-runner.en.md b/docs/concepts/rl-runner.en.md index 8ce8b555..43a5db6a 100644 --- a/docs/concepts/rl-runner.en.md +++ b/docs/concepts/rl-runner.en.md @@ -1,228 +1,42 @@ -# RL runner +# RL Runner -`genelab.rl` is the thin RSL-RL integration that turns a registered task into -a runnable PPO training or replay session. The CLI subcommands `play` and -`train` dispatch into the two entry points covered below; library callers can -also reach them directly. +`genelab.rl` connects registered tasks to RSL-RL. It is deliberately thin: GeneLab owns task +resolution, config mutation, VecEnv adaptation, logs, profiling hooks, and distributed launch +helpers; RSL-RL owns the learning algorithm. -## Module surface - -| Name | Purpose | -|---|---| -| `train_task(task_id, agent_cfg, **kwargs) -> Path` | Build the task's env, wrap it in `RslRlVecEnvWrapper`, and drive RSL-RL's `OnPolicyRunner.learn`. Returns the resolved log directory. | -| `play_task(task_id, **kwargs) -> None` | Build the task's env (using `play_env` when set), then loop `policy(obs) → env.step` until the viewer closes or `max_steps` is reached. Supports `"zero"` / `"random"` / `"trained"` policies. | -| `AgentKind` | `Literal["zero", "random", "trained"]`. | -| `RslRlOnPolicyRunnerCfg` | Top-level PPO config (extends `RslRlBaseRunnerCfg`); composes `RslRlModelCfg` for actor / critic and `RslRlPpoAlgorithmCfg` for the optimiser. | -| `RslRlVecEnvWrapper` | Adapter from `ManagerBasedRlEnv` to RSL-RL's `VecEnv` protocol. Dynamic subclass of `rsl_rl.env.VecEnv` when RSL-RL is installed. | -| `VecEnvBase` | Minimal stand-alone VecEnv ABC for non-RSL-RL callers. | -| `maybe_profile(...)` | Context manager wrapping `torch.profiler`. No-op when disabled or off rank 0. | -| `profiler_enabled()` | `True` when `GENELAB_PROFILE=1`. | - -`genelab.cli` wires `play` / `train` into `play_task` / `train_task` via -`_dispatch_play` / `_dispatch_train` in `src/genelab/cli/__init__.py:384,417`. - -## train_task - -`train_task` resolves the env cfg off the registered task, applies CLI -overrides, wraps the env, runs PPO, and returns the log directory. - -| Argument | Default | Effect | -|---|---|---| -| `task_id: str` | — | Required. Looked up in the `TASKS` registry. | -| `agent_cfg: RslRlOnPolicyRunnerCfg` | — | Required. The CLI pulls this off the task's `cfg.agent`; library callers pass it directly. | -| `num_envs: int \| None` | `None` | Overrides `env_cfg.simulation.num_envs`. | -| `max_iterations: int \| None` | `None` | Overrides `agent_cfg.max_iterations`. | -| `seed: int \| None` | `None` | Sets both `env_cfg.seed` and `agent_cfg.seed`. | -| `log_root: Path \| None` | `Path("logs/rsl_rl")` | Parent under which `/` is created. | -| `log_dir: Path \| None` | `None` | Pre-resolved final directory. Takes precedence over `log_root`. The torchrun relaunch path uses this so every rank lands in the same folder. | -| `resume_from: Path \| None` | `None` | Existing checkpoint to load before `runner.learn`. | -| `prof`, `prof_out`, `prof_wait`, `prof_warmup`, `prof_active`, `prof_repeat`, `prof_record_shapes`, `prof_with_stack` | `None` | Forwarded to `maybe_profile`. Each kwarg overrides the matching `GENELAB_PROFILE_*` env var; see "Profiling integration" below. | - -Side effects on rank 0: - -- Creates `/params/env.json` and `/params/agent.json` with a - recursive JSON dump of both cfgs. -- RSL-RL writes `/model_.pt` every `agent_cfg.save_interval` - iterations. -- TensorBoard event files land under `/` directly. - -When profiling is on, `train_task` hooks `RslRlVecEnvWrapper.step` so the -profiler schedule advances once per env step. To express the schedule in PPO -iterations instead, multiply `wait` / `warmup` / `active` by -`agent_cfg.num_steps_per_env`. - -## play_task - -`play_task` builds the env (preferring `task_cfg.play_env` when set), picks a -policy according to `agent`, then loops until the viewer closes. - -| Argument | Default | Effect | -|---|---|---| -| `task_id: str` | — | Required. | -| `checkpoint: Path \| None` | `None` | Required when `agent="trained"`. | -| `num_envs: int \| None` | `None` | Overrides `env_cfg.simulation.num_envs`. | -| `agent: "zero" \| "random" \| "trained" \| None` | `None` | Defaults to `"trained"` if `checkpoint` is given, else `"zero"`. | -| `agent_cfg: RslRlOnPolicyRunnerCfg \| None` | `None` | Required only for `agent="trained"`; falls back to `task.cfg.agent`. | -| `deterministic: bool` | `True` | Forwarded to RSL-RL inference helpers. | -| `max_steps: int \| None` | `None` | Cap the rollout. `None` runs until the viewer closes. | -| `prof*` | `None` | Same semantics as `train_task`. | - -The play loop polls `env.viewer_closed` every step, so a closed Genesis -viewer terminates the rollout cleanly without raising. `env.close()` runs in -a `finally` block. - -## RslRlOnPolicyRunnerCfg - -The top-level dataclass an extension registers as `TaskCfg.agent`. Train -relies on `class_name="OnPolicyRunner"`; an extension config with any other -runner class is rejected at dispatch. +## Training flow +```text +TASKS.get(task_id) +└── TaskCfg.env + TaskCfg.agent + └── ManagerBasedRlEnv + └── RslRlVecEnvWrapper + └── rsl_rl OnPolicyRunner.learn() ``` -RslRlOnPolicyRunnerCfg -├── (inherited from RslRlBaseRunnerCfg) -│ seed=42, num_steps_per_env=24, max_iterations=300 -│ obs_groups={"actor": ("policy",), "critic": ("critic",)} -│ save_interval=50, experiment_name="exp1", run_name="" -│ logger="tensorboard" | "wandb" -│ wandb_project="genelab", wandb_tags=() -│ resume=False, load_run=".*", load_checkpoint="model_.*.pt" -│ clip_actions: float | None, upload_model=False -├── class_name="OnPolicyRunner" -├── actor: RslRlModelCfg -├── critic: RslRlModelCfg -└── algorithm: RslRlPpoAlgorithmCfg -``` - -`RslRlModelCfg` knobs (per actor / critic): `hidden_dims=(128, 128, 128)`, -`activation="elu"`, `obs_normalization=False`, optional `cnn_cfg` / -`distribution_cfg` / `rnn_type` / `rnn_hidden_dim` / `rnn_num_layers`, -`class_name="MLPModel"`. Only `class_name`, `hidden_dims`, `activation`, -`obs_normalization`, and `distribution_cfg` reach the MLP constructor — other -fields are pruned by `_prune_model_cfg`. - -`RslRlPpoAlgorithmCfg` knobs: `num_learning_epochs=5`, `num_mini_batches=4`, -`learning_rate=1e-3`, `schedule="adaptive"|"fixed"`, `gamma=0.99`, `lam=0.95`, -`entropy_coef=0.005`, `desired_kl=0.01`, `max_grad_norm=1.0`, -`value_loss_coef=1.0`, `use_clipped_value_loss=True`, `clip_param=0.2`, -`optimizer="adam"|"adamw"|"sgd"|"rmsprop"`, `class_name="PPO"`. - -## AgentKind -`AgentKind = Literal["zero", "random", "trained"]`. `play_task` selects the -policy callable from `kind`: +The main process writes `params/env.json`, `params/agent.json`, TensorBoard events, profiler traces, +and checkpoints. -| Kind | Policy | Use case | -|---|---|---| -| `"zero"` | Returns `torch.zeros((num_envs, num_actions))`. | Visual smoke before any training; passive-pose viewer rollouts. | -| `"random"` | Returns `2.0 * rand(...) - 1.0` (uniform `[-1, 1]`). | Sanity-check action bounds and reward shaping. | -| `"trained"` | Loads the checkpoint, instantiates `OnPolicyRunner`, and calls `runner.get_inference_policy()`. | Inspect a trained policy. Requires `checkpoint`. | +## Playback flow -## VecEnv wrapper +`play_task` prefers `TaskCfg.play_env` when present. It selects a policy source: -`RslRlVecEnvWrapper` adapts `ManagerBasedRlEnv` to the interface RSL-RL's -`OnPolicyRunner` expects: - -| Attribute / method | Source | +| Agent | Source | |---|---| -| `num_envs`, `device`, `max_episode_length` | `env.num_envs`, `env.device`, `env.max_episode_length`. | -| `num_actions` | `env.action_manager.total_action_dim`. | -| `num_obs`, `num_privileged_obs` | Last-axis dim of the `"policy"` / `"critic"` observation group, pre-warmed via one `observation_manager.compute()` call at construct. | -| `reset()` | `env.reset()`. Observations are wrapped in a `TensorDict` when `tensordict` is installed; otherwise returned as a plain `dict`. | -| `step(actions)` | Clips actions to `clip_actions` when set, calls `env.step`, packs `(obs, reward, dones, extras)` with `dones = terminated \| truncated` (long) and `extras["time_outs"] = truncated`. | -| `episode_length_buf`, `cfg`, `unwrapped` | Pass-throughs to the underlying env. | +| `zero` | Returns zero actions. | +| `random` | Uniform random actions. | +| `trained` | Loads a checkpoint and calls RSL-RL inference policy. | -The wrapper dynamically subclasses `rsl_rl.env.VecEnv` at import time via -`_attach_rsl_rl_base()`. When RSL-RL is missing (e.g. unit tests on a -CPU-only host) it falls back to a plain object with the same attributes. +Playback exits cleanly when the Genesis viewer closes or when `max_steps` is reached. ## Distributed training -The CLI surfaces `--gpus N` on `train`. The flag is rewritten by -`_relaunch_under_torchrun` in `cli/__init__.py:457`: - -1. Pre-resolves the log directory under `//` - on the parent process so every rank picks up the same path. -2. Strips `--gpus N` from `sys.argv` to avoid an infinite relaunch loop. -3. Appends `--log-dir ` when the original argv did not already - carry one. -4. `execvp`s `python -m torch.distributed.run --standalone --nproc_per_node=N - -m genelab.cli `. - -```bash -uv run genelab train Genelab-Velocity-Flat-Unitree-G1-v0 --gpus 4 --num-envs 4096 -``` - -Inside each rank, `genelab.rl.distributed` reads `WORLD_SIZE` / `LOCAL_RANK` / -`RANK` from the environment: - -| Helper | Returns | -|---|---| -| `world_size()` | `int(os.environ["WORLD_SIZE"])` or `1`. | -| `local_rank()` | `int(os.environ["LOCAL_RANK"])` or `0`. | -| `global_rank()` | `int(os.environ["RANK"])` or `0`. | -| `is_distributed()` | `world_size() > 1`. | -| `is_main_process()` | `global_rank() == 0`. | -| `pin_cuda_device()` | `"cuda:0"` in distributed runs (the CLI bootstrap pins `CUDA_VISIBLE_DEVICES` per worker, so each rank sees a single device); `None` in single-GPU mode. | - -Only the main process writes checkpoints, TensorBoard events, profiler -traces, and the `params/*.json` dump. RSL-RL handles the inter-rank gradient -all-reduce internally. - -## Profiling integration - -`maybe_profile` wraps `torch.profiler.profile` with a `schedule` and a -TensorBoard trace handler. The CLI surfaces eight flags on both `play` and -`train`: - -| CLI flag | `maybe_profile` kwarg | Env var fallback | Default | -|---|---|---|---| -| `--prof` | `enabled` | `GENELAB_PROFILE=1` | off | -| `--prof-out` | `out_dir` | `GENELAB_PROFILE_OUT` | `logs/torch_profile` | -| `--prof-wait` | `wait` | `GENELAB_PROFILE_WAIT` | `10` | -| `--prof-warmup` | `warmup` | `GENELAB_PROFILE_WARMUP` | `5` | -| `--prof-active` | `active` | `GENELAB_PROFILE_ACTIVE` | `10` | -| `--prof-repeat` | `repeat` | `GENELAB_PROFILE_REPEAT` | `2` | -| `--prof-record-shapes` | `record_shapes` | `GENELAB_PROFILE_RECORD_SHAPES=1` | off | -| `--prof-with-stack` | `with_stack` | `GENELAB_PROFILE_WITH_STACK=1` | off | - -The schedule fires one trace per `wait + warmup + active` steps, repeated -`repeat` times. The handler writes `*.pt.trace.json` + `*.tar.gz` under -`out_dir`; `uv run genelab prof open ` launches TensorBoard against it. - -```bash -uv run genelab train Genelab-Velocity-Flat-Unitree-G1-v0 \ - --prof --prof-active 3 --prof-repeat 1 --max-iterations 10 -uv run genelab prof open logs/torch_profile -``` - -## Failure modes worth knowing - -!!! warning "Mismatched agent cfg class" - `_dispatch_train` rejects any `task.cfg.agent` whose type is not - `RslRlOnPolicyRunnerCfg`. Extensions importing a hand-rolled runner cfg - that "happens to" have the same fields are not accepted. - -!!! warning "agent='trained' requires --checkpoint" - `play_task` raises `SystemExit("agent='trained' requires a --checkpoint - path")` when the checkpoint is omitted. Either pass `--checkpoint` or - pick `--agent zero` / `--agent random` for the visual smoke. - -!!! warning "torchrun relaunch loops" - `_relaunch_under_torchrun` only re-execs when `--gpus N` is set *and* - `TORCHELASTIC_RUN_ID` is absent. Running `torchrun ... -m genelab.cli - train --gpus 4` (with `--gpus` left in argv inside the torchrun env) - skips the relaunch because the env var is now present; this is intended. - The infinite loop only happens if `--gpus` is forwarded *without* - torchrun's env vars — typically when wrapping `genelab` inside another - launcher. - -!!! tip "Profile a short run first" - Profiler traces grow fast. Start with `--prof --prof-active 3 - --prof-repeat 1 --max-iterations 10` to size the trace before running a - full PPO schedule. +`genelab train TASK --gpus N` relaunches the current command under `torchrun`. The parent computes a +shared log directory so every rank writes into the same run. `--num_envs` is total across ranks; +`--num_envs_per_gpu` is per rank. -## See also +## Where to continue +- [Run RL Experiments](../best-practices/rl-experiments.md) - [Play and Train](../cli/play-train.md) -- [Profiling](../cli/profiling.md) -- [Managers and MDP terms](managers.md) +- [API Reference](../api/reference.md) diff --git a/docs/concepts/rl-runner.zh.md b/docs/concepts/rl-runner.zh.md index a11a477b..8214dd52 100644 --- a/docs/concepts/rl-runner.zh.md +++ b/docs/concepts/rl-runner.zh.md @@ -1,214 +1,38 @@ # RL runner -`genelab.rl` 是把已注册 task 跑成 PPO 训练 / 回放会话的薄 RSL-RL 集成层。CLI 子命令 -`play` 与 `train` 直接派发到下面两个入口；库调用者也可直接用。 +`genelab.rl` 把已注册 task 接到 RSL-RL。它刻意保持很薄：GeneLab 负责 task 解析、配置修改、 +VecEnv 适配、日志、profiling hook 和分布式启动 helper；RSL-RL 负责学习算法。 -## 模块表面 - -| 名字 | 用途 | -|---|---| -| `train_task(task_id, agent_cfg, **kwargs) -> Path` | 构造 task 的 env、用 `RslRlVecEnvWrapper` 包装、驱动 RSL-RL 的 `OnPolicyRunner.learn`。返回解析后的 log 目录。 | -| `play_task(task_id, **kwargs) -> None` | 构造 task 的 env（设置了 `play_env` 时优先用它），按 `agent` 选 policy，循环 `policy(obs) → env.step` 直到 viewer 关闭或达到 `max_steps`。支持 `"zero"` / `"random"` / `"trained"` 三种 policy。 | -| `AgentKind` | `Literal["zero", "random", "trained"]`。 | -| `RslRlOnPolicyRunnerCfg` | 顶层 PPO 配置（继承 `RslRlBaseRunnerCfg`）；组合 actor / critic 的 `RslRlModelCfg` 与优化器的 `RslRlPpoAlgorithmCfg`。 | -| `RslRlVecEnvWrapper` | 把 `ManagerBasedRlEnv` 适配到 RSL-RL 的 `VecEnv` 协议。装了 RSL-RL 时动态成为 `rsl_rl.env.VecEnv` 的子类。 | -| `VecEnvBase` | 给非 RSL-RL 调用方的最小 VecEnv ABC。 | -| `maybe_profile(...)` | `torch.profiler` 的上下文管理器。关闭或非 rank 0 时 no-op。 | -| `profiler_enabled()` | `GENELAB_PROFILE=1` 时返回 `True`。 | - -`genelab.cli` 把 `play` / `train` 通过 `src/genelab/cli/__init__.py:384,417` 中的 -`_dispatch_play` / `_dispatch_train` 接到 `play_task` / `train_task`。 - -## train_task - -`train_task` 从已注册 task 上解析 env cfg、应用 CLI override、包装 env、跑 PPO 并 -返回 log 目录。 - -| 参数 | 默认 | 行为 | -|---|---|---| -| `task_id: str` | — | 必填。在 `TASKS` 注册表里查找。 | -| `agent_cfg: RslRlOnPolicyRunnerCfg` | — | 必填。CLI 从 task 的 `cfg.agent` 上拿；库调用者直接传。 | -| `num_envs: int \| None` | `None` | 覆盖 `env_cfg.simulation.num_envs`。 | -| `max_iterations: int \| None` | `None` | 覆盖 `agent_cfg.max_iterations`。 | -| `seed: int \| None` | `None` | 同时设置 `env_cfg.seed` 与 `agent_cfg.seed`。 | -| `log_root: Path \| None` | `Path("logs/rsl_rl")` | 在该目录下创建 `/`。 | -| `log_dir: Path \| None` | `None` | 预解析的最终目录，优先级高于 `log_root`。torchrun 重启路径用这个让每个 rank 落在同一文件夹。 | -| `resume_from: Path \| None` | `None` | `runner.learn` 之前要加载的已有 checkpoint。 | -| `prof`、`prof_out`、`prof_wait`、`prof_warmup`、`prof_active`、`prof_repeat`、`prof_record_shapes`、`prof_with_stack` | `None` | 转发给 `maybe_profile`。每个 kwarg 覆盖对应的 `GENELAB_PROFILE_*` env var；参见下面 "Profiler 集成" 节。 | - -Rank 0 的副作用： - -- 在 `/params/env.json` 与 `/params/agent.json` 中递归 JSON dump - 两份 cfg。 -- RSL-RL 按 `agent_cfg.save_interval` 写 `/model_.pt`。 -- TensorBoard event 直接落到 `/` 下。 - -打开 profiler 时，`train_task` 会 hook `RslRlVecEnvWrapper.step`，使 profiler -schedule 每 env step 推进一次。若想以 PPO iteration 表达 schedule，把 -`wait` / `warmup` / `active` 乘以 `agent_cfg.num_steps_per_env`。 - -## play_task - -`play_task` 构造 env（设置 `task_cfg.play_env` 时优先），按 `agent` 选 policy，循环 -直到 viewer 关闭。 - -| 参数 | 默认 | 行为 | -|---|---|---| -| `task_id: str` | — | 必填。 | -| `checkpoint: Path \| None` | `None` | `agent="trained"` 时必填。 | -| `num_envs: int \| None` | `None` | 覆盖 `env_cfg.simulation.num_envs`。 | -| `agent: "zero" \| "random" \| "trained" \| None` | `None` | 给了 `checkpoint` 默认 `"trained"`，否则 `"zero"`。 | -| `agent_cfg: RslRlOnPolicyRunnerCfg \| None` | `None` | 只有 `agent="trained"` 才需要；缺省 fallback 到 `task.cfg.agent`。 | -| `deterministic: bool` | `True` | 传给 RSL-RL 推理辅助函数。 | -| `max_steps: int \| None` | `None` | rollout 上限。`None` 表示跑到 viewer 关闭。 | -| `prof*` | `None` | 语义同 `train_task`。 | - -Play 循环每步检测 `env.viewer_closed`，关 Genesis viewer 时干净退出。`env.close()` -放在 `finally` 块里。 - -## RslRlOnPolicyRunnerCfg - -扩展通过 `TaskCfg.agent` 注册的顶层 dataclass。`train` 依赖 -`class_name="OnPolicyRunner"`；扩展 cfg 用了其他 runner 类会在 dispatch 时被拒。 +## 训练流程 +```text +TASKS.get(task_id) +└── TaskCfg.env + TaskCfg.agent + └── ManagerBasedRlEnv + └── RslRlVecEnvWrapper + └── rsl_rl OnPolicyRunner.learn() ``` -RslRlOnPolicyRunnerCfg -├──（继承自 RslRlBaseRunnerCfg） -│ seed=42、num_steps_per_env=24、max_iterations=300 -│ obs_groups={"actor": ("policy",), "critic": ("critic",)} -│ save_interval=50、experiment_name="exp1"、run_name="" -│ logger="tensorboard" | "wandb" -│ wandb_project="genelab"、wandb_tags=() -│ resume=False、load_run=".*"、load_checkpoint="model_.*.pt" -│ clip_actions: float | None、upload_model=False -├── class_name="OnPolicyRunner" -├── actor: RslRlModelCfg -├── critic: RslRlModelCfg -└── algorithm: RslRlPpoAlgorithmCfg -``` - -`RslRlModelCfg` 旋钮（actor / critic 各一份）：`hidden_dims=(128, 128, 128)`、 -`activation="elu"`、`obs_normalization=False`，可选 `cnn_cfg` / `distribution_cfg` / -`rnn_type` / `rnn_hidden_dim` / `rnn_num_layers`、`class_name="MLPModel"`。 -只有 `class_name`、`hidden_dims`、`activation`、`obs_normalization`、 -`distribution_cfg` 会进入 MLP 构造函数；其余字段被 `_prune_model_cfg` 过滤掉。 - -`RslRlPpoAlgorithmCfg` 旋钮：`num_learning_epochs=5`、`num_mini_batches=4`、 -`learning_rate=1e-3`、`schedule="adaptive"|"fixed"`、`gamma=0.99`、`lam=0.95`、 -`entropy_coef=0.005`、`desired_kl=0.01`、`max_grad_norm=1.0`、 -`value_loss_coef=1.0`、`use_clipped_value_loss=True`、`clip_param=0.2`、 -`optimizer="adam"|"adamw"|"sgd"|"rmsprop"`、`class_name="PPO"`。 - -## AgentKind -`AgentKind = Literal["zero", "random", "trained"]`。`play_task` 按 `kind` 挑 policy -可调用： +main process 写入 `params/env.json`、`params/agent.json`、TensorBoard event、profiler trace 和 checkpoint。 -| Kind | Policy | 适用场景 | -|---|---|---| -| `"zero"` | 返回 `torch.zeros((num_envs, num_actions))`。 | 训练前的视觉 smoke；被动姿态 viewer rollout。 | -| `"random"` | 返回 `2.0 * rand(...) - 1.0`（uniform `[-1, 1]`）。 | sanity-check action 边界与 reward shaping。 | -| `"trained"` | 加载 checkpoint，实例化 `OnPolicyRunner`，调 `runner.get_inference_policy()`。 | 看训好的 policy。需要 `checkpoint`。 | +## 回放流程 -## VecEnv wrapper +`play_task` 在存在 `TaskCfg.play_env` 时优先使用它。策略来源： -`RslRlVecEnvWrapper` 把 `ManagerBasedRlEnv` 适配到 RSL-RL 的 `OnPolicyRunner` -期望的接口： - -| 属性 / 方法 | 来源 | +| Agent | 来源 | |---|---| -| `num_envs`、`device`、`max_episode_length` | `env.num_envs`、`env.device`、`env.max_episode_length`。 | -| `num_actions` | `env.action_manager.total_action_dim`。 | -| `num_obs`、`num_privileged_obs` | `"policy"` / `"critic"` 观测组的末轴维数，构造时先跑一次 `observation_manager.compute()` 取出。 | -| `reset()` | `env.reset()`。装了 `tensordict` 时把 obs 包成 `TensorDict`；否则原样返回 plain `dict`。 | -| `step(actions)` | 设了 `clip_actions` 就先 clip，调 `env.step`，打包 `(obs, reward, dones, extras)`，其中 `dones = terminated \| truncated`（long）、`extras["time_outs"] = truncated`。 | -| `episode_length_buf`、`cfg`、`unwrapped` | 透传到底层 env。 | +| `zero` | 返回零动作。 | +| `random` | 均匀随机动作。 | +| `trained` | 加载 checkpoint，并调用 RSL-RL inference policy。 | -Wrapper 在 import 时通过 `_attach_rsl_rl_base()` 动态成为 `rsl_rl.env.VecEnv` -的子类。没装 RSL-RL（例如 CPU-only 主机上的单元测试）时回退到拥有同样属性的 plain -对象。 +Genesis viewer 关闭或达到 `max_steps` 时，回放会干净退出。 ## 分布式训练 -CLI 的 `train` 上有 `--gpus N` 旗标。`cli/__init__.py:457` 的 -`_relaunch_under_torchrun` 在父进程里改写它： - -1. 在父进程预解析 log 目录为 `//`，让每个 rank - 拿到同一路径。 -2. 从 `sys.argv` 里剥掉 `--gpus N` 避免重启循环。 -3. 原始 argv 里没带 `--log-dir` 时追加 `--log-dir <预解析路径>`。 -4. `execvp`：`python -m torch.distributed.run --standalone --nproc_per_node=N - -m genelab.cli `。 - -```bash -uv run genelab train Genelab-Velocity-Flat-Unitree-G1-v0 --gpus 4 --num-envs 4096 -``` - -每个 rank 内部，`genelab.rl.distributed` 从环境读 `WORLD_SIZE` / `LOCAL_RANK` / -`RANK`： - -| Helper | 返回 | -|---|---| -| `world_size()` | `int(os.environ["WORLD_SIZE"])` 或 `1`。 | -| `local_rank()` | `int(os.environ["LOCAL_RANK"])` 或 `0`。 | -| `global_rank()` | `int(os.environ["RANK"])` 或 `0`。 | -| `is_distributed()` | `world_size() > 1`。 | -| `is_main_process()` | `global_rank() == 0`。 | -| `pin_cuda_device()` | 分布式 run 中返回 `"cuda:0"`（CLI bootstrap 已按 rank 钉 `CUDA_VISIBLE_DEVICES`，每个 rank 只看见一张卡）；单 GPU 模式返回 `None`。 | - -只有 main process 写 checkpoint、TensorBoard event、profiler trace 与 -`params/*.json` 转储。RSL-RL 内部跨 rank all-reduce 梯度。 - -## Profiler 集成 - -`maybe_profile` 用一个 `schedule` 与 TensorBoard trace handler 包住 -`torch.profiler.profile`。CLI 在 `play` 与 `train` 上各暴露 8 个 flag： - -| CLI flag | `maybe_profile` kwarg | Env var fallback | 默认 | -|---|---|---|---| -| `--prof` | `enabled` | `GENELAB_PROFILE=1` | off | -| `--prof-out` | `out_dir` | `GENELAB_PROFILE_OUT` | `logs/torch_profile` | -| `--prof-wait` | `wait` | `GENELAB_PROFILE_WAIT` | `10` | -| `--prof-warmup` | `warmup` | `GENELAB_PROFILE_WARMUP` | `5` | -| `--prof-active` | `active` | `GENELAB_PROFILE_ACTIVE` | `10` | -| `--prof-repeat` | `repeat` | `GENELAB_PROFILE_REPEAT` | `2` | -| `--prof-record-shapes` | `record_shapes` | `GENELAB_PROFILE_RECORD_SHAPES=1` | off | -| `--prof-with-stack` | `with_stack` | `GENELAB_PROFILE_WITH_STACK=1` | off | - -Schedule 每 `wait + warmup + active` 步出一份 trace，重复 `repeat` 次。Handler 在 -`out_dir` 下写 `*.pt.trace.json` + `*.tar.gz`；`uv run genelab prof open ` -对其起 TensorBoard。 - -```bash -uv run genelab train Genelab-Velocity-Flat-Unitree-G1-v0 \ - --prof --prof-active 3 --prof-repeat 1 --max-iterations 10 -uv run genelab prof open logs/torch_profile -``` - -## 已知失效模式 - -!!! warning "agent cfg 类型不匹配" - `_dispatch_train` 拒绝任何 `task.cfg.agent` 类型不是 `RslRlOnPolicyRunnerCfg` - 的 task。"字段刚好一样"的自造 runner cfg 不被接受。 - -!!! warning "agent='trained' 需要 --checkpoint" - 缺 checkpoint 时 `play_task` 抛 `SystemExit("agent='trained' requires a - --checkpoint path")`。要么传 `--checkpoint`，要么挑 `--agent zero` / - `--agent random` 做视觉 smoke。 - -!!! warning "torchrun 重启回环" - `_relaunch_under_torchrun` 仅在 `--gpus N` 出现 *且* `TORCHELASTIC_RUN_ID` - 缺失时重新 exec。手工跑 `torchrun ... -m genelab.cli train --gpus 4`（在 - torchrun 内部 argv 里仍保留 `--gpus`）会跳过重启 —— 因为 env var 已存在；这是 - 预期行为。无限循环只在 `--gpus` 被转发但 *没有* torchrun 环境变量时出现， - 通常是 `genelab` 被其他 launcher 套进去导致的。 - -!!! tip "先 profile 一段短 run" - Profiler trace 增长很快。从 `--prof --prof-active 3 --prof-repeat 1 - --max-iterations 10` 起步，先看 trace 体量再跑完整 PPO schedule。 +`genelab train TASK --gpus N` 会把当前命令重新拉起到 `torchrun` 下。父进程预先计算共享日志目录，让每个 rank 写入同一个 run。`--num_envs` 是所有 rank 总数；`--num_envs_per_gpu` 是每 rank 数量。 -## See also +## 继续阅读 -- [Play and Train](../cli/play-train.md) -- [Profiling](../cli/profiling.md) -- [Managers and MDP terms](managers.md) +- [运行 RL 实验](../best-practices/rl-experiments.md) +- [play 与 train](../cli/play-train.md) +- [API 参考](../api/reference.md) diff --git a/docs/concepts/scene.en.md b/docs/concepts/scene.en.md index 7ee4dd37..a3a82e99 100644 --- a/docs/concepts/scene.en.md +++ b/docs/concepts/scene.en.md @@ -1,160 +1,36 @@ -# Scene and entities +# Scene and Entities -`genelab.scene.InteractiveScene` is the composition root that owns a Genesis -`gs.Scene` together with the articulated robot, rigid props, optional terrain, and -the mouse-interaction plugin. `ManagerBasedRlEnv` defers all sim-level orchestration -to the scene; the env layer only manages the seven MDP terms. +`InteractiveScene` is GeneLab's Genesis scene owner. It translates declarative configs into live +Genesis handles and exposes Isaac Lab-shaped entity wrappers to the rest of the environment. -## Composing a Genesis scene +## Scene boundary -A scene is built from two cfg objects passed to the env. `SimulationCfg` carries the -Genesis runtime knobs (timestep, parallel-env count, viewer toggle); `InteractiveSceneCfg` -carries the composition (entities, terrain, sensors, mouse plugin, BatchRenderer -toggle). The scene parses the cfg, allocates wrapper objects, then defers the actual -Genesis allocation to the `build()` call. +`InteractiveSceneCfg` describes what should exist: env spacing, entities, terrain, sensors, +recordings, viewer interaction, and batch rendering. `InteractiveScene` owns what actually exists: +the Genesis `Scene`, articulations, rigid objects, sensors, terrain importer, recorder bridge, and +viewer state. -```python -from genelab.asset_zoo import CartpoleCfg -from genelab.configs import InteractiveSceneCfg, SimulationCfg -from genelab.envs.manager_based_rl_env import ManagerBasedRlEnvCfg +This separation keeps configs serializable and lets tasks be inspected before Genesis starts. -env_cfg = ManagerBasedRlEnvCfg( - simulation=SimulationCfg(num_envs=4096, dt=0.005, substeps=1, vis=False), - scene=InteractiveSceneCfg(env_spacing=(2.5, 2.5)), - robot=CartpoleCfg(), -) -``` +## Entities -## Lifecycle +| Entity | Purpose | +|---|---| +| `Articulation` | Robot wrapper with joint/link names, default joint state, limits, and refreshed `RobotState`. | +| `RigidObject` | Non-articulated object wrapper. | +| `RobotState` | Batched tensors read by observations, rewards, sensors, and events. | -`InteractiveScene` follows a strict three-step lifecycle, enforced at runtime: +`ManagerBasedRlEnv` adds the configured robot as the `"robot"` articulation and exposes convenient +properties such as `env.robot_state`, `env.joint_names`, `env.link_names`, and `env.scene`. -1. **Construct**: `InteractiveScene(sim_cfg, scene_cfg, device_hint=...)` parses the - cfgs and allocates wrapper objects for every entity declared in - `scene_cfg.entities`. No Genesis import yet. -2. **Add entities**: `add_entity(name, cfg)` registers extra entities — the env uses - this to inject `ManagerBasedRlEnvCfg.robot` under the name `"robot"`. Calling - `add_entity` after `build()` raises `RuntimeError`. -3. **Build**: `build()` imports Genesis, initialises the backend, creates the - `gs.Scene`, spawns ground / terrain / entities / cameras, attaches the mouse - plugin when configured, then calls `gs_scene.build(n_envs, env_spacing)`. +## Why wrappers matter -`ManagerBasedRlEnv.__init__` runs all three steps in order and post-build binds every -articulation so the per-joint and per-link tensors land. +Genesis APIs and Isaac Lab-style task code use different vocabulary. Wrappers isolate that +difference: MDP terms read stable GeneLab properties while the backend integration handles Genesis +details. -## SimulationCfg +## Where to continue -| Field | Default | Meaning | -|---|---|---| -| `vis` | `False` | Open the Genesis viewer. Forces `num_envs=1` in practice; the viewer can only render env 0. | -| `gpu` | `False` | Use the CUDA backend (`gs.gpu`). Required for `BatchRenderer`. | -| `steps` | `240` | Default number of steps the CLI `play` runner takes when no override is given. | -| `dt` | `0.01` | Physics timestep in seconds. | -| `substeps` | `4` | Genesis substeps per physics step. | -| `num_envs` | `1` | Parallel-env count. Most managers tensorise across this dimension. | - -## InteractiveSceneCfg - -| Field | Default | Meaning | -|---|---|---| -| `env_spacing` | `(2.0, 2.0)` | Per-env XY offset Genesis applies when `num_envs > 1`. | -| `sensors` | `()` | Tuple of `SensorCfg` instances built and bound after the scene is constructed. | -| `mouse_interaction` | `False` | Attach the GeneLab mouse-drag plugin to the viewer (only effective with `vis=True`). | -| `entities` | `{}` | Additional `ArticulationCfg` / `RigidObjectCfg` keyed by name. The env injects the robot as `"robot"`. | -| `terrain` | `None` | A `TerrainGeneratorCfg`. Default `None` adds a flat `gs.morphs.Plane`. | -| `batch_render` | `False` | Pass `gs.renderers.BatchRenderer(use_rasterizer=False)` to `gs.Scene`. Required for `CameraSensor` to emit per-env RGB-D tensors. Linux x86-64 + CUDA only. | - -## Articulation - -`Articulation` wraps an articulated robot. Construct from `ArticulationCfg` and add -to the scene via `scene.add_entity("robot", cfg)` (or set -`ManagerBasedRlEnvCfg.robot` and let the env wire it). - -| Field | Type | Meaning | -|---|---|---| -| `mjcf_path` | `str` | Absolute path to a MuJoCo XML file. | -| `init_pos` | `tuple[float, float, float]` | World-frame spawn offset in metres. | -| `init_quat` | `tuple[float, float, float, float]` | World-frame spawn orientation (wxyz). | -| `default_joint_pos` | `dict[str, float]` | Regex-keyed default joint position; last match wins. | -| `actuators` | `dict[str, ActuatorBaseCfg]` | Joint groups. Every actuated joint must be covered by exactly one group. | -| `foot_link_names` | `tuple[str, ...]` | Optional metadata for downstream MDP terms (e.g. contact sensors). | - -The pre-M3 `joint_kp`, `joint_kv`, and dict-shaped `action_scale` knobs are -removed — configure equivalent groups through `actuators`. Passive joints take an -explicit zero-gain `ImplicitPDActuatorCfg` so the topology stays visible. - -```python -from genelab.actuator import ImplicitPDActuatorCfg -from genelab.entity import ArticulationCfg - -cartpole_cfg = ArticulationCfg( - mjcf_path="/path/to/cartpole.xml", - init_pos=(0.0, 0.0, 1.5), - default_joint_pos={"cart_slide": 0.0, "pole_hinge": 0.0}, - actuators={ - "cart": ImplicitPDActuatorCfg( - target_names_expr=("cart_slide",), stiffness=80.0, damping=8.0, - ), - "pole": ImplicitPDActuatorCfg( - target_names_expr=("pole_hinge",), stiffness=0.0, damping=0.0, - ), - }, -) -``` - -## RigidObject - -`RigidObject` represents a non-articulated body. No actuators, no per-step state — it -is geometry that participates in collisions and contacts but is never driven. Spawn -via `scene.add_entity("name", RigidObjectCfg(...))` before `build()`. - -| Field | Type | Meaning | -|---|---|---| -| `morph` | `Literal["plane", "box", "sphere", "mesh", "mjcf"]` | Genesis morph selector. | -| `file` | `str \| None` | Path to mesh / MJCF; required for `"mesh"` / `"mjcf"`. | -| `size` | `tuple[float, ...]` | `(x, y, z)` for box, `(radius,)` for sphere. | -| `init_pos`, `init_quat` | tuple | World-frame spawn pose (wxyz quat). | -| `fixed` | `bool` | When `True`, the body is welded to the world frame. | - -Typical uses are static obstacles, target markers, and passive payloads. For -articulated joints — even one — use `Articulation` instead. - -## Failure modes worth knowing - -!!! warning "`add_entity` after `build`" - `InteractiveScene.add_entity` raises `RuntimeError` once `build()` has run. - Construct the env, then add every extra entity, then let the env call `build()`. - -!!! warning "Empty `actuators` dict" - `Articulation.bind` rejects an empty `actuators` dict. Every actuated joint must - be covered by exactly one group; unmatched or doubly-matched joints raise - `ValueError` with the conflicting joint names listed. - -!!! warning "Genesis backend mismatch" - `SimulationCfg.gpu=True` requires a working CUDA install. On macOS or CPU-only - Linux, leave `gpu=False`; the CPU backend handles single-env play loops fine. - `batch_render=True` further requires Linux x86-64 + CUDA + Madrona — there is no - CPU fallback. - -!!! note "Viewer and parallel envs" - `SimulationCfg.vis=True` is only meaningful with `num_envs=1`. The viewer - renders env 0 regardless, so leave `num_envs` high for training and switch to a - dedicated `play_env` with `num_envs=1` for visual inspection. - -!!! tip "Viewer closed mid-rollout" - When the user closes the Genesis viewer, the kernel catches - `GenesisException("Viewer closed.")` inside `InteractiveScene.step` and flips - `env.viewer_closed = True`. Subsequent `env.step` calls become no-ops. Loop - drivers should poll the flag instead of writing their own try / except: - ```python - for step in range(max_steps): - obs, *_ = env.step(action) - if env.viewer_closed: - break - ``` - -## See also - -- [Configs](configs.md) +- [Module Map](../reference/module-map.md) +- [Sensors](sensors.md) - [Actuators](actuators.md) -- [Asset zoo](asset_zoo.md) diff --git a/docs/concepts/scene.zh.md b/docs/concepts/scene.zh.md index e3e8afd5..fe1c53a3 100644 --- a/docs/concepts/scene.zh.md +++ b/docs/concepts/scene.zh.md @@ -1,152 +1,33 @@ # 场景与实体 -`genelab.scene.InteractiveScene` 是组合根 (composition root)，持有一个 Genesis `gs.Scene` -以及关节机器人、刚体道具、可选地形、鼠标交互插件。`ManagerBasedRlEnv` 把所有 sim 层 -编排都交给 scene；env 层只管 7 个 MDP term。 +`InteractiveScene` 是 GeneLab 的 Genesis scene owner。它把声明式配置转换成 live Genesis handle， +并向环境其他部分暴露 Isaac Lab 风格的 entity wrapper。 -## 组合 Genesis 场景 +## Scene 边界 -scene 从两个传给 env 的 cfg 对象构造。`SimulationCfg` 携带 Genesis 运行时旋钮 (时间步、 -并行 env 数、viewer 开关)；`InteractiveSceneCfg` 携带组合内容 (entities、terrain、sensors、 -mouse 插件、BatchRenderer 开关)。scene 解析 cfg、分配 wrapper 对象，真正的 Genesis 分配 -延后到 `build()` 调用。 +`InteractiveSceneCfg` 描述应该存在什么：env spacing、entities、terrain、sensors、recordings、 +viewer interaction、batch rendering。`InteractiveScene` 持有实际存在的东西：Genesis `Scene`、 +articulation、rigid object、sensor、terrain importer、recorder bridge 和 viewer 状态。 -```python -from genelab.asset_zoo import CartpoleCfg -from genelab.configs import InteractiveSceneCfg, SimulationCfg -from genelab.envs.manager_based_rl_env import ManagerBasedRlEnvCfg +这种分离让配置可序列化，也让任务能在 Genesis 启动前被检查。 -env_cfg = ManagerBasedRlEnvCfg( - simulation=SimulationCfg(num_envs=4096, dt=0.005, substeps=1, vis=False), - scene=InteractiveSceneCfg(env_spacing=(2.5, 2.5)), - robot=CartpoleCfg(), -) -``` +## Entities -## 生命周期 +| Entity | 用途 | +|---|---| +| `Articulation` | 机器人 wrapper，包含 joint/link 名、默认 joint state、limits、刷新后的 `RobotState`。 | +| `RigidObject` | 非关节物体 wrapper。 | +| `RobotState` | observation、reward、sensor、event 读取的 batched tensor。 | -`InteractiveScene` 遵循严格的三步生命周期，运行时强校验： +`ManagerBasedRlEnv` 把配置中的机器人作为 `"robot"` articulation 加入，并暴露 +`env.robot_state`、`env.joint_names`、`env.link_names`、`env.scene` 等便捷属性。 -1. **构造**：`InteractiveScene(sim_cfg, scene_cfg, device_hint=...)` 解析 cfg 并为 - `scene_cfg.entities` 里每个实体分配 wrapper。此刻还没 import Genesis。 -2. **追加实体**：`add_entity(name, cfg)` 注册额外实体——env 用它把 - `ManagerBasedRlEnvCfg.robot` 以 `"robot"` 名注入。`build()` 之后再调 - `add_entity` 会抛 `RuntimeError`。 -3. **构建**：`build()` 导入 Genesis、初始化 backend、创建 `gs.Scene`、生成地面 / 地形 / - 实体 / 相机，按需挂上鼠标插件，最后 `gs_scene.build(n_envs, env_spacing)`。 +## 为什么需要 wrapper -`ManagerBasedRlEnv.__init__` 顺序跑完三步，build 之后再 bind 每个 articulation，让 -per-joint / per-link tensor 落位。 +Genesis API 和 Isaac Lab 风格任务代码使用的词汇不同。wrapper 隔离这种差异：MDP term 读取稳定的 GeneLab 属性，后端集成负责 Genesis 细节。 -## SimulationCfg +## 继续阅读 -| 字段 | 默认 | 含义 | -|---|---|---| -| `vis` | `False` | 打开 Genesis viewer。实际只配合 `num_envs=1`；viewer 只能渲染 env 0。 | -| `gpu` | `False` | 使用 CUDA backend (`gs.gpu`)。`BatchRenderer` 需要它。 | -| `steps` | `240` | CLI `play` 在没指定 override 时默认跑的步数。 | -| `dt` | `0.01` | 物理时间步 (秒)。 | -| `substeps` | `4` | 每个物理步内的 Genesis 子步数。 | -| `num_envs` | `1` | 并行 env 数。大多数 manager 在这个维度上向量化。 | - -## InteractiveSceneCfg - -| 字段 | 默认 | 含义 | -|---|---|---| -| `env_spacing` | `(2.0, 2.0)` | `num_envs > 1` 时 Genesis 在 XY 上为每个 env 留的间距。 | -| `sensors` | `()` | scene 构造完后批量 build + bind 的 `SensorCfg` 元组。 | -| `mouse_interaction` | `False` | 给 viewer 挂上 GeneLab 的鼠标拖拽插件 (仅在 `vis=True` 时有效)。 | -| `entities` | `{}` | 按名字索引的额外 `ArticulationCfg` / `RigidObjectCfg`。env 会以 `"robot"` 名注入主机器人。 | -| `terrain` | `None` | `TerrainGeneratorCfg`；默认 `None` 时铺一张 `gs.morphs.Plane` 平面。 | -| `batch_render` | `False` | 把 `gs.renderers.BatchRenderer(use_rasterizer=False)` 传给 `gs.Scene`。`CameraSensor` 输出 per-env RGB-D 张量必需。仅支持 Linux x86-64 + CUDA。 | - -## Articulation - -`Articulation` 包装一个关节机器人。从 `ArticulationCfg` 构造，再通过 -`scene.add_entity("robot", cfg)` 加入 scene (或者直接设 `ManagerBasedRlEnvCfg.robot`，让 -env 自动接管)。 - -| 字段 | 类型 | 含义 | -|---|---|---| -| `mjcf_path` | `str` | MuJoCo XML 文件的绝对路径。 | -| `init_pos` | `tuple[float, float, float]` | 世界系初始位移 (米)。 | -| `init_quat` | `tuple[float, float, float, float]` | 世界系初始姿态 (wxyz 四元数)。 | -| `default_joint_pos` | `dict[str, float]` | 正则键控的默认关节位置；按 last-match-wins。 | -| `actuators` | `dict[str, ActuatorBaseCfg]` | 关节分组。每个被驱动关节必须恰被一个分组覆盖。 | -| `foot_link_names` | `tuple[str, ...]` | 给下游 MDP term (例如 contact sensor) 的可选元数据。 | - -M3 之前的 `joint_kp` / `joint_kv` / dict 形 `action_scale` 旋钮已经移除——同等行为请 -通过 `actuators` 配。被动关节用一个零增益的 `ImplicitPDActuatorCfg` 显式覆盖，保持 -拓扑可见。 - -```python -from genelab.actuator import ImplicitPDActuatorCfg -from genelab.entity import ArticulationCfg - -cartpole_cfg = ArticulationCfg( - mjcf_path="/path/to/cartpole.xml", - init_pos=(0.0, 0.0, 1.5), - default_joint_pos={"cart_slide": 0.0, "pole_hinge": 0.0}, - actuators={ - "cart": ImplicitPDActuatorCfg( - target_names_expr=("cart_slide",), stiffness=80.0, damping=8.0, - ), - "pole": ImplicitPDActuatorCfg( - target_names_expr=("pole_hinge",), stiffness=0.0, damping=0.0, - ), - }, -) -``` - -## RigidObject - -`RigidObject` 表示一个非关节刚体。无 actuator、无 per-step 状态——它就是几何，参与碰撞 -与接触，但永远不被驱动。`build()` 之前通过 `scene.add_entity("name", RigidObjectCfg(...))` -加入。 - -| 字段 | 类型 | 含义 | -|---|---|---| -| `morph` | `Literal["plane", "box", "sphere", "mesh", "mjcf"]` | Genesis morph 选择。 | -| `file` | `str \| None` | mesh / MJCF 路径；`"mesh"` / `"mjcf"` 必填。 | -| `size` | `tuple[float, ...]` | box 是 `(x, y, z)`；sphere 是 `(radius,)`。 | -| `init_pos`、`init_quat` | tuple | 世界系初始位姿 (wxyz quat)。 | -| `fixed` | `bool` | `True` 时把刚体焊死在世界系。 | - -典型用法是静态障碍、目标 marker、被动负载。只要有一个关节就用 `Articulation`，不要用 -`RigidObject`。 - -## 已知失效模式 - -!!! warning "`build` 之后追加实体" - `build()` 之后再调 `InteractiveScene.add_entity` 会抛 `RuntimeError`。先构造 env，把 - 所有额外实体加完，再让 env 调用 `build()`。 - -!!! warning "空 `actuators` 字典" - `Articulation.bind` 拒绝空 `actuators`。每个被驱动关节必须被恰一个分组覆盖；未匹配 - 或被双重匹配的关节会带着冲突关节名抛 `ValueError`。 - -!!! warning "Genesis backend 不匹配" - `SimulationCfg.gpu=True` 需要可用的 CUDA 安装。macOS / 纯 CPU Linux 上保持 - `gpu=False`；CPU backend 跑单 env play 循环没问题。`batch_render=True` 额外要求 - Linux x86-64 + CUDA + Madrona——没有 CPU 回退。 - -!!! note "Viewer 与并行 env" - `SimulationCfg.vis=True` 仅在 `num_envs=1` 时有意义。viewer 只渲染 env 0，所以训练 - 时把 `num_envs` 设高，再单独配一份 `num_envs=1` 的 `play_env` 用于人眼检查。 - -!!! tip "Rollout 中途关闭 viewer" - 用户关掉 Genesis viewer 后，内核会在 `InteractiveScene.step` 里捕获 - `GenesisException("Viewer closed.")` 并把 `env.viewer_closed` 置 `True`，后续 - `env.step` 自动 no-op。上层循环只需轮询这个 flag，不必自己写 try / except： - ```python - for step in range(max_steps): - obs, *_ = env.step(action) - if env.viewer_closed: - break - ``` - -## See also - -- [Configs](configs.md) -- [Actuators](actuators.md) -- [Asset zoo](asset_zoo.md) +- [模块地图](../reference/module-map.md) +- [传感器](sensors.md) +- [执行器](actuators.md) diff --git a/docs/concepts/sensors.en.md b/docs/concepts/sensors.en.md index 38bc37bf..8e7ba0b6 100644 --- a/docs/concepts/sensors.en.md +++ b/docs/concepts/sensors.en.md @@ -1,284 +1,45 @@ # Sensors -Genesis does not parse the MJCF `` block, so GeneLab introduces a backend-agnostic -sensor abstraction. The interface mirrors mjlab's `SensorCfg` / `Sensor[T]`, but every concrete -sensor reads from the env's `RobotState` instead of MuJoCo sensordata — observation and reward -terms transfer between backends without ceremony. +Genesis does not expose every signal through an MJCF sensor block, so GeneLab provides a +backend-aware sensor abstraction. Sensors produce cached data that observations, rewards, +recordings, and custom code can consume. ## Lifecycle -`bind(env)` runs once at construction. Each control step, `update(dt)` invalidates the cache; -the first access to `data` triggers `_compute_data` lazily. `reset(env_ids)` invalidates the -cache and lets stateful sensors clear per-env buffers. +Sensors are declared in `InteractiveSceneCfg.sensors`. `ManagerBasedRlEnv` builds and binds them +after the scene exists. Each control step invalidates cached data; the first access to `sensor.data` +computes fresh output. Reset clears per-env sensor state. -```python -class Sensor[T](ABC): - def bind(self, env: "ManagerBasedRlEnv") -> None: ... - @property - def data(self) -> T: ... - def update(self, dt: float) -> None: ... - def reset(self, env_ids: torch.Tensor | None = None) -> None: ... - @abstractmethod - def _compute_data(self) -> T: ... -``` +This lazy cache keeps repeated observation and reward reads consistent within one control step. -The env wires the lifecycle automatically: sensors built from `InteractiveSceneCfg.sensors` -get `update` called after each articulation refresh and `reset` called from inside -`_reset_idx`, so reward and observation terms always see fresh sensor data. +## Built-in sensor families -## Registering on a scene +| Sensor | Use case | +|---|---| +| `BodyVelocitySensor` | Link linear or angular velocity at an offset. | +| `IMUSensor` | Orientation, projected gravity, linear/angular acceleration. | +| `CameraSensor` | RGB-D camera attached to a link. | +| `ContactSensor` | Per-link contact force and optional air/contact time. | +| `FrameTransformerSensor` | Target frame poses relative to a source frame. | +| `RayCastSensor` | Grid, ring, or hemisphere ray patterns. | +| `TerrainHeightSensor` | Downward height scan for terrain-aware locomotion. | +| `SelfContactSensor` | Self-collision/contact signals. | +| `RootAngularMomentumSensor` | Root angular momentum diagnostics. | -`InteractiveSceneCfg.sensors` is a tuple of `SensorCfg`. `ManagerBasedRlEnv.__init__` calls -`build()` on each cfg and binds the resulting sensor to the env. Access at runtime is via -`env.sensors[name].data`. +## Sensor data in MDP terms -```python -from genelab.configs import InteractiveSceneCfg, SimulationCfg -from genelab.sensor import BodyVelocitySensorCfg, ContactSensorCfg +Observation terms can read sensors through `mdp.sensor_data` or a custom callable. Rewards and +metrics can access `env.sensors[name].data` directly. -simulation = SimulationCfg(num_envs=4096) -scene = InteractiveSceneCfg( - sensors=( - BodyVelocitySensorCfg( - name="imu_lin_vel", - link_name="pelvis", - offset=(0.04525, 0.0, -0.08339), - measure="lin_vel", - ), - ContactSensorCfg( - name="feet_ground_contact", - link_names=("left_ankle_roll_link", "right_ankle_roll_link"), - track_air_time=True, - ), - ), -) -``` +Keep sensor names stable; they become config paths, recording sources, and term params. -## Built-in sensors +## Camera caveat -### BodyVelocitySensor +Parallel RGB-D rendering requires CUDA and `InteractiveSceneCfg.batch_render=True`. Treat camera +tasks as a separate hardware profile from light state-based RL tasks. -Mirrors a MuJoCo `` / `` on a site rigidly attached to a robot link. Returns -the linear or angular velocity of the site in the link's body frame, with a configurable -lever-arm offset and optional per-env uniform bias. +## Where to continue -| Field | Type | Meaning | -|-------|------|---------| -| `link_name` | `str` | Link the site is rigidly attached to. | -| `offset` | `tuple[float, float, float]` | Site position in the link's local frame. Ignored for `ang_vel`. | -| `measure` | `Literal["lin_vel", "ang_vel"]` | Velocimeter (linear) or gyro (angular). | -| `bias_range` | `tuple[float, float] \| None` | Uniform per-env bias resampled on `reset`. | - -The velocimeter math is `v_site = v_link + ω × (R_link · offset)` rotated into the body frame — -matching MuJoCo's lever-arm convention so a GeneLab-trained policy reads the same signal as the -mjlab reference. - -### IMUSensor - -Inertial Measurement Unit at a site rigidly attached to a link. Outputs orientation -(`link_quat_w`), body-frame projected unit gravity, and body-frame linear / angular -acceleration. Accelerations are computed by finite difference of the world-frame velocity -buffers; the first control step after every `reset` returns zero acceleration to avoid a -spurious spike from a stale `prev` value — document this in any reward / observation term -that consumes `lin_acc_b` or `ang_acc_b` near reset boundaries. - -| Field | Type | Meaning | -|-------|------|---------| -| `link_name` | `str` | Link the IMU is rigidly attached to. | -| `offset` | `tuple[float, float, float]` | Site position in the link's local frame; contributes the lever-arm term to linear acceleration. | -| `gravity_bias` | `bool` | When `True`, output is the specific force `R^T (a_w - g_w)` — matches a real accelerometer at rest reading `+g` along its up axis. | -| `bias_range_lin_acc` | `tuple[float, float] \| None` | Per-env constant accelerometer bias resampled on `reset`. | -| `bias_range_ang_acc` | `tuple[float, float] \| None` | Per-env constant gyro-derivative bias resampled on `reset`. | - -`IMUSensor` does not output linear or angular velocity — that remains the role of -`BodyVelocitySensor`, which can coexist on the same link when both signals are needed. The -finite-difference at the control rate is **not** filtered internally; layer -`ObservationTermCfg.noise` + `scale` on top, or apply a custom EMA in the observation term. - -### CameraSensor - -Rigid-mount RGB-D camera bolted to a named link via a fixed 4×4 transform. -`pre_build_genesis()` runs before `gs_scene.build` so the camera is registered in -time for `BatchRenderer` to snapshot it; it allocates the Genesis camera handle -through `gs_scene.add_camera`, builds the offset matrix from `offset_pos` / -`offset_quat`, and calls `cam.attach`. `bind()` then validates the link name -against `env.link_names`. Each `data` access invokes `cam.move_to_attach()` and -`cam.render(...)` once per control step; the cached `CameraData` returns `rgb` -(uint8) and / or `depth` (float meters), shaped `(num_envs, H, W, 3)` / -`(num_envs, H, W)`. - -| Field | Type | Meaning | -|-------|------|---------| -| `link_name` | `str` | Link the camera is bolted to (resolved through `env.link_names`). | -| `offset_pos` | `tuple[float, float, float]` | Camera origin in the link's local frame. | -| `offset_quat` | `tuple[float, float, float, float]` | Camera orientation (wxyz) relative to the link; `+x` is forward in Genesis convention. | -| `width`, `height` | `int` | Pixel resolution; all BatchRender cameras on a scene must share resolution. | -| `fov` | `float` | Vertical field-of-view in degrees. | -| `near`, `far` | `float` | Depth clip planes in meters. | -| `render_rgb`, `render_depth` | `bool` | Independently toggle each channel; the disabled channel is `None` in the output dataclass. | - -!!! warning "BatchRenderer is the only parallel-env backend" - Multi-env RGB-D requires `gs.init(backend=gs.cuda)` and - `InteractiveSceneCfg.batch_render=True` — the scene then passes - `gs.renderers.BatchRenderer(use_rasterizer=False)` to `gs.Scene`. Linux x86-64 + - CUDA only. The module file can still be imported on macOS / CPU, but - `gs_scene.build` will raise once it inspects the camera list against an - incompatible renderer. - -### ContactSensor - -Per-link aggregate of `robot.get_links_net_contact_force()`. With `track_air_time=True`, an -internal state machine advances `current_air_time` / `current_contact_time` per env, and snaps -the completed durations into `last_air_time` / `last_contact_time` at the contact transition. - -| Field | Type | Meaning | -|-------|------|---------| -| `link_names` | `tuple[str, ...]` | Explicit list of link names to monitor. | -| `link_names_expr` | `str \| None` | Regex matched against `env.link_names`. | -| `force_threshold` | `float` | Force magnitude (N) above which `found` is true. | -| `track_air_time` | `bool` | Allocate the air-time / contact-time state machine. | - -`data` is a `ContactData` dataclass with `force`, `force_norm`, `found`, plus the four air-time -buffers. The matching obs terms — `mdp.foot_air_time`, `mdp.foot_contact`, -`mdp.foot_contact_forces` — read straight off this dataclass. - -### FrameTransformerSensor - -Stateless forward-kinematics probe: outputs the pose of one or more target frames relative -to a single source frame, with rigid offsets in each link's local frame. Both the world-frame -pose and the source-frame pose are exposed in the data class so reward / observation terms -can pick either basis without re-running the math themselves. Typical uses include reading -an end-effector's pose in the base link's frame, comparing two foot-tip frames during gait -shaping, or expressing a payload's pose relative to a grasp site. - -| Field | Type | Meaning | -|-------|------|---------| -| `source_link_name` | `str` | Reference frame the targets are expressed in. | -| `source_offset_pos` | `tuple[float, float, float]` | Source-side offset in the link's local frame. | -| `source_offset_quat` | `tuple[float, float, float, float]` | Source-side rotation offset (wxyz). | -| `target_frames` | `tuple[TargetFrameCfg, ...]` | Order-preserving target list — the output's `N` axis follows cfg order. | - -Each `TargetFrameCfg` carries `link_name`, optional `name` (defaults to `link_name`), -`offset_pos`, and `offset_quat`. Sensor attribute `target_names: tuple[str, ...]` is exposed -so consumers can resolve a column by name. `bind` rejects unknown link names and empty -`target_frames`. - -### Ray-cast patterns - -`RayCastSensorCfg.pattern` accepts any of the three bundled pattern dataclasses. Custom -patterns satisfy the same informal protocol — `num_rays() -> int` and -`generate(device) -> (starts, dirs)` with both tensors shaped `(M, 3)` in the sensor's local -frame — and slot in without changing `RayCastSensor` itself. - -`GridPattern` is the default; rays are parallel and arranged on a 2D rectangle. `RingPattern` -emits `num_horizontal × num_vertical` rays from the origin, evenly spaced in azimuth and -elevation — the typical multi-line LIDAR layout. `HemispherePattern` distributes -`num_rays_target` rays on a spherical cap of half-angle `polar_fov_deg` around `pole_axis` -using a Fibonacci lattice; 90° covers a full hemisphere, 180° a full sphere. - -| Pattern | Key fields | Use case | -|---------|------------|----------| -| `GridPattern` | `resolution`, `size`, `direction` | Height-scan grids, area sweeps | -| `RingPattern` | `num_horizontal`, `num_vertical`, `horizontal_fov_deg`, `vertical_fov_deg` | Planar / multi-line LIDAR | -| `HemispherePattern` | `num_rays_target`, `pole_axis`, `polar_fov_deg` | Proximity dome, downward coverage | - -`RingPattern` treats a horizontal span of exactly ±360° as a wrap-around and drops the -duplicate closing azimuth; any other span (e.g. `(-30, 30)` for a forward-facing scanner) is -inclusive on both endpoints. `HemispherePattern.num_rays()` returns `num_rays_target` -exactly — the Fibonacci lattice produces no rounding error. - -### TerrainHeightSensor - -2D grid of downward rays anchored to a robot link. Output is per-ray height above the terrain -(positive = above), useful as a privileged `height_scan` critic observation. The default -backend intersects every ray against a horizontal plane at `ground_height`; when the scene -attaches a `TerrainImporter`, the inner `RayCastSensor` bilinearly samples the height-field -instead. Subclassing `RayCastSensor` and overriding `_intersect_world_rays` is the extension -point for BVH or other custom backends. - -| Field | Type | Meaning | -|-------|------|---------| -| `link_name` | `str` | Anchor link for the grid origin. | -| `pattern` | `GridPattern \| RingPattern \| HemispherePattern` | Pattern geometry. | -| `attach_yaw_only` | `bool` | Rotate the pattern by yaw only so it stays horizon-aligned. | -| `max_distance` | `float` | Distance clamp for the ray cast. | -| `ground_height` | `float` | Plane height used by the default flat-plane backend. | - -## Observations with noise - -`ObservationTermCfg.noise` accepts an additive noise model (`Unoise(n_min, n_max)` or -`Gnoise(mean, std)`). `ObservationGroupCfg.enable_corruption` gates whether the noise is -applied — disabled by default to keep the critic on ground truth. The per-term pipeline order -is **noise → scale → clip**, so noise magnitudes live in raw signal space: `Unoise(-1.5, 1.5)` -on a raw `joint_vel` term with `scale=0.05` ends up as ±0.075 final jitter. - -The canonical pattern shares terms between policy and critic and differs only on -`enable_corruption`: - -```python -from genelab import mdp -from genelab.managers import ObservationGroupCfg, ObservationTermCfg -from genelab.mdp.noise import Unoise - - -def _obs_terms() -> dict[str, ObservationTermCfg]: - return { - "base_lin_vel": ObservationTermCfg( - func=mdp.sensor_data, - params={"sensor_name": "imu_lin_vel"}, - noise=Unoise(-0.5, 0.5), - ), - "joint_vel": ObservationTermCfg( - func=mdp.joint_vel_rel, - scale=0.05, - noise=Unoise(-1.5, 1.5), - ), - } - - -policy = ObservationGroupCfg(enable_corruption=True, terms=_obs_terms()) -critic = ObservationGroupCfg(enable_corruption=False, terms=_obs_terms()) -``` - -## Writing a custom sensor - -Subclass `Sensor[T]` with the desired return type and implement `_compute_data`. Override -`bind` to cache link indices once, `update` to advance integrators, and `reset` to clear per-env -state — always calling `super()` first so the cache-invalidation chain stays intact. - -```python -from dataclasses import dataclass -from typing import TYPE_CHECKING - -import torch - -from genelab.sensor import Sensor, SensorCfg - -if TYPE_CHECKING: - from genelab.envs.manager_based_rl_env import ManagerBasedRlEnv - - -@dataclass -class JointTorqueSensorCfg(SensorCfg): - def build(self) -> "JointTorqueSensor": - return JointTorqueSensor(self) - - -class JointTorqueSensor(Sensor[torch.Tensor]): - def bind(self, env: "ManagerBasedRlEnv") -> None: - super().bind(env) - # Cache anything that depends on env.link_names / env.joint_names here. - - def _compute_data(self) -> torch.Tensor: - assert self._env is not None - rs = self._env.robot_state - return self._env.joint_kp * (rs.joint_pos - self._env.default_joint_pos) -``` - -Add the cfg to `InteractiveSceneCfg.sensors` and the sensor is reachable as -`env.sensors[name].data`. - -## See also - -- [Configs](configs.md) +- [Recording and plotting](recording.md) +- [Module Map](../reference/module-map.md) - [API Reference](../api/reference.md) diff --git a/docs/concepts/sensors.zh.md b/docs/concepts/sensors.zh.md index 1fa18590..b6eed6ba 100644 --- a/docs/concepts/sensors.zh.md +++ b/docs/concepts/sensors.zh.md @@ -1,265 +1,41 @@ # 传感器 -Genesis 不解析 MJCF 中的 `` 段，因此 GeneLab 提供了一套 backend-agnostic 的传感器抽象。 -接口与 mjlab 的 `SensorCfg` / `Sensor[T]` 对齐，每个具体传感器从 env 的 `RobotState` 而非 -MuJoCo sensordata 取数 —— 观测与奖励 term 可在两套后端之间无缝迁移。 +Genesis 不会通过 MJCF sensor block 暴露所有信号，因此 GeneLab 提供后端感知的传感器抽象。 +传感器产出带缓存的数据，供 observation、reward、recording 和自定义代码消费。 ## 生命周期 -`bind(env)` 在构造时调用一次。每个控制步 `update(dt)` 失效缓存；首次访问 `data` 触发 -`_compute_data` 惰性求值；`reset(env_ids)` 同样失效缓存，并允许有状态传感器清空对应 env 的 -缓冲区。 +传感器声明在 `InteractiveSceneCfg.sensors` 中。`ManagerBasedRlEnv` 在 scene 存在后构建并绑定它们。 +每个控制步会使缓存失效；第一次访问 `sensor.data` 时计算新输出。reset 会清理每 env 状态。 -```python -class Sensor[T](ABC): - def bind(self, env: "ManagerBasedRlEnv") -> None: ... - @property - def data(self) -> T: ... - def update(self, dt: float) -> None: ... - def reset(self, env_ids: torch.Tensor | None = None) -> None: ... - @abstractmethod - def _compute_data(self) -> T: ... -``` +这种惰性缓存保证同一个控制步内多次 observation 和 reward 读取一致。 -env 自动接好生命周期：`InteractiveSceneCfg.sensors` 里构造出的传感器，`update` 在每次 -articulation 刷新之后被调用，`reset` 在 `_reset_idx` 内部被调用 —— 奖励和观测 term 总能读 -到当前步的数据。 +## 内置传感器族 -## 注册到 scene +| 传感器 | 用途 | +|---|---| +| `BodyVelocitySensor` | offset 处 link 线速度或角速度。 | +| `IMUSensor` | 姿态、projected gravity、线/角加速度。 | +| `CameraSensor` | 挂到 link 上的 RGB-D 相机。 | +| `ContactSensor` | 每 link 接触力和可选 air/contact time。 | +| `FrameTransformerSensor` | target frame 相对 source frame 的姿态。 | +| `RayCastSensor` | grid、ring、hemisphere ray pattern。 | +| `TerrainHeightSensor` | 地形感知 locomotion 的向下 height scan。 | +| `SelfContactSensor` | 自碰撞/自接触信号。 | +| `RootAngularMomentumSensor` | root angular momentum 诊断。 | -`InteractiveSceneCfg.sensors` 是一个 `SensorCfg` 元组。`ManagerBasedRlEnv.__init__` 对每 -个 cfg 调用 `build()` 并把生成的传感器绑定到 env。运行时通过 `env.sensors[name].data` 访 -问。 +## MDP term 中的传感器数据 -```python -from genelab.configs import InteractiveSceneCfg, SimulationCfg -from genelab.sensor import BodyVelocitySensorCfg, ContactSensorCfg +observation term 可以通过 `mdp.sensor_data` 或自定义 callable 读取传感器。reward 和 metric 可以直接访问 `env.sensors[name].data`。 -simulation = SimulationCfg(num_envs=4096) -scene = InteractiveSceneCfg( - sensors=( - BodyVelocitySensorCfg( - name="imu_lin_vel", - link_name="pelvis", - offset=(0.04525, 0.0, -0.08339), - measure="lin_vel", - ), - ContactSensorCfg( - name="feet_ground_contact", - link_names=("left_ankle_roll_link", "right_ankle_roll_link"), - track_air_time=True, - ), - ), -) -``` +保持传感器名称稳定；它们会成为配置路径、recording source 和 term params。 -## 内置传感器 +## Camera 注意事项 -### BodyVelocitySensor +并行 RGB-D 渲染需要 CUDA 和 `InteractiveSceneCfg.batch_render=True`。把 camera task 当成不同于轻量状态 RL task 的硬件 profile。 -对应 MuJoCo 的 `` / ``：在与 link 刚性连接的 site 上读取线速度或角速度， -结果旋转到 link body frame。支持杠杆臂偏移与可选的每 env 均匀分布偏置。 +## 继续阅读 -| 字段 | 类型 | 含义 | -|------|------|------| -| `link_name` | `str` | site 刚性连接到的 link。 | -| `offset` | `tuple[float, float, float]` | site 在 link 局部系下的位置，`ang_vel` 模式忽略。 | -| `measure` | `Literal["lin_vel", "ang_vel"]` | 选择 velocimeter（线速度）或 gyro（角速度）。 | -| `bias_range` | `tuple[float, float] \| None` | 每 env 均匀偏置，`reset` 时重采样。 | - -velocimeter 计算公式为 `v_site = v_link + ω × (R_link · offset)`，再旋转到 body frame —— -与 MuJoCo 的杠杆臂约定一致，使得 GeneLab 训出的策略读到的信号与 mjlab 参考实现完全相同。 - -### IMUSensor - -刚性附着在某个 link 上的惯性测量单元，输出姿态（`link_quat_w`）、body frame 投影单位 -重力、以及 body frame 下的线 / 角加速度。加速度通过对世界系速度做有限差分得到；每次 -`reset` 之后的第一个控制步加速度恒为零，避免使用上一阶段残留的 `prev` 缓冲产生 -伪冲击 —— 接近 reset 边界的 reward / observation term 在消费 `lin_acc_b` 或 -`ang_acc_b` 时需要意识到这点。 - -| 字段 | 类型 | 含义 | -|------|------|------| -| `link_name` | `str` | IMU 刚性附着的 link。 | -| `offset` | `tuple[float, float, float]` | site 在 link 局部系下的位置；线加速度计算中携带杠杆臂项。 | -| `gravity_bias` | `bool` | 为 `True` 时输出比力 `R^T (a_w - g_w)`，与真实加速度计一致 —— 静止时读到 `+g` 沿向上轴。 | -| `bias_range_lin_acc` | `tuple[float, float] \| None` | 加速度计每 env 常数偏置，`reset` 时重采样。 | -| `bias_range_ang_acc` | `tuple[float, float] \| None` | 陀螺导出量每 env 常数偏置，`reset` 时重采样。 | - -`IMUSensor` 不输出线速度或角速度 —— 这是 `BodyVelocitySensor` 的职责，二者在同一 link -上可以并存。控制步频率下的有限差分内部不做滤波；如需平滑，可在 `ObservationTermCfg` -上叠加 `noise` + `scale`，或者在观测 term 里自行做 EMA。 - -### CameraSensor - -刚性挂载的 RGB-D 相机，通过 4×4 变换螺接到指定 link 上。`pre_build_genesis()` 在 -`gs_scene.build` 之前先跑，正好让相机赶在 `BatchRenderer` 快照之前注册——它调 -`gs_scene.add_camera` 拿一个 Genesis camera 句柄，根据 `offset_pos` / `offset_quat` -构造偏置矩阵后 `cam.attach`。`bind()` 之后再去 `env.link_names` 里做 link 名校验。每次 -访问 `data` 都会先 `cam.move_to_attach()` 再 `cam.render(...)`，缓存的 `CameraData` -携带 `rgb` (uint8) 与/或 `depth` (米，float)，形状 `(num_envs, H, W, 3)` / -`(num_envs, H, W)`。 - -| 字段 | 类型 | 含义 | -|------|------|------| -| `link_name` | `str` | 相机挂载到的 link（通过 `env.link_names` 解析）。 | -| `offset_pos` | `tuple[float, float, float]` | 相机原点在 link 局部系下的位置。 | -| `offset_quat` | `tuple[float, float, float, float]` | 相机相对 link 的姿态（wxyz）；Genesis 约定 `+x` 为前向。 | -| `width`、`height` | `int` | 像素分辨率；同一 scene 内所有 BatchRender 相机分辨率必须一致。 | -| `fov` | `float` | 垂直视场角（度）。 | -| `near`、`far` | `float` | 深度裁剪面（米）。 | -| `render_rgb`、`render_depth` | `bool` | 两条通道独立开关；关闭的那条在输出数据类里为 `None`。 | - -!!! warning "BatchRenderer 是唯一的多 env 后端" - 多 env RGB-D 需要 `gs.init(backend=gs.cuda)` 加 `InteractiveSceneCfg.batch_render=True` - —— scene 会把 `gs.renderers.BatchRenderer(use_rasterizer=False)` 传给 `gs.Scene`。 - 仅 Linux x86-64 + CUDA 可用。macOS / 纯 CPU 环境下模块本身可正常 import，但 - `gs_scene.build` 一开始按相机列表配 renderer 时就会抛错。 - -### ContactSensor - -按 link 名汇总 `robot.get_links_net_contact_force()` 输出。开启 `track_air_time=True` 时， -一个内部状态机会推进每 env 的 `current_air_time` / `current_contact_time`，并在接触状态翻转 -的瞬间把已完成的时长快照到 `last_air_time` / `last_contact_time`。 - -| 字段 | 类型 | 含义 | -|------|------|------| -| `link_names` | `tuple[str, ...]` | 显式列出需要监控的 link。 | -| `link_names_expr` | `str \| None` | 对 `env.link_names` 做正则匹配。 | -| `force_threshold` | `float` | 力幅值（N）大于此值时 `found` 为真。 | -| `track_air_time` | `bool` | 是否分配 air-time / contact-time 状态机。 | - -`data` 是 `ContactData` 数据类，包含 `force`、`force_norm`、`found` 以及四个 air-time 缓冲。 -配套的观测 term —— `mdp.foot_air_time`、`mdp.foot_contact`、`mdp.foot_contact_forces` —— -直接读取这个数据类。 - -### FrameTransformerSensor - -无状态的正向运动学探针：输出一个或多个目标系相对于单一源系的位姿，源 / 目标在各自 -link 局部系下都支持刚性偏移。数据类同时暴露世界系位姿与源系位姿，下游 reward / -observation term 可按需取用而不必自行复合运算。典型场景：在 base link 系下读取末端执行器 -位姿、对比两个脚尖系做步态约束、把负载位姿表达在抓取点局部系下。 - -| 字段 | 类型 | 含义 | -|------|------|------| -| `source_link_name` | `str` | 表达 target 所用的参考系。 | -| `source_offset_pos` | `tuple[float, float, float]` | 源端在 link 局部系下的位移偏移。 | -| `source_offset_quat` | `tuple[float, float, float, float]` | 源端旋转偏移（wxyz）。 | -| `target_frames` | `tuple[TargetFrameCfg, ...]` | 保序的 target 列表 —— 输出 `N` 轴顺序即 cfg 顺序。 | - -每个 `TargetFrameCfg` 含 `link_name`、可选的 `name`（默认为 `link_name`）、`offset_pos` -与 `offset_quat`。传感器属性 `target_names: tuple[str, ...]` 暴露给下游按名字定位列。 -`bind` 会拒绝未知 link 名以及空 `target_frames`。 - -### Ray-cast patterns - -`RayCastSensorCfg.pattern` 接受三种内置 pattern 数据类。自定义 pattern 满足同一非正式协议 -即可接入 —— `num_rays() -> int` 和 `generate(device) -> (starts, dirs)`，两个张量都为 -`(M, 3)`，定义在传感器局部系下 —— 无须改动 `RayCastSensor` 本身。 - -`GridPattern` 是默认：所有光线平行，原点排在 2D 矩形上。`RingPattern` 从原点发出 -`num_horizontal × num_vertical` 条光线，方位角和俯仰角各自均匀分布 —— 典型的多线 LIDAR -布局。`HemispherePattern` 用 Fibonacci 格点把 `num_rays_target` 条光线均匀分布到以 -`pole_axis` 为极轴、半角为 `polar_fov_deg` 的球冠上；90° 覆盖整个半球，180° 即整个球面。 - -| Pattern | 关键字段 | 典型用途 | -|---------|----------|----------| -| `GridPattern` | `resolution`、`size`、`direction` | 高度场网格、面扫 | -| `RingPattern` | `num_horizontal`、`num_vertical`、`horizontal_fov_deg`、`vertical_fov_deg` | 平面 / 多线 LIDAR | -| `HemispherePattern` | `num_rays_target`、`pole_axis`、`polar_fov_deg` | 接近告警球罩、向下覆盖 | - -`RingPattern` 将水平角恰好 ±360° 视为环绕扫描，自动丢弃首末重复方位；任意其它跨度（例如 -`(-30, 30)` 的前向扫描器）则把两端点都包含进去。`HemispherePattern.num_rays()` 严格返回 -`num_rays_target`——Fibonacci 格点不会产生取整误差。 - -### TerrainHeightSensor - -锚定在某个 link 上的 2D 下射光线网格，输出每条光线相对地形的高度（正值表示在地形上方）， -适合作为 critic 的 privileged `height_scan` 观测。默认 backend 把每条光线打到位于 -`ground_height` 的水平面上；当场景挂上 `TerrainImporter` 时，内部 `RayCastSensor` 会 -对 height-field 做双线性采样。非平地场景的扩展点是继承 `RayCastSensor` 并重载 -`_intersect_world_rays`，可换成 BVH 或其他自定义 backend。 - -| 字段 | 类型 | 含义 | -|------|------|------| -| `link_name` | `str` | 网格原点锚定的 link。 | -| `pattern` | `GridPattern \| RingPattern \| HemispherePattern` | Pattern 几何。 | -| `attach_yaw_only` | `bool` | 仅按 yaw 旋转 pattern，使其保持水平。 | -| `max_distance` | `float` | 光线距离上限。 | -| `ground_height` | `float` | 默认平面 backend 使用的地面高度。 | - -## 给观测加噪 - -`ObservationTermCfg.noise` 接收加性噪声模型（`Unoise(n_min, n_max)` 或 -`Gnoise(mean, std)`）。`ObservationGroupCfg.enable_corruption` 控制是否真正加噪 —— 默认关闭， -让 critic 看到 ground truth。逐 term 的管线顺序是 **noise → scale → clip**，因此噪声幅值 -定义在原始信号空间：在带 `scale=0.05` 的原始 `joint_vel` 上加 `Unoise(-1.5, 1.5)`，最终的 -抖动量为 ±0.075。 - -惯例做法是 policy / critic 共享 term，只在 `enable_corruption` 上分叉： - -```python -from genelab import mdp -from genelab.managers import ObservationGroupCfg, ObservationTermCfg -from genelab.mdp.noise import Unoise - - -def _obs_terms() -> dict[str, ObservationTermCfg]: - return { - "base_lin_vel": ObservationTermCfg( - func=mdp.sensor_data, - params={"sensor_name": "imu_lin_vel"}, - noise=Unoise(-0.5, 0.5), - ), - "joint_vel": ObservationTermCfg( - func=mdp.joint_vel_rel, - scale=0.05, - noise=Unoise(-1.5, 1.5), - ), - } - - -policy = ObservationGroupCfg(enable_corruption=True, terms=_obs_terms()) -critic = ObservationGroupCfg(enable_corruption=False, terms=_obs_terms()) -``` - -## 自定义传感器 - -继承 `Sensor[T]` 并指定返回类型，实现 `_compute_data`。如需缓存 link 索引、推进积分器或 -清空 env 状态，重载 `bind` / `update` / `reset` 并先调用 `super()`，让缓存失效链路保持完整。 - -```python -from dataclasses import dataclass -from typing import TYPE_CHECKING - -import torch - -from genelab.sensor import Sensor, SensorCfg - -if TYPE_CHECKING: - from genelab.envs.manager_based_rl_env import ManagerBasedRlEnv - - -@dataclass -class JointTorqueSensorCfg(SensorCfg): - def build(self) -> "JointTorqueSensor": - return JointTorqueSensor(self) - - -class JointTorqueSensor(Sensor[torch.Tensor]): - def bind(self, env: "ManagerBasedRlEnv") -> None: - super().bind(env) - # 在这里缓存依赖 env.link_names / env.joint_names 的索引。 - - def _compute_data(self) -> torch.Tensor: - assert self._env is not None - rs = self._env.robot_state - return self._env.joint_kp * (rs.joint_pos - self._env.default_joint_pos) -``` - -把 cfg 加入 `InteractiveSceneCfg.sensors` 后，传感器即可通过 `env.sensors[name].data` 访问。 - -## See also - -- [配置系统](configs.md) +- [数据录制与实时绘图](recording.md) +- [模块地图](../reference/module-map.md) - [API 参考](../api/reference.md) diff --git a/docs/concepts/terrains.en.md b/docs/concepts/terrains.en.md index d0f8981f..223e9876 100644 --- a/docs/concepts/terrains.en.md +++ b/docs/concepts/terrains.en.md @@ -1,128 +1,36 @@ # Terrains -`genelab.terrains` is a thin GeneLab-shaped wrapper over Genesis's native -`gs.morphs.Terrain` morph. Sub-terrain configs map onto Genesis's built-in subterrain -string types, `TerrainGenerator` assembles them into a 2D grid, and `TerrainImporter` -spawns the terrain into the scene plus tracks per-env curriculum state. +GeneLab terrains are generated height-field grids that can be imported into Genesis scenes and read +by ray-based sensors and curricula. -## Why a dedicated layer +## Why a terrain layer -Genesis ships the height-field terrain primitives, but its native API uses pydantic -options keyed by string names (`"pyramid_stairs_terrain"`, `"random_uniform_terrain"`). -A GeneLab cfg layer keeps the surface dataclass-shaped (matching the rest of the project), -gives every sub-terrain a typed Python class with explicit fields, and tracks per-env -spawn / level state so curriculum terms can promote and demote without touching the -Genesis handle directly. +Locomotion tasks need terrain geometry, spawn origins, difficulty levels, and height scans to agree. +Keeping terrain generation in one layer lets the scene, curriculum, and sensors share the same +source of truth. -## Five shipped sub-terrains +## Building blocks -| Class | Genesis type | Parameters | -|---|---|---| -| `FlatPatchCfg` | `flat_terrain` | none | -| `PyramidStairsCfg` | `pyramid_stairs_terrain` | `step_width`, `step_height` | -| `RandomRoughCfg` | `random_uniform_terrain` | `min_height`, `max_height`, `step`, `downsampled_scale` | -| `SlopeCfg` | `sloped_terrain` | `slope` | -| `WaveCfg` | `wave_terrain` | `num_waves`, `amplitude` | +| Config | Shape | +|---|---| +| `FlatPatchCfg` | Flat sub-terrain. | +| `PyramidStairsCfg` | Stair pattern. | +| `RandomRoughCfg` | Random rough surface. | +| `SlopeCfg` | Sloped patch. | +| `WaveCfg` | Wave-like height field. | +| `TerrainGeneratorCfg` | Grid layout and selected sub-terrains. | +| `TerrainImporter` | Runtime bridge into Genesis. | -`PyramidStairsCfg(step_height=-0.1)` builds concentric descending steps; positive -`step_height` would invert the pyramid. `RandomRoughCfg` samples bumps at -`downsampled_scale` then upsamples, so the apparent feature size is independent of -`horizontal_scale`. `SlopeCfg(slope=-0.5)` tilts the patch uniformly (negative follows -Genesis's default downhill direction); `WaveCfg` lays sinusoidal undulations across -the patch — a useful intermediate difficulty between flat and pyramid-stairs. Four -more Genesis types (`pyramid_sloped`, `discrete_obstacles`, `stairs`, `stepping_stones`) -are not wrapped yet — extending `SubTerrainCfg` to expose them is a 20-line task. +## Integration points -## Composing a terrain grid +| Consumer | Uses terrain for | +|---|---| +| `InteractiveScene` | Adds generated geometry to the Genesis scene. | +| `TerrainHeightSensor` | Samples heights under a ray grid. | +| `terrain_levels_vel` curriculum | Promotes/demotes env difficulty by performance. | -```python -from genelab.terrains import ( - FlatPatchCfg, - PyramidStairsCfg, - RandomRoughCfg, - TerrainGeneratorCfg, -) -from genelab.configs import InteractiveSceneCfg +## Where to continue -scene = InteractiveSceneCfg( - env_spacing=(0.0, 0.0), - terrain=TerrainGeneratorCfg( - num_rows=4, - num_cols=4, - subterrain_size=(8.0, 8.0), - horizontal_scale=0.1, - vertical_scale=0.005, - sub_terrains={ - "flat": FlatPatchCfg(proportion=1.0), - "stairs": PyramidStairsCfg(step_width=0.5, step_height=-0.08, proportion=2.0), - "rough": RandomRoughCfg(min_height=-0.05, max_height=0.05, proportion=1.0), - }, - curriculum=True, - seed=0, - ), -) -``` - -`sub_terrains` is a `dict[str, SubTerrainCfg]`; the key is the local cfg label, the -value carries the Genesis parameters. `layout` defaults to `None` (random tiling by -`proportion`); pass a `tuple[tuple[str, ...], ...]` of cfg keys to pin the grid -deterministically. - -!!! warning "One parameter set per Genesis type" - Genesis keys `subterrain_parameters` by the *type string*, not per cell. Two - `PyramidStairsCfg` instances with different `step_width` collapse to a single - parameter set. Use distinct keys for distinct geometries, or rely on - `randomize=True` on the importer for in-type variation. - -## Ray-cast sensor integration - -When `InteractiveSceneCfg.terrain` is set, `InteractiveScene.build()` spawns -`gs.morphs.Terrain` instead of the default plane and exposes the importer as -`scene.terrain`. `RayCastSensor` checks for it and, when present, bilinearly samples -`terrain.heightfield_tensor` at each ray's world `(x, y)` to compute hit elevation. -`TerrainHeightSensor` is a thin wrapper around the same path, so the per-link height -scan output reflects the true terrain instead of a constant plane. - -Vertical and near-vertical rays match the underlying height field to within -`vertical_scale`. Oblique rays use the elevation at the ray-start `(x, y)` — accurate -for height-scan grids but approximate for BVH-style queries; override -`RayCastSensor._intersect_world_rays` for those. - -## Curriculum: `terrain_levels_vel` - -```python -from genelab.managers import CurriculumTermCfg -from genelab.mdp import terrain_levels_vel - -curriculum_cfg = { - "terrain_levels": CurriculumTermCfg( - func=terrain_levels_vel, - params={"distance_threshold": 2.0, "demote_ratio": 0.5}, - ), -} -``` - -Each reset, the term compares every env's walked distance (XY norm of `root_pos - -spawn_pos`) against `distance_threshold`. Envs that walked further than the threshold -move up one row of the difficulty grid (capped at `num_rows - 1`); envs that walked -less than `distance_threshold * demote_ratio` move down (floored at 0). The curriculum -then writes the new spawn pose into the sim via `Articulation.write_root_state`, so -the next episode starts on the new sub-terrain. The manager logs the mean level under -`Curriculum/terrain_levels`. - -## Failure modes worth knowing - -* **Empty `sub_terrains`** — `TerrainGenerator.__init__` raises immediately; the - generator has nothing to tile. -* **`layout` shape mismatch** — `ValueError` lists the actual vs declared shape. -* **`layout` references unknown key** — `ValueError` lists the missing keys. -* **Per-env state access before build** — `terrain_levels` / `terrain_cols` / - `spawn_pos` raise `RuntimeError` until `init_per_env_state` runs; - `InteractiveScene.build` calls it automatically. -* **`heightfield` access before spawn** — `RuntimeError`; spawn first, then read. - -## See also - -- [Configs](configs.md) - [Sensors](sensors.md) -- [Actuators](actuators.md) +- [Managers and MDP terms](managers.md) +- [API Reference](../api/reference.md) diff --git a/docs/concepts/terrains.zh.md b/docs/concepts/terrains.zh.md index 1a83d125..832566a3 100644 --- a/docs/concepts/terrains.zh.md +++ b/docs/concepts/terrains.zh.md @@ -1,90 +1,33 @@ # 地形 -`genelab.terrains` 是 Genesis 原生 `gs.morphs.Terrain` morph 之上的一层 GeneLab 风格薄包装。子地形 cfg 对应 Genesis 内建的 subterrain 字符串类型，`TerrainGenerator` 把它们拼成一个二维网格，`TerrainImporter` 负责把地形 spawn 进场景并跟踪每个 env 的课程状态。 +GeneLab 地形是生成式 height-field 网格，可导入 Genesis scene，并被 ray-based sensor 和 curriculum 读取。 -## 为什么需要专门的一层 +## 为什么需要地形层 -Genesis 自带 height-field 地形原语，但其原生 API 使用按字符串名键控的 pydantic options（`"pyramid_stairs_terrain"`、`"random_uniform_terrain"`）。GeneLab 的 cfg 层把接口保持成 dataclass 形态（与项目其它部分一致），让每种子地形拥有带显式字段的 typed Python 类，并跟踪每个 env 的 spawn / level 状态，让课程项可以提升和降级而不直接接触 Genesis 句柄。 +locomotion 任务需要地形几何、spawn origin、难度等级和 height scan 保持一致。把地形生成放在一层里，可以让 scene、curriculum 和 sensor 共享同一事实来源。 -## 自带 5 种子地形 +## 构件 -| 类 | Genesis 类型 | 参数 | -|---|---|---| -| `FlatPatchCfg` | `flat_terrain` | 无 | -| `PyramidStairsCfg` | `pyramid_stairs_terrain` | `step_width`、`step_height` | -| `RandomRoughCfg` | `random_uniform_terrain` | `min_height`、`max_height`、`step`、`downsampled_scale` | -| `SlopeCfg` | `sloped_terrain` | `slope` | -| `WaveCfg` | `wave_terrain` | `num_waves`、`amplitude` | +| 配置 | 形态 | +|---|---| +| `FlatPatchCfg` | 平坦子地形。 | +| `PyramidStairsCfg` | 楼梯 pattern。 | +| `RandomRoughCfg` | 随机粗糙表面。 | +| `SlopeCfg` | 斜坡 patch。 | +| `WaveCfg` | 波浪状 height field。 | +| `TerrainGeneratorCfg` | 网格布局与子地形选择。 | +| `TerrainImporter` | 到 Genesis 的运行时桥接。 | -`PyramidStairsCfg(step_height=-0.1)` 生成一圈圈向下的同心方形台阶；正的 `step_height` 会反转金字塔方向。`RandomRoughCfg` 在 `downsampled_scale` 下采样后再上采样，所以表观特征尺寸与 `horizontal_scale` 无关。`SlopeCfg(slope=-0.5)` 把整块斜成一个固定坡（负号沿 Genesis 默认的下坡方向）；`WaveCfg` 在整块上铺正弦起伏 —— 介于平地与金字塔台阶之间的中间难度。Genesis 还有另外 4 种类型（`pyramid_sloped`、`discrete_obstacles`、`stairs`、`stepping_stones`）尚未包装 —— 扩展 `SubTerrainCfg` 暴露它们是一个 20 行的小活。 +## 接入点 -## 拼装地形网格 +| 消费者 | 用途 | +|---|---| +| `InteractiveScene` | 把生成几何加入 Genesis scene。 | +| `TerrainHeightSensor` | 在 ray grid 下采样高度。 | +| `terrain_levels_vel` curriculum | 按表现提升/降低 env 难度。 | -```python -from genelab.terrains import ( - FlatPatchCfg, - PyramidStairsCfg, - RandomRoughCfg, - TerrainGeneratorCfg, -) -from genelab.configs import InteractiveSceneCfg +## 继续阅读 -scene = InteractiveSceneCfg( - env_spacing=(0.0, 0.0), - terrain=TerrainGeneratorCfg( - num_rows=4, - num_cols=4, - subterrain_size=(8.0, 8.0), - horizontal_scale=0.1, - vertical_scale=0.005, - sub_terrains={ - "flat": FlatPatchCfg(proportion=1.0), - "stairs": PyramidStairsCfg(step_width=0.5, step_height=-0.08, proportion=2.0), - "rough": RandomRoughCfg(min_height=-0.05, max_height=0.05, proportion=1.0), - }, - curriculum=True, - seed=0, - ), -) -``` - -`sub_terrains` 是 `dict[str, SubTerrainCfg]`；key 是本地 cfg 标签，value 携带 Genesis 参数。`layout` 默认为 `None`（按 `proportion` 随机平铺）；传入 `tuple[tuple[str, ...], ...]` 形态的 cfg key 二维表则可以确定性地固定网格。 - -!!! warning "每个 Genesis 类型只有一套参数" - Genesis 的 `subterrain_parameters` 按*类型字符串*键控，而不是按单元格。两个 `step_width` 不同的 `PyramidStairsCfg` 实例会塌成单一一套参数。要区分几何就用不同的 key，要做同类型内变化就在 importer 上启用 `randomize=True`。 - -## ray-cast 传感器接入 - -当 `InteractiveSceneCfg.terrain` 被设置时，`InteractiveScene.build()` 会 spawn `gs.morphs.Terrain` 而不是默认 plane，并把 importer 暴露为 `scene.terrain`。`RayCastSensor` 检测到地形后，会对每条射线在世界坐标 `(x, y)` 处对 `terrain.heightfield_tensor` 做双线性采样，得到命中点高度。`TerrainHeightSensor` 是同一路径上的薄封装，所以每个关节的 height scan 输出反映的是真实地形而不是常数平面。 - -垂直 / 接近垂直的射线，输出与底层 height field 的误差不超过 `vertical_scale`。斜射线使用射线起点 `(x, y)` 处的高度 —— 对 height-scan 网格足够准，但对 BVH 风格查询是近似的；这类场景请覆写 `RayCastSensor._intersect_world_rays`。 - -## 课程项: `terrain_levels_vel` - -```python -from genelab.managers import CurriculumTermCfg -from genelab.mdp import terrain_levels_vel - -curriculum_cfg = { - "terrain_levels": CurriculumTermCfg( - func=terrain_levels_vel, - params={"distance_threshold": 2.0, "demote_ratio": 0.5}, - ), -} -``` - -每次 reset，该项把每个 env 的已走距离（`root_pos - spawn_pos` 在 XY 上的范数）与 `distance_threshold` 比较。走得超过阈值的 env 在难度网格中上移一行（封顶在 `num_rows - 1`）；走得不到 `distance_threshold * demote_ratio` 的 env 下移一行（地板在 0）。课程项随后通过 `Articulation.write_root_state` 把新 spawn 位姿写入仿真，下一个 episode 就从新子地形开始。manager 自动以 `Curriculum/terrain_levels` 记录平均 level。 - -## 需要知道的失败模式 - -* **`sub_terrains` 为空** —— `TerrainGenerator.__init__` 立刻抛错；生成器没东西可拼。 -* **`layout` 形状不匹配** —— `ValueError` 列出实际形状与声明形状。 -* **`layout` 引用未知 key** —— `ValueError` 列出缺失的 key。 -* **build 前访问 per-env 状态** —— `terrain_levels` / `terrain_cols` / `spawn_pos` 在 `init_per_env_state` 之前抛 `RuntimeError`；`InteractiveScene.build` 会自动调一次。 -* **spawn 前访问 `heightfield`** —— `RuntimeError`；先 spawn 再读。 - -## See also - -- [Configs](configs.md) -- [Sensors](sensors.md) -- [Actuators](actuators.md) +- [传感器](sensors.md) +- [Manager 与 MDP term](managers.md) +- [API 参考](../api/reference.md) diff --git a/docs/contributing.en.md b/docs/contributing.en.md index b28faeaf..ca993754 100644 --- a/docs/contributing.en.md +++ b/docs/contributing.en.md @@ -1,11 +1,48 @@ # Contributing -The canonical contributor guide lives in [`CONTRIBUTING.md`](https://github.com/KraHsu/GeneLab/blob/main/CONTRIBUTING.md) -at the repository root and is mirrored below. +This page is for contributors changing GeneLab itself. Downstream robot projects should usually use +an extension package instead. ---- +## Development setup -{% - include-markdown "../CONTRIBUTING.md" - heading-offset=1 -%} +```bash +uv sync --extra torch-cpu +uv run genelab cache +uv run pytest +uv run ruff check +uv run pyright +``` + +Pick one CUDA extra instead of `torch-cpu` when developing GPU workflows. + +## Documentation rules + +The docs follow Diátaxis: + +| Type | Purpose | +|---|---| +| Tutorial | Learning path from zero to a working result. | +| How-to | Task-focused instructions for experienced users. | +| Reference | Facts, flags, APIs, and defaults. | +| Explanation | Architecture and design rationale. | + +Keep pages in one type where possible and cross-link rather than mixing long conceptual digressions +into task guides. + +## Code style + +- Use Python 3.12+ syntax. +- Keep registry-time imports light; avoid starting Genesis at import time. +- Prefer dataclass configs for user-visible knobs. +- Keep task/example code outside `src/genelab/` unless it is part of the framework. + +## Before a PR + +```bash +uv run pytest +uv run ruff check +uv run pyright +uv run mkdocs build --strict +``` + +For docs-only changes, still run the MkDocs strict build. diff --git a/docs/contributing.zh.md b/docs/contributing.zh.md index d09e5d24..ce0419d6 100644 --- a/docs/contributing.zh.md +++ b/docs/contributing.zh.md @@ -1,94 +1,46 @@ # 贡献指南 -详细的贡献者指南以英文为权威版本，见仓库根目录 -[`CONTRIBUTING.md`](https://github.com/KraHsu/GeneLab/blob/main/CONTRIBUTING.md)。 +这页面向修改 GeneLab 本身的贡献者。下游机器人项目通常应使用扩展包。 ## 开发环境 -GeneLab 使用 [uv](https://github.com/astral-sh/uv) 并要求 Python `>=3.12`。挑选一个互斥的 -`torch-*` extra： - ```bash -uv sync --extra torch-cpu # 或 torch-cu126 / torch-cu128 / torch-cu130 -uv run genelab --help # CLI 烟雾测试 -uv run genelab cache # 创建 Genesis / Matplotlib 用的项目本地 .cache 目录 +uv sync --extra torch-cpu +uv run genelab cache +uv run pytest +uv run ruff check +uv run pyright ``` -`examples/genelab_examples/` 已加入 `pytest` 的 `pythonpath`，测试可直接 import 而无需安装。 +开发 GPU 工作流时，用一个 CUDA extra 替代 `torch-cpu`。 -## Python 3.12+ 风格 +## 文档规则 -GeneLab 锁定 Python `>=3.12`。只写现代代码，不要为旧解释器加兼容垫片： +文档遵循 Diátaxis： -- **不要** `from __future__ import annotations`。注解会立即被求值；只在 `TYPE_CHECKING` 下导入的 - 名字在使用点引号化（`def f(env: "ManagerBasedRlEnv") -> ...`）。 -- **PEP 604 union**：`X | Y`、`X | None`，不要 `Optional[X]` / `Union[X, Y]`。 -- **PEP 585 内置泛型**：`list[T]`、`dict[K, V]`、`tuple[T, ...]`、`type[T]`，不要 `typing.List` 等。 -- **PEP 695 泛型**：`class Foo[T]:`、`def f[T](...)`、`type Alias = ...`，不要 `TypeVar + Generic[T]`。 -- 用 `collections.abc` 代替 `typing` 中的 `Callable` / `Iterable` / `Iterator` / `Sequence` / `Mapping`。 -- 仅以下 `typing` 名字预期出现：`Any`、`cast`、`Protocol`、`Literal`、`Final`、`Annotated`、 - `TYPE_CHECKING`、`runtime_checkable`、`get_args`、`get_origin`、`get_type_hints`。其他名字 - 视为可疑。 +| 类型 | 用途 | +|---|---| +| Tutorial | 从零到可运行结果的学习路径。 | +| How-to | 面向有经验用户的任务指南。 | +| Reference | 事实、flag、API、默认值。 | +| Explanation | 架构和设计原因。 | -## 文档约定 +尽量让单页只承担一种文档类型，通过交叉链接连接，而不是把长篇概念解释塞进任务指南。 -文档位于 `docs/`，由 MkDocs Material 与 `mkdocs-static-i18n` 插件构建。每个内容页都有 -`.en.md` / `.zh.md` 双语版本；站点在 `/` 提供英文、`/zh/` 提供中文。下列规则同时适用于两种 -语言： +## 代码风格 -- **双语平行强制要求。** `.en.md` 修改时，同次提交需修改对应的 `.zh.md`。两文件需在标题数与 - 顺序、代码块（变量占位符除外）、表格结构、admonition、`See also` 条目上保持一致。 -- **章节标题用名词或动名词短语**，不要祈使语气。例如 `Scaffold output` / `生成的目录结构`、 - `Running a task` / `运行任务` —— 而不是 `Play a task` / `运行一个任务`。结尾块统一叫 - `## See also`，不要 `## Next steps`。 -- **不要写"本页介绍……"式开场白。** 避免在第一段铺垫"本页覆盖……"、`This walks through…`、 - `In this guide we'll…`。直接进入实质内容。 -- **避免第二人称。** 去掉 `you` / `你` / `您`。改用省略主语（操作命令用祈使语气）或名词短语 - （`The CLI exposes…` / `CLI 暴露…`）。 -- **正文不要插中段跨页跳转链接。** 不在正文段落里写 `see [Foo](...)` / `详见 [Foo](...)`。 - 把相关页面收在结尾的 `## See also` 块里，**≤ 3 条**，只放补充阅读（必要的下一步页面由 - 左侧导航承担）。 -- **callout 用 admonition，不用 blockquote。** `!!! warning "Title"` / `!!! tip "Title"` / - `!!! note "Title"`。`>` 只用于真正的引文。 -- **稳定显式锚点。** 含非 ASCII 字符、数字前缀或可能改名的标题，追加 `{ #stable-id }`。例如： - `## 5. Advanced: end-to-end RL on Unitree G1 { #unitree-g1 }`。跨链时使用该 slug，不要使用自动 - 生成的 slug。 -- **`.zh.md` 中的中英混排空格。** 中文字符与相邻的 ASCII 单词、数字或行内代码之间留一个空格 - （`运行 \`uv sync\``，不是 `运行\`uv sync\``）。 -- **`mkdocs build --strict` 必须通过。** 提交涉及文档的 PR 前运行 - `uv run mkdocs build --strict`——mkdocs 默认随 `uv sync` 安装。strict 模式会因任何未解析的 - 相对链接或锚点失败。 +- 使用 Python 3.12+ 语法。 +- 保持注册阶段导入轻量；不要在 import 时启动 Genesis。 +- 用户可见旋钮优先使用 dataclass 配置。 +- 除非属于框架本身，否则 task/example 代码不要放进 `src/genelab/`。 -## PR 前检查 +## PR 前 ```bash -uv run ruff check # lint -uv run ruff format --check # 格式（用 `uv run ruff format` 自动修复） -uv run pyright # src/genelab 的 strict 类型检查 -uv run pytest # 完整测试套件 +uv run pytest +uv run ruff check +uv run pyright +uv run mkdocs build --strict ``` -四项均须通过 —— CI 把同一组检查设为每个 PR 的必需 status checks。 - -## 分支与 PR 流程 - -GeneLab 采用两层流程：feature 分支先汇入 `dev`，再由 `dev` 通过 PR 提升到 `main`。`main` -受保护 —— 直接推送会被拒绝，必需的 CI 检查（`lint`、`typecheck`、`test`）必须全绿后才能 -合入 PR。 - -``` -feat/* ──┐ -fix/* ──┼──> dev ──PR──> main -docs/* ──┘ -``` - -1. 从 `dev` 派生分支，使用描述性前缀：`fix/...`、`feat/...`、`ci/...`、`chore/...`、`docs/...`。 -2. 本地跑上述检查。 -3. 推送后选择开 PR 合入 `dev`，或直接本地合入 `dev` 后推送 `dev` —— 视改动大小而定。 -4. 把 `dev` 提升到 `main` 时，开 `dev` → `main` 的 PR。CI 在每次 push 时自动运行，全绿后才能合入。 -5. 评审期间若 `dev` 前进，把 `dev` 合入分支（或 rebase），让 PR 保持最新。 - -## 提交信息 - -- 祈使语气、首字母大写："Bump…"、"Fix…"、"Add…"。不要前缀、不要 emoji。 -- 主题行控制在约 72 字符内。正文解释 *why* —— *what* 已经被 diff 表达。 +docs-only 改动也要跑 MkDocs strict 构建。 diff --git a/docs/examples/inverted-pendulum.en.md b/docs/examples/inverted-pendulum.en.md index 0ca033a5..566b3cd3 100644 --- a/docs/examples/inverted-pendulum.en.md +++ b/docs/examples/inverted-pendulum.en.md @@ -1,93 +1,47 @@ # Inverted Pendulum -`examples/inverted_pendulum/` ships two PPO tasks for the classical cart-pole problem on flat -ground. The extension mirrors `examples/unitree/`: `ManagerBasedRlEnv` over Genesis, rsl_rl PPO, -a `BodyVelocitySensor` on the pole link, and the unified `genelab train` / `genelab play` CLI. +The inverted pendulum example is the recommended first runnable task. It is small enough for smoke +tests but exercises the full GeneLab train/play path. ## Tasks -| Task id | Problem | -|---------|---------| -| `GeneLab-Inverted-Pendulum-v0` | Single inverted pole on a cart. | -| `GeneLab-Double-Inverted-Pendulum-v0` | Two stacked inverted poles on a cart. | +| Task id | Description | +|---|---| +| `GeneLab-Inverted-Pendulum-v0` | Single pole balancing. | +| `GeneLab-Double-Inverted-Pendulum-v0` | Two-link pole balancing. | -## Installation - -Pick the `torch-*` extra that matches the hardware. +## Install and list ```bash -uv sync --extra torch-cu128 uv pip install -e examples/inverted_pendulum - uv run genelab list tasks -# -> GeneLab-Inverted-Pendulum-v0 -# -> GeneLab-Double-Inverted-Pendulum-v0 ``` -## Single inverted pendulum +Without installation: ```bash -uv run genelab train GeneLab-Inverted-Pendulum-v0 \ - --num-envs 4096 --max-iterations 150 - -uv run genelab play GeneLab-Inverted-Pendulum-v0 \ - --checkpoint logs/rsl_rl/inverted_pendulum_flat//model_150.pt --vis +PYTHONPATH=examples/inverted_pendulum/src \ + uv run genelab --import genelab_inverted_pendulum.tasks list tasks ``` -`--checkpoint` makes `play` route through the RL runner with `--agent trained` by default. - -## Double inverted pendulum +## Run ```bash -uv run genelab train GeneLab-Double-Inverted-Pendulum-v0 \ - --num-envs 4096 --max-iterations 300 - -uv run genelab play GeneLab-Double-Inverted-Pendulum-v0 \ - --checkpoint logs/rsl_rl/double_inverted_pendulum_flat//model_300.pt --vis +uv run genelab play GeneLab-Inverted-Pendulum-v0 --steps 64 +uv run genelab play GeneLab-Inverted-Pendulum-v0 --vis --steps 500 +uv run genelab train GeneLab-Inverted-Pendulum-v0 --num_envs 64 --max_iterations 2 ``` -## Sensor and underactuation - -Only the cart slide joint is PD-controlled. The pole hinges default to `kp=0, kv=0` so the -pendulum stays underactuated and the policy must learn balance through cart motion alone. A -`BodyVelocitySensor` attached to the top pole supplies a noisy angular-velocity observation -(corrupted with `Unoise` in the policy group, clean in the critic group). - -## Interactive disturbance - -Play mode launches a single environment (`num_envs=1`) and enables Genesis' -`MouseInteractionPlugin`. Left-click on the cart or pole and drag — a spring force pulls the -clicked link toward the cursor while the policy keeps balancing. Scroll wheel rotates the drag -plane around the surface normal. Release the button to remove the force. - -## Live pole-angle plot - -In play mode, both env configs wire a `RecordingCfg` that reads -`env.robot_state.joint_pos / joint_vel` for the pole hinge(s) of env 0 and feeds them to a -single `PyQtPlotCfg` window with two stacked subplots — angle (rad) on top, angular velocity -(rad/s) below. Joint convention: `pole_hinge = 0` corresponds to the pole vertically up, the -same zero used by the `pole_upright` / `pole_angle_l2` rewards. The double-pendulum config -plots both poles in the same subplots with separate series. History length is 400 ticks -(≈2 s at the play decimation). The recorder is only attached when `play=True`, so the -4096-env training rollout is not slowed by callable invocations. - -!!! tip "Smoke-test budget" - A 5–10 iteration run with `--num-envs 64 --max-iterations 5` is enough to validate wiring - end-to-end. The reward signal will still be noisy at that scale; convergence requires the - 150 / 300 iteration budgets above. - -## Logs - -Both tasks write to `logs/rsl_rl//_/` like the Unitree examples: +## Code entry points -- `params/env.json` and `params/agent.json` — frozen configs at run time. -- `model_.pt` — checkpoints saved every `save_interval` iterations. -- TensorBoard event files alongside the checkpoints. +| File | Role | +|---|---| +| `tasks.py` | Registers robots, envs, and tasks. | +| `single/env_cfg.py` | Single-pole manager-based env config. | +| `double/env_cfg.py` | Double-pole manager-based env config. | +| `mdp.py` | Example-specific reward and termination helpers. | ## See also -- [Unitree G1 quickstart](../getting-started/quickstart.md#unitree-g1) -- [Sensors](../concepts/sensors.md) -- [Recording and plotting](../concepts/recording.md) -- [Managers and MDP terms](../concepts/managers.md) -- [Play and Train CLI](../cli/play-train.md) +- [Tutorial](../tutorial.md) +- [Task design](../best-practices/task-design.md) diff --git a/docs/examples/inverted-pendulum.zh.md b/docs/examples/inverted-pendulum.zh.md index 10fad7e2..04a99cb2 100644 --- a/docs/examples/inverted-pendulum.zh.md +++ b/docs/examples/inverted-pendulum.zh.md @@ -1,88 +1,46 @@ # 倒立摆 -`examples/inverted_pendulum/` 提供两个在平面上的 cart-pole 经典控制 PPO 任务。整条训练栈与 -`examples/unitree/` 对齐：基于 Genesis 的 `ManagerBasedRlEnv`、rsl_rl PPO、挂在杆上的 -`BodyVelocitySensor`，以及统一的 `genelab train` / `genelab play` CLI。 +倒立摆示例是推荐的第一个可运行任务。它足够小，适合 smoke test，同时覆盖完整的 GeneLab train/play 路径。 -## 任务列表 +## 任务 -| Task id | 问题 | -|---------|------| -| `GeneLab-Inverted-Pendulum-v0` | 小车 + 单杆的倒立摆。 | -| `GeneLab-Double-Inverted-Pendulum-v0` | 小车 + 串联双杆的倒立摆。 | +| Task id | 描述 | +|---|---| +| `GeneLab-Inverted-Pendulum-v0` | 单杆平衡。 | +| `GeneLab-Double-Inverted-Pendulum-v0` | 双连杆平衡。 | -## 安装 - -`torch-*` extra 按硬件挑选。 +## 安装并列出 ```bash -uv sync --extra torch-cu128 uv pip install -e examples/inverted_pendulum - uv run genelab list tasks -# -> GeneLab-Inverted-Pendulum-v0 -# -> GeneLab-Double-Inverted-Pendulum-v0 ``` -## 单倒立摆 +不安装时： ```bash -uv run genelab train GeneLab-Inverted-Pendulum-v0 \ - --num-envs 4096 --max-iterations 150 - -uv run genelab play GeneLab-Inverted-Pendulum-v0 \ - --checkpoint logs/rsl_rl/inverted_pendulum_flat//model_150.pt --vis +PYTHONPATH=examples/inverted_pendulum/src \ + uv run genelab --import genelab_inverted_pendulum.tasks list tasks ``` -传入 `--checkpoint` 会让 `play` 自动经过 RL runner，并默认使用 `--agent trained`。 - -## 双倒立摆 +## 运行 ```bash -uv run genelab train GeneLab-Double-Inverted-Pendulum-v0 \ - --num-envs 4096 --max-iterations 300 - -uv run genelab play GeneLab-Double-Inverted-Pendulum-v0 \ - --checkpoint logs/rsl_rl/double_inverted_pendulum_flat//model_300.pt --vis +uv run genelab play GeneLab-Inverted-Pendulum-v0 --steps 64 +uv run genelab play GeneLab-Inverted-Pendulum-v0 --vis --steps 500 +uv run genelab train GeneLab-Inverted-Pendulum-v0 --num_envs 64 --max_iterations 2 ``` -## 传感器与欠驱动 - -只有小车的 slide 关节通过 PD 控制。两个 pole hinge 默认 `kp=0, kv=0`，保证整体处于欠驱动状态， -策略必须通过小车水平运动间接稳定杆。顶端 pole 上挂载的 `BodyVelocitySensor` 给出一路带噪声的 -角速度观测：policy 观测组用 `Unoise` 做 corruption，critic 观测组直接读取干净值。 - -## 交互式扰动 - -`play` 默认只开 1 个环境（`num_envs=1`），并启用 Genesis 的 `MouseInteractionPlugin`。 -左键点击 cart 或 pole 并拖动，会有一根弹簧把所点击的 link 拉向光标位置；策略仍然在背后试图 -保持平衡。滚轮可绕表面法线旋转拖拽平面，松开左键即移除外力。 - -## 杆角度实时曲线 - -`play` 模式下两份 env cfg 都会接入一个 `RecordingCfg`：读 env 0 上 pole hinge 的 -`env.robot_state.joint_pos / joint_vel`，喂给一个 `PyQtPlotCfg` 窗口，里面两幅上下叠放的 -子图——上面是角度（rad），下面是角速度（rad/s）。关节约定 `pole_hinge = 0` 表示杆竖直朝上， -和 `pole_upright` / `pole_angle_l2` reward 的零点一致。双倒立摆把两根杆放在同样的两幅子图里， -每根一条独立曲线。History 长度 400 个 tick（在 play 的 decimation 下约 2 秒）。recorder 只在 -`play=True` 时挂载，避免 4096 env 训练 rollout 因为 callable 调用变慢。 - -!!! tip "Smoke-test 预算" - 使用 `--num-envs 64 --max-iterations 5` 跑 5–10 次迭代足以验证整条链路。此时 reward 信号 - 仍非常嘈杂，真正收敛需要上面给出的 150 / 300 次迭代预算。 - -## 日志 - -两个任务都把日志写到 `logs/rsl_rl//_/`，结构与 Unitree 示例一致： +## 代码入口 -- `params/env.json` 与 `params/agent.json` —— 运行时冻结的配置快照。 -- `model_.pt` —— 按 `save_interval` 保存的 checkpoint。 -- 同目录下的 TensorBoard 事件文件。 +| 文件 | 作用 | +|---|---| +| `tasks.py` | 注册 robot、env、task。 | +| `single/env_cfg.py` | 单杆 manager-based env 配置。 | +| `double/env_cfg.py` | 双杆 manager-based env 配置。 | +| `mdp.py` | 示例自己的 reward 与 termination helper。 | -## See also +## 另见 -- [Unitree G1 快速开始](../getting-started/quickstart.md#unitree-g1) -- [传感器](../concepts/sensors.md) -- [数据录制与绘图](../concepts/recording.md) -- [Manager 与 MDP term](../concepts/managers.md) -- [play 与 train CLI](../cli/play-train.md) +- [教程](../tutorial.md) +- [设计任务](../best-practices/task-design.md) diff --git a/docs/examples/overview.en.md b/docs/examples/overview.en.md index dcd09598..8c6a33ec 100644 --- a/docs/examples/overview.en.md +++ b/docs/examples/overview.en.md @@ -1,72 +1,33 @@ # Examples -The repository ships several reference extensions under `examples/`. They double as -integration tests for the CLI and registry. Every runnable example has its own page -under this section; this overview is the entry point. +Examples are extension packages that exercise GeneLab capabilities without adding project-specific +code to `src/genelab/`. ## Capability map -| Example | Demonstrates | Doc | +| Example | Package | Shows | |---|---|---| -| `inverted_pendulum` | `ManagerBasedRlEnv` + PPO, `BodyVelocitySensor`, `RecordingCfg`, `MouseInteractionPlugin` | [Inverted Pendulum](inverted-pendulum.md) | -| `unitree` | PPO velocity-tracking + motion imitation on the G1 humanoid, asset zoo + motion-clip pipeline | [Unitree G1](unitree-g1.md) | -| `genelab_showcase` | Seven per-feature minimal play tasks (sensors, ray-cast, contact, terrain, curriculum, actuators, recording) | [Showcase](showcase.md) | -| `genelab_examples/rubiks` | Custom play runner, MJCF generated at runtime, Genesis dynamic constraint API | [Rubik's Cube](rubiks-cube.md) | -| `genelab_examples/wuji_hand` | MJCF asset directory, joint-name → DOF mapping, fixed-trajectory replay | [Wuji Hand](wuji-hand.md) | -| `external_project` | Scaffolding template — same shape as `genelab project new` output | see [Project New CLI](../cli/project-new.md) | +| [Inverted Pendulum](inverted-pendulum.md) | `examples/inverted_pendulum` | Minimal train/play loop, manager terms, RSL-RL integration. | +| [Unitree G1](unitree-g1.md) | `examples/unitree` | Humanoid locomotion, velocity commands, motion imitation. | +| [Showcase](showcase.md) | `examples/genelab_showcase` | Sensors, ray casts, contact, terrains, curricula, actuators, recording. | +| [Rubik's Cube](rubiks-cube.md) | `examples/genelab_examples` | Rigid-object composition and visual interaction. | +| [Wuji Hand](wuji-hand.md) | `examples/genelab_examples` | Articulated hand playback and asset packaging. | -## inverted_pendulum +## Install examples -Two PPO cart-pole tasks built on the same `ManagerBasedRlEnv` + rsl_rl stack as the -Unitree example, sized to fit in a laptop training budget: +```bash +uv pip install -e examples/inverted_pendulum +uv pip install -e examples/genelab_examples +uv run genelab list tasks +``` -- **`GeneLab-Inverted-Pendulum-v0`** — single inverted pole on a cart. -- **`GeneLab-Double-Inverted-Pendulum-v0`** — two stacked inverted poles on a cart. +Install Unitree only when needed: -Source at `examples/inverted_pendulum/`. Full walkthrough on the -[Inverted Pendulum](inverted-pendulum.md) page. - -## unitree - -Two PPO tasks on the Unitree G1 humanoid — velocity tracking and motion imitation — -ported from mjlab and adapted to Genesis. Same extension shape as `genelab_examples` -(entry point, `register()`, per-module registration files). Source at -`examples/unitree/`. Full walkthrough on the [Unitree G1](unitree-g1.md) page. - -## genelab_showcase - -Seven play-only tasks, one per GeneLab building block, each driving a real Franka or G1 -into a minimal `ManagerBasedRlEnv` and dumping a per-feature evidence file under -`logs/showcase//`. Designed for human visual / numerical verification rather than -training. Source at `examples/genelab_showcase/`. Full walkthrough on the -[Showcase](showcase.md) page. - -## genelab_examples - -The canonical in-tree extension, wiring two **play-only** Genesis demos that bypass -`ManagerBasedRlEnv`: - -- **`GeneLab-Rubiks-Play-v0`** — 27-cubie cube that dynamically welds and re-articulates - itself through Genesis's constraint API. See [Rubik's Cube](rubiks-cube.md). -- **`GeneLab-Wuji-Hand-Playback-v0`** — Wuji five-finger hand replaying a 20-DoF - `.npy` trajectory through `control_dofs_position`. See [Wuji Hand](wuji-hand.md). - -`pyproject.toml` declares the `genelab.extensions` entry point, so the extension is -discovered automatically once installed. The project's `pyproject.toml` also adds the -source directory to pytest's `pythonpath`, so tests can import from it without -installation. Source at `examples/genelab_examples/`. - -## external_project - -A minimal downstream-project template. `genelab project new` produces a project of the -same shape; this directory is kept in-tree as a reference for the scaffolding output. -It is **not** a runnable example task — instead see the -[Project New CLI](../cli/project-new.md) and [Extensions](../concepts/extensions.md) -pages for how to bootstrap your own downstream project from it. Source at -`examples/external_project/`. +```bash +uv pip install -e examples/unitree +``` ## See also -- [Quickstart](../getting-started/quickstart.md) -- [Extensions](../concepts/extensions.md) -- [Project New CLI](../cli/project-new.md) +- [Tutorial](../tutorial.md) +- [Build an Extension Project](../best-practices/extension-projects.md) diff --git a/docs/examples/overview.zh.md b/docs/examples/overview.zh.md index 60c99724..eb740309 100644 --- a/docs/examples/overview.zh.md +++ b/docs/examples/overview.zh.md @@ -1,62 +1,32 @@ # 示例 -仓库在 `examples/` 下提供数个参考扩展，同时也是 CLI 与注册表的集成测试。每个可运行的示例 -都有独立的文档页面，本页是入口。 +示例是扩展包，用来展示 GeneLab 能力，而不把项目专用代码放进 `src/genelab/`。 -## 能力一览 +## 能力地图 -| 示例 | 演示了什么 | 文档 | +| 示例 | 包 | 展示内容 | |---|---|---| -| `inverted_pendulum` | `ManagerBasedRlEnv` + PPO、`BodyVelocitySensor`、`RecordingCfg`、`MouseInteractionPlugin` | [倒立摆](inverted-pendulum.md) | -| `unitree` | G1 人形机器人速度跟踪 + 动作模仿、asset zoo 与 motion-clip 管线 | [Unitree G1](unitree-g1.md) | -| `genelab_showcase` | 七个单功能 play 任务（sensor、ray-cast、contact、terrain、curriculum、actuator、recording） | [Showcase](showcase.md) | -| `genelab_examples/rubiks` | 自定义 play runner、运行时生成 MJCF、Genesis 动态约束 API | [魔方](rubiks-cube.md) | -| `genelab_examples/wuji_hand` | MJCF 资产目录、关节名 → DOF 映射、固定轨迹重放 | [五指手](wuji-hand.md) | -| `external_project` | 脚手架模板，与 `genelab project new` 产出形状一致 | 见 [Project New CLI](../cli/project-new.md) | +| [倒立摆](inverted-pendulum.md) | `examples/inverted_pendulum` | 最小 train/play 闭环、manager term、RSL-RL 集成。 | +| [Unitree G1](unitree-g1.md) | `examples/unitree` | 人形机器人 locomotion、速度命令、动作模仿。 | +| [Showcase](showcase.md) | `examples/genelab_showcase` | 传感器、ray cast、接触、地形、课程、执行器、录制。 | +| [魔方](rubiks-cube.md) | `examples/genelab_examples` | 刚体组合与可视交互。 | +| [五指手](wuji-hand.md) | `examples/genelab_examples` | 灵巧手 playback 与资产打包。 | -## inverted_pendulum +## 安装示例 -两个 PPO cart-pole 任务，训练栈与 Unitree 示例相同（`ManagerBasedRlEnv` + rsl_rl），训练预算 -控制在单机能跑完的量级： +```bash +uv pip install -e examples/inverted_pendulum +uv pip install -e examples/genelab_examples +uv run genelab list tasks +``` -- **`GeneLab-Inverted-Pendulum-v0`** —— 小车 + 单杆倒立摆。 -- **`GeneLab-Double-Inverted-Pendulum-v0`** —— 小车 + 串联双杆倒立摆。 +只有需要 Unitree 时再安装： -源码位于 `examples/inverted_pendulum/`。完整走读见 [倒立摆](inverted-pendulum.md)。 +```bash +uv pip install -e examples/unitree +``` -## unitree +## 另见 -Unitree G1 人形机器人的两个 PPO 任务 —— 速度跟踪与动作模仿，从 mjlab 移植并适配到 Genesis。 -形态与 `genelab_examples` 相同（entry point、`register()`、按模块拆分的注册文件）。源码位于 -`examples/unitree/`。完整走读见 [Unitree G1](unitree-g1.md)。 - -## genelab_showcase - -7 个 play-only 任务，每一个对应一个 GeneLab 构建块，把真实 Franka 或 G1 投入一个最小的 -`ManagerBasedRlEnv`，并把对应特征的证据落到 `logs/showcase//`。面向人眼/数值核对， -不做训练。源码位于 `examples/genelab_showcase/`。完整走读见 [Showcase](showcase.md)。 - -## genelab_examples - -仓库内的标准扩展，接通两个**绕过 `ManagerBasedRlEnv`** 的 play-only Genesis 演示： - -- **`GeneLab-Rubiks-Play-v0`** —— 27 cubie 的魔方，通过 Genesis 约束 API 动态焊接 / 重新关节化 - 自己。见 [魔方](rubiks-cube.md)。 -- **`GeneLab-Wuji-Hand-Playback-v0`** —— Wuji 五指手用 `control_dofs_position` 重放 20 自由度的 - `.npy` 轨迹。见 [五指手](wuji-hand.md)。 - -`pyproject.toml` 声明了 `genelab.extensions` entry point，因此安装该包后会被自动发现； -项目 `pyproject.toml` 的 `pythonpath` 设置也让测试可以直接 import 而无需安装。源码位于 -`examples/genelab_examples/`。 - -## external_project - -下游项目最小模板。`genelab project new` 生成的内容与之结构一致，留在仓库里作为脚手架输出 -参考。它**不是**一个可运行的示例任务——使用方式见 [Project New CLI](../cli/project-new.md) -和 [扩展加载](../concepts/extensions.md)。源码位于 `examples/external_project/`。 - -## See also - -- [快速开始](../getting-started/quickstart.md) -- [扩展加载](../concepts/extensions.md) -- [Project New CLI](../cli/project-new.md) +- [教程](../tutorial.md) +- [构建扩展项目](../best-practices/extension-projects.md) diff --git a/docs/examples/rubiks-cube.en.md b/docs/examples/rubiks-cube.en.md index 21a7fb73..30f88db0 100644 --- a/docs/examples/rubiks-cube.en.md +++ b/docs/examples/rubiks-cube.en.md @@ -1,116 +1,35 @@ # Rubik's Cube -`examples/genelab_examples/src/genelab_examples/rubiks/` ships a play-only Genesis demo of a -force-driven 3×3 Rubik's cube. The task is **not** an RL task — it bypasses -`ManagerBasedRlEnv` and drives a Genesis `Scene` directly through a custom controller. It -exists primarily as an end-to-end example of (1) registering a non-RL task with the GeneLab -registry, (2) generating an MJCF asset on the fly, and (3) using Genesis's dynamic -constraint API to switch a rigid body between articulation modes. +The Rubik's Cube example is a play-only scene that demonstrates rigid-object composition and visual +interaction. ## Task -| Task id | Problem | -|---------|---------| -| `GeneLab-Rubiks-Play-v0` | A 27-cubie cube that dynamically welds into one rigid body, detects when it has enough angular velocity to "twist", and re-constrains itself into three slabs along the rotating axis. | - -## Installation - -```bash -uv sync --extra torch-cu128 -uv pip install -e examples/genelab_examples - -uv run genelab list tasks | grep Rubiks -# -> GeneLab-Rubiks-Play-v0 -``` - -Or run without installing by adding the example `src` to `PYTHONPATH`: - -```bash -PYTHONPATH=examples/genelab_examples/src \ - uv run genelab --import genelab_examples.tasks list tasks +```text +GeneLab-Rubiks-Play-v0 ``` ## Run ```bash -uv run genelab play GeneLab-Rubiks-Play-v0 --vis --steps 600 - -# Bigger cubie, smaller gap. -uv run genelab play GeneLab-Rubiks-Play-v0 --steps 5 \ - --env.robot.cubie_size 0.04 --env.robot.gap 0.002 - -# Statically weld every cubie (mode comparison). -uv run genelab play GeneLab-Rubiks-Play-v0 --env.robot.welded true --steps 5 - -# Interactive mouse drag (requires --vis). -uv run genelab play GeneLab-Rubiks-Play-v0 --vis --steps 0 \ - --env.interaction.interactive_force true +uv pip install -e examples/genelab_examples +uv run genelab play GeneLab-Rubiks-Play-v0 --vis --steps 500 ``` -`--steps 0` runs the viewer loop until you close the window. Without `--steps` the env -falls back to the value baked into `SimulationCfg.steps`. - -## What this example demonstrates - -| GeneLab capability | Where it appears | Concept doc | -|---|---|---| -| Task registration via `register_task` + `TaskCfg` | `tasks.py` | [Registry](../concepts/registry.md) | -| Robot registration via `register_robot` (MJCF written from a `RubiksCubeSpec`) | `robots.py`, `rubiks/assets.py:write_mjcf` | [Registry](../concepts/registry.md), [Asset zoo](../concepts/asset_zoo.md) | -| Configuration dataclass pattern (`RubiksEnvCfg` extends `ManagerBasedEnvCfg`) | `rubiks/config.py` | [Configs](../concepts/configs.md) | -| Custom `play()` runner that bypasses the manager pipeline | `envs.py:RubiksPlayEnv.play` | Contrasts with [Managers and MDP terms](../concepts/managers.md) | -| Genesis dynamic constraint API (weld / hinge / external torque) | `rubiks/sim.py:ForceDrivenCubeController` | [Scene and entities](../concepts/scene.md) | -| Genesis `MouseInteractionPlugin` for live force drags | `envs.py:RubiksPlayEnv.play` | Same plugin as the [inverted pendulum](inverted-pendulum.md#interactive-disturbance) | - -## Code walkthrough - -The package is split so each file owns one concern: - -- **`rubiks/config.py` (49 lines)** — `RubiksEnvCfg`, `RubiksRobotCfg`, `RubiksInteractionCfg`, - `ForceDrivenCubeConfig`. All overridable through dotted CLI flags - (`--env.robot.cubie_size 0.04`, `--env.interaction.mouse_spring 25`, ...). -- **`rubiks/assets.py` (249 lines)** — Pure geometry. `RubiksCubeSpec` parameterises cubie - size, sticker inset, friction, mass, etc. `iter_cubie_coords` and `exposed_faces` enumerate - the 27-cubie lattice. `write_mjcf` emits the runtime MJCF that Genesis loads. -- **`rubiks/sim.py` (1061 lines)** — The bulk of the example. Two controllers live here: - - `RubiksCubeController` — legacy kinematic turner (`set_qpos` per layer). Used when - `legacy_turn.enabled=true` for visual ground truth. - - `ForceDrivenCubeController` — the default. A state machine that: - 1. Welds all 27 cubies into one rigid body via `solver.add_weld_constraint`. - 2. Watches the welded body's angular velocity. When `|ω| > enter_ang_vel` along an - axis, it deletes the welds *for that axis only* and adds hinge-constraint pairs - so the cube splits into three slabs. - 3. Applies a small `joint_spring + joint_damping` torque through - `solver.apply_links_external_torque` to keep the side slabs at a quantised - quarter-turn rest position. - 4. When the slab angle settles below `exit_angle` *and* below `exit_ang_vel`, it - deletes the hinges and re-welds the slabs back together. - - This length is intrinsic: the cube has 27 cubies × 3 axes × 2 sides × multi-step - settle logic, and every state transition needs Genesis-constraint bookkeeping. The - file is not bloated; trimming it would remove behaviour, not noise. -- **`envs.py:RubiksPlayEnv`** — Builds the Genesis `Scene` (rigid options tuned for the - many-constraint regime: `max_dynamic_constraints=128`, `noslip_iterations=5`), loads the - generated MJCF, optionally attaches `MouseInteractionPlugin`, and runs the controller - step-by-step for `cfg.simulation.steps` ticks. **No `ManagerBasedRlEnv` involved** — this - task does not have observations, actions, or rewards. -- **`tasks.py`** — Registers `GeneLab-Rubiks-Play-v0` as a play-only task - (`trainable=False`). Invoking `genelab train` on it raises `NotImplementedError`. - -## Smoke test +Useful overrides: ```bash -PYTHONPATH=examples/genelab_examples/src \ - uv run genelab --import genelab_examples.tasks play GeneLab-Rubiks-Play-v0 --steps 5 +uv run genelab play GeneLab-Rubiks-Play-v0 --env.robot.cubie_size 0.04 +uv run genelab play GeneLab-Rubiks-Play-v0 --env.robot.welded true ``` -A 5-step play run is enough to validate asset generation, scene construction, and one -controller step. The first run compiles Genesis kernels and is slower. +## Shows + +- Non-RL play task registration. +- Scene composition from many rigid bodies. +- Config overrides for visual parameters. ## See also -- [Wuji Hand](wuji-hand.md) — the sibling play-only demo packaged in the same extension. -- [Registry](../concepts/registry.md) — how `register_task` / `register_robot` work. -- [Configs](../concepts/configs.md) — the dataclass override mechanism that - `--env.robot.welded true` rides on. -- [Scene and entities](../concepts/scene.md) — Genesis primitives used by the controller. -- [Inverted Pendulum](inverted-pendulum.md) — the other example that uses - `MouseInteractionPlugin`. +- [Scene and entities](../concepts/scene.md) +- [Configuration Reference](../reference/configuration.md) diff --git a/docs/examples/rubiks-cube.zh.md b/docs/examples/rubiks-cube.zh.md index 96a31122..102aa270 100644 --- a/docs/examples/rubiks-cube.zh.md +++ b/docs/examples/rubiks-cube.zh.md @@ -1,109 +1,34 @@ # 魔方 -`examples/genelab_examples/src/genelab_examples/rubiks/` 是一个 play-only 的 Genesis 演示， -模拟一个力驱动的 3×3 魔方。这个任务**不是** RL 任务——它绕开 `ManagerBasedRlEnv`， -通过一个自定义控制器直接驱动 Genesis `Scene`。它存在的意义在于演示：(1) 如何在 GeneLab 注册表 -中注册一个非 RL 任务；(2) 如何在运行时即时生成 MJCF 资产；(3) 如何用 Genesis 的动态约束 API -让一个刚体在不同关节形态之间切换。 +魔方示例是 play-only scene，展示刚体组合和可视交互。 -## 任务列表 +## 任务 -| Task id | 问题 | -|---------|------| -| `GeneLab-Rubiks-Play-v0` | 27 个 cubie 起初被动态焊成一整块刚体；一旦绕某根世界轴的角速度足够大，便沿该轴拆成三层平板，每层之间用 hinge 约束相连。 | - -## 安装 - -```bash -uv sync --extra torch-cu128 -uv pip install -e examples/genelab_examples - -uv run genelab list tasks | grep Rubiks -# -> GeneLab-Rubiks-Play-v0 -``` - -或者把 `src` 加到 `PYTHONPATH` 上免安装跑： - -```bash -PYTHONPATH=examples/genelab_examples/src \ - uv run genelab --import genelab_examples.tasks list tasks +```text +GeneLab-Rubiks-Play-v0 ``` ## 运行 ```bash -uv run genelab play GeneLab-Rubiks-Play-v0 --vis --steps 600 - -# 更大的 cubie、更小的间隙。 -uv run genelab play GeneLab-Rubiks-Play-v0 --steps 5 \ - --env.robot.cubie_size 0.04 --env.robot.gap 0.002 - -# 把每个 cubie 静态焊死（用于和默认模式对比）。 -uv run genelab play GeneLab-Rubiks-Play-v0 --env.robot.welded true --steps 5 - -# 鼠标拖拽交互（需要 --vis）。 -uv run genelab play GeneLab-Rubiks-Play-v0 --vis --steps 0 \ - --env.interaction.interactive_force true +uv pip install -e examples/genelab_examples +uv run genelab play GeneLab-Rubiks-Play-v0 --vis --steps 500 ``` -`--steps 0` 让 viewer 循环到窗口被关闭为止。不传 `--steps` 时使用 -`SimulationCfg.steps` 的默认值。 - -## 这个示例演示了什么 - -| GeneLab 能力 | 出现位置 | 概念文档 | -|---|---|---| -| 通过 `register_task` + `TaskCfg` 注册任务 | `tasks.py` | [Registry](../concepts/registry.md) | -| 通过 `register_robot` 注册机器人（MJCF 由 `RubiksCubeSpec` 即时写出） | `robots.py`、`rubiks/assets.py:write_mjcf` | [Registry](../concepts/registry.md)、[Asset zoo](../concepts/asset_zoo.md) | -| 配置 dataclass 模式（`RubiksEnvCfg` 继承 `ManagerBasedEnvCfg`） | `rubiks/config.py` | [Configs](../concepts/configs.md) | -| 自定义 `play()` runner，完全绕过 manager pipeline | `envs.py:RubiksPlayEnv.play` | 对比 [Manager 与 MDP term](../concepts/managers.md) | -| Genesis 动态约束 API（weld / hinge / 外加力矩） | `rubiks/sim.py:ForceDrivenCubeController` | [场景与实体](../concepts/scene.md) | -| Genesis `MouseInteractionPlugin` 实时拖拽 | `envs.py:RubiksPlayEnv.play` | 与[倒立摆](inverted-pendulum.md)使用同一插件 | - -## 代码走读 - -每个文件只负责一件事： - -- **`rubiks/config.py`（49 行）** —— `RubiksEnvCfg`、`RubiksRobotCfg`、 - `RubiksInteractionCfg`、`ForceDrivenCubeConfig`，全部可以通过点号 CLI flag 覆写 - （`--env.robot.cubie_size 0.04`、`--env.interaction.mouse_spring 25` 等）。 -- **`rubiks/assets.py`（249 行）** —— 纯几何。`RubiksCubeSpec` 参数化 cubie 尺寸、贴纸内缩、 - 摩擦、质量等。`iter_cubie_coords` 与 `exposed_faces` 枚举 27 个 cubie。`write_mjcf` - 生成 Genesis 加载的 MJCF。 -- **`rubiks/sim.py`（1061 行）** —— 示例的主要逻辑所在。这里有两个控制器： - - `RubiksCubeController` —— 历史遗留的运动学转动器，靠 `set_qpos` 直接改层位姿， - 在 `legacy_turn.enabled=true` 时作为视觉参照保留。 - - `ForceDrivenCubeController` —— 默认控制器，一个状态机： - 1. 用 `solver.add_weld_constraint` 把 27 个 cubie 焊成一整块。 - 2. 监测整块的角速度，发现 `|ω| > enter_ang_vel` 沿某条轴超过阈值时， - **只**沿该轴拆掉对应焊点，并加上 hinge-constraint 对，使 cube 拆成三层平板。 - 3. 通过 `solver.apply_links_external_torque` 施加一个 - `joint_spring + joint_damping` 弹簧+阻尼力矩，把边层拉到最近的四分之一圈位置。 - 4. 当边层角度小于 `exit_angle` **且**角速度小于 `exit_ang_vel`，就拆掉 hinge、 - 重新焊回。 - - 这部分代码长是因为本质复杂：27 个 cubie × 3 根轴 × 2 个方向 × 多步稳定逻辑， - 每次状态切换都要做 Genesis 约束簿记。删掉任何一段都会改变行为，并非"水分"。 -- **`envs.py:RubiksPlayEnv`** —— 构造 Genesis `Scene`（针对大量约束调优 rigid options： - `max_dynamic_constraints=128`、`noslip_iterations=5`），加载生成的 MJCF，按需挂上 - `MouseInteractionPlugin`，然后跑 `cfg.simulation.steps` 步控制器循环。**没有 - `ManagerBasedRlEnv` 参与**——这个任务没有 obs、action、reward。 -- **`tasks.py`** —— 把 `GeneLab-Rubiks-Play-v0` 注册为 play-only 任务 - （`trainable=False`），对它调用 `genelab train` 会直接抛 `NotImplementedError`。 - -## Smoke test +常用 override： ```bash -PYTHONPATH=examples/genelab_examples/src \ - uv run genelab --import genelab_examples.tasks play GeneLab-Rubiks-Play-v0 --steps 5 +uv run genelab play GeneLab-Rubiks-Play-v0 --env.robot.cubie_size 0.04 +uv run genelab play GeneLab-Rubiks-Play-v0 --env.robot.welded true ``` -5 步就足够验证资产生成、场景构造和一步控制器逻辑。首次运行会编译 Genesis kernel， -比较慢，属于正常现象。 +## 展示内容 + +- 非 RL play task 注册。 +- 多刚体 scene 组合。 +- 可视参数配置 override。 -## See also +## 另见 -- [五指手](wuji-hand.md) —— 同一个 extension 里另一个 play-only 示例。 -- [Registry](../concepts/registry.md) —— `register_task` / `register_robot` 的工作机制。 -- [Configs](../concepts/configs.md) —— `--env.robot.welded true` 走的 dataclass 覆写机制。 -- [场景与实体](../concepts/scene.md) —— 控制器用到的 Genesis 原语。 -- [倒立摆](inverted-pendulum.md) —— 另一个用 `MouseInteractionPlugin` 的示例。 +- [场景与实体](../concepts/scene.md) +- [配置参考](../reference/configuration.md) diff --git a/docs/examples/showcase.en.md b/docs/examples/showcase.en.md index b05e9b16..a2bb7406 100644 --- a/docs/examples/showcase.en.md +++ b/docs/examples/showcase.en.md @@ -1,198 +1,31 @@ # Showcase -`examples/genelab_showcase/` ships seven play-only tasks that exercise every M1–M4 -building block in a single command. Each task drops a real robot — Franka or -Unitree G1 — into a minimal `ManagerBasedRlEnv`, runs a scripted action loop, and -writes a per-feature evidence file under `logs/showcase//`. The intent is -human visual / numerical verification, not training. +`examples/genelab_showcase` contains focused tasks for individual framework capabilities. Use it +when you want a runnable example of one subsystem without reading a full robot task. -## Shared runner +## Capabilities -All seven tasks share the same `ShowcaseRunner` base class in -`examples/genelab_showcase/src/genelab_showcase/runner.py`. It owns the env lifecycle, -the fixed-length scripted-action loop, log-directory creation, and viewer ticking. -Subclasses override two hooks: `_scripted_action(env, step)` to return the per-step -action tensor, and `_dump(env, step, log_root)` to write whatever evidence is relevant -(PNG frames, histograms, tracking-error logs). The recording showcase is the one -exception — it leaves `_dump` untouched because the live plots and file writers run on -Genesis's recorder threads, driven entirely by the env-cfg `recordings=(...)` tuple. +| Area | Demonstrates | +|---|---| +| Sensors | Body velocity, IMU-like state, contact, ray-cast, terrain height. | +| Contact | Contact forces, air time, landing and slip metrics. | +| Terrain | Generated terrain grids and height scans. | +| Curriculum | Terrain-level progression. | +| Actuators | Switching actuator models and action behavior. | +| Recording | Live plots, files, and video-oriented recording configs. | -## Tasks - -| Task id | Robot | Feature exercised | -|---|---|---| -| `GeneLab-Sensors-Showcase-v0` | Franka | `CameraSensor` (RGB+depth PNG dump), `IMUSensor`, `FrameTransformerSensor` | -| `GeneLab-RayCast-Showcase-v0` | Franka | `RayCastSensor` × 3 patterns (`GridPattern`, `RingPattern`, `HemispherePattern`) | -| `GeneLab-Contact-Showcase-v0` | Unitree G1 | `ContactSensor` with `track_air_time=True` on both ankle-roll links | -| `GeneLab-Terrain-Showcase-v0` | Unitree G1 | `TerrainGeneratorCfg` 1×5 row of the five built-in sub-terrains | -| `GeneLab-Curriculum-Showcase-v0` | Unitree G1 | `terrain_levels_vel` curriculum on a 5×5 RandomRough grid | -| `GeneLab-Actuator-Showcase-v0` | Franka | `IdealPDActuator` arm (force-channel control); tracking-error dump | -| `GeneLab-Recording-Showcase-v0` | Franka | `genelab.recording` — live PyQt + MPL plots, NPZ + CSV dumps from an IMU | - -## Installation +## Run ```bash -uv sync --extra torch-cu128 # or whichever torch-* extra matches the host uv pip install -e examples/genelab_showcase - -uv run genelab list tasks | grep Showcase -``` - -The showcase extension auto-loads through the `genelab.extensions` entry point once -installed; no `--import` flag is needed. - -## Sensors - -```bash -uv run genelab play GeneLab-Sensors-Showcase-v0 --vis --steps 200 -``` - -Joint 1 is driven by a slow sine (±0.5 rad over a 4 s period) while the rest of the -arm holds the Menagerie home pose. Every 20 control steps the runner dumps: - -- `logs/showcase/sensors/rgb_.png` — wrist-camera RGB at 160×120. -- `logs/showcase/sensors/depth_.png` — wrist-camera depth as 16-bit gray scaled - to the camera's [near, far] = [0.02, 2.5] m clipping range. -- `logs/showcase/sensors/frame.log` — single-line record of IMU orientation, IMU - linear / angular acceleration, and end-effector + link 7 pose in the base frame. - -!!! warning "BatchRenderer is the only parallel-env backend" - `InteractiveSceneCfg.batch_render=True` is already set in the env config. The - sensor showcase therefore requires Linux x86-64 + CUDA + a Genesis build with - Madrona compiled in. On other platforms the env constructor will raise as soon - as Genesis allocates the renderer. - -**See also**: [`CameraSensor` / `IMUSensor` / `FrameTransformerSensor`](../concepts/sensors.md). - -## Ray-cast patterns - -```bash -uv run genelab play GeneLab-RayCast-Showcase-v0 --vis --steps 200 +uv run genelab list tasks +uv run genelab play --vis --steps 300 ``` -Three `RayCastSensor` instances co-exist on the Franka base — one `GridPattern` -(81 rays, 0.8×0.8 m), one `RingPattern` (32 horizontal × 1 vertical, full ±180°), and -one `HemispherePattern` (128 Fibonacci-distributed rays, 70° polar cap pointing down). -Every 20 steps the runner appends a three-line block to -`logs/showcase/raycast/distances.log`: - -``` -step=0020 - raycast_grid: rays=81 min=… mean=… max=… - raycast_ring: rays=32 min=… mean=… max=… - raycast_hemi: rays=128 min=… mean=… max=… -``` - -`max` clamps at the configured `max_distance`; rays that miss the ground plane (rare -when the pole axis points down) take that ceiling value. - -**See also**: [Ray-cast patterns](../concepts/sensors.md#ray-cast-patterns). - -## Contact and air time - -```bash -uv run genelab play GeneLab-Contact-Showcase-v0 --vis --steps 200 -``` - -The G1 settles into its default standing pose; both feet make ground contact almost -immediately. `logs/showcase/contact/air_time.log` records — every 20 steps — the -per-foot `found` flag, the running `current_contact_time` / `current_air_time`, and -the last completed `last_contact_time` / `last_air_time` snapshots. - -A clean reset cycle every 4 s (`episode_length_s=4`) lets the air-time counters tick -through several boundaries so the snapshot semantics are visible in the log. - -**See also**: [`ContactSensor`](../concepts/sensors.md#contactsensor). - -## Terrains - -```bash -uv run genelab play GeneLab-Terrain-Showcase-v0 --vis --steps 200 -``` - -A 1×5 sub-terrain row instantiates each built-in surface in a single scene: - -| Column | Sub-terrain | Visible feature | -|---|---|---| -| 0 | `FlatPatchCfg` | Flat reference patch at z=0 | -| 1 | `PyramidStairsCfg(step_width=0.4, step_height=-0.08)` | Concentric descending stairs | -| 2 | `RandomRoughCfg(min=-0.08, max=0.08)` | Uniform bumps | -| 3 | `SlopeCfg(slope=-0.25)` | Linear incline | -| 4 | `WaveCfg(num_waves=2, amplitude=0.08)` | Sinusoidal swell | - -G1 spawns at the row centre with default pose and is allowed to fall. A small -downward `GridPattern` ray-cast on the pelvis records the local heightfield response -into `logs/showcase/terrain/terrain.log`. - -**See also**: [Terrains](../concepts/terrains.md). - -## Curriculum - -```bash -uv run genelab play GeneLab-Curriculum-Showcase-v0 --vis --steps 400 -``` - -16 G1 instances spawn on a 5×5 `RandomRoughCfg` grid where row index controls -roughness amplitude. The runner teleports half the envs 1.5 m forward of spawn every -30 control steps to trip the `walked > distance_threshold` branch on auto-reset; the -remaining envs sit still and demote toward level 0. `logs/showcase/curriculum/levels.log` -appends a per-env level vector plus the row histogram every 30 steps. - -**See also**: [Managers and MDP terms](../concepts/managers.md), [Terrains](../concepts/terrains.md). - -## Actuators - -```bash -uv run genelab play GeneLab-Actuator-Showcase-v0 --vis --steps 200 -``` - -Franka is rebuilt with `IdealPDActuator` driving the seven arm joints — same -`stiffness=400`, `damping=80` numbers as the asset-zoo default, but routed through -`control_dofs_force` instead of Genesis's implicit PD. A 0.8-rad sine target is sent -to joint 1; `logs/showcase/actuators/tracking.log` records target vs actual position -and joint velocity every 20 steps so the tracking quality is directly comparable to -the sensors showcase (which uses `ImplicitPDActuator`). - -**See also**: [Actuators](../concepts/actuators.md). - -## Recording - -```bash -uv run genelab play GeneLab-Recording-Showcase-v0 --vis --steps 400 -``` - -Franka with one `IMUSensor` on the hand link plus three `RecordingCfg` sinks defined on -the env config: - -- IMU `lin_acc_b` → `PyQtPlotCfg` live window **and** `CSVFileCfg` dump to - `logs/showcase/recording/lin_acc.csv`. -- IMU `orientation` → `NPZFileCfg` per-episode dump to - `logs/showcase/recording/orientation_*.npz` (`save_on_reset=True`). -- A custom callable returning `env.robot_state.joint_pos[:, 0]` → `MPLPlotCfg` live - window. - -The scripted runner just waves joint 1 with a 0.5 rad / 4 s sine; the recorders run on -Genesis's own recorder threads, so no `_dump` hook is needed. - -**See also**: [Recording and plotting](../concepts/recording.md). - -!!! tip "Smoke-test budget" - Every showcase finishes in well under a minute at the default 200 / 400 steps on - a single env. Increase `--steps` only when accumulating longer log evidence; - none of the showcases benefit from large parallel-env counts. - -## Logs - -Each showcase writes to `logs/showcase//` with the slugs `sensors`, `raycast`, -`contact`, `terrain`, `curriculum`, `actuators`, `recording`. PNG / CSV / NPZ dumps land -alongside the text logs so a single directory is all that needs archiving when -comparing runs. +Use `genelab info ` to inspect the exact override paths for each showcase task. ## See also - [Sensors](../concepts/sensors.md) -- [Terrains](../concepts/terrains.md) -- [Actuators](../concepts/actuators.md) - [Recording and plotting](../concepts/recording.md) -- [Managers and MDP terms](../concepts/managers.md) -- [Scene and entities](../concepts/scene.md) +- [Terrains](../concepts/terrains.md) diff --git a/docs/examples/showcase.zh.md b/docs/examples/showcase.zh.md index 70895ce5..7567494c 100644 --- a/docs/examples/showcase.zh.md +++ b/docs/examples/showcase.zh.md @@ -1,181 +1,30 @@ -# 可视化 showcase +# Showcase -`examples/genelab_showcase/` 提供 7 个 play-only 任务，单条命令就能跑过 M1–M4 全部 -building block。每个任务把真实机器人 (Franka 或 Unitree G1) 投入一个最小的 -`ManagerBasedRlEnv`，跑一段脚本动作循环，把对应特征的证据写进 -`logs/showcase//`。目标是人眼 / 数值核对，不做训练。 +`examples/genelab_showcase` 包含面向单个框架能力的聚焦任务。当你想看某个子系统的可运行示例，而不是完整机器人任务时，用它。 -## 共用 runner +## 能力 -7 个任务都继承自 `examples/genelab_showcase/src/genelab_showcase/runner.py` 中的 -`ShowcaseRunner` 基类。它负责 env 生命周期、定长 scripted-action 循环、日志目录创建以及 -viewer tick。子类只需重载两个 hook：`_scripted_action(env, step)` 返回每步动作张量， -`_dump(env, step, log_root)` 写各自的证据文件（PNG 帧、直方图、跟踪误差日志……）。 -Recording showcase 是唯一不重载 `_dump` 的——它的实时图和文件写入全部跑在 Genesis 的 -recorder 线程上，由 env-cfg 的 `recordings=(...)` 元组直接驱动。 +| 区域 | 展示内容 | +|---|---| +| 传感器 | Body velocity、IMU-like state、contact、ray-cast、terrain height。 | +| 接触 | 接触力、air time、landing 和 slip metric。 | +| 地形 | 生成式地形网格和 height scan。 | +| Curriculum | terrain-level progression。 | +| 执行器 | 切换 actuator 模型和 action 行为。 | +| Recording | 实时曲线、文件、面向视频的 recording 配置。 | -## 任务列表 - -| Task id | Robot | 演示的功能 | -|---|---|---| -| `GeneLab-Sensors-Showcase-v0` | Franka | `CameraSensor` (RGB+depth PNG)、`IMUSensor`、`FrameTransformerSensor` | -| `GeneLab-RayCast-Showcase-v0` | Franka | `RayCastSensor` × 3 种 pattern (`GridPattern`、`RingPattern`、`HemispherePattern`) | -| `GeneLab-Contact-Showcase-v0` | Unitree G1 | `ContactSensor` 在双脚踝 link 上 `track_air_time=True` | -| `GeneLab-Terrain-Showcase-v0` | Unitree G1 | `TerrainGeneratorCfg` 1×5 行覆盖 5 种内置 sub-terrain | -| `GeneLab-Curriculum-Showcase-v0` | Unitree G1 | 5×5 RandomRough 网格上的 `terrain_levels_vel` 课程 | -| `GeneLab-Actuator-Showcase-v0` | Franka | `IdealPDActuator` 驱动机械臂 (force 通道)；记录跟踪误差 | -| `GeneLab-Recording-Showcase-v0` | Franka | `genelab.recording` —— 实时 PyQt + MPL 绘图，外加 NPZ + CSV 数据落盘 | - -## 安装 +## 运行 ```bash -uv sync --extra torch-cu128 # 选与本机匹配的 torch-* extra uv pip install -e examples/genelab_showcase - -uv run genelab list tasks | grep Showcase -``` - -装完后 showcase 扩展通过 `genelab.extensions` entry point 自动加载，不需要 `--import`。 - -## 传感器 - -```bash -uv run genelab play GeneLab-Sensors-Showcase-v0 --vis --steps 200 -``` - -Joint 1 跟一段慢正弦 (±0.5 rad，4 秒周期) 摆动，机械臂其余关节保持 Menagerie home 姿态。 -每 20 控制步 dump 一次： - -- `logs/showcase/sensors/rgb_.png` — 腕部相机 RGB，160×120。 -- `logs/showcase/sensors/depth_.png` — 腕部相机深度，16-bit 灰度，按相机 - [near, far] = [0.02, 2.5] m 的裁剪范围拉伸。 -- `logs/showcase/sensors/frame.log` — 单行日志，记录 IMU 姿态、IMU 线/角加速度，以及 - 末端和 link 7 在 base 坐标系下的位置。 - -!!! warning "BatchRenderer 是唯一支持并行 env 的渲染器" - env cfg 已经设 `InteractiveSceneCfg.batch_render=True`。所以这个 showcase 需要 - Linux x86-64 + CUDA + 编译了 Madrona 的 Genesis。其它平台上 env 构造会在 Genesis - 分配 renderer 时立刻报错。 - -**See also**：[`CameraSensor` / `IMUSensor` / `FrameTransformerSensor`](../concepts/sensors.md)。 - -## Ray-cast 形态 - -```bash -uv run genelab play GeneLab-RayCast-Showcase-v0 --vis --steps 200 -``` - -3 个 `RayCastSensor` 并列挂在 Franka 底座上——一个 `GridPattern` (81 条射线、0.8×0.8 m)、 -一个 `RingPattern` (32 横 × 1 纵，±180° 满圈)、一个 `HemispherePattern` (128 条 Fibonacci -分布射线、70° 极轴朝下)。每 20 步往 `logs/showcase/raycast/distances.log` 追加三行： - +uv run genelab list tasks +uv run genelab play --vis --steps 300 ``` -step=0020 - raycast_grid: rays=81 min=… mean=… max=… - raycast_ring: rays=32 min=… mean=… max=… - raycast_hemi: rays=128 min=… mean=… max=… -``` - -`max` 顶到所配的 `max_distance`；没打到地面的射线 (极轴朝下时极少) 也会取这个上限。 - -**See also**：[Ray-cast 形态](../concepts/sensors.md#ray-cast-patterns)。 - -## 接触与离地时间 - -```bash -uv run genelab play GeneLab-Contact-Showcase-v0 --vis --steps 200 -``` - -G1 在默认站立姿态稳定下来，双脚很快与地面接触。`logs/showcase/contact/air_time.log` 每 -20 步记录：双脚的 `found` 旗、运行中的 `current_contact_time` / `current_air_time`，以及 -最近一次完整的 `last_contact_time` / `last_air_time` 快照。 - -每 4 秒一次完整 reset (`episode_length_s=4`) 让 air-time 计数走过若干边界，snapshot -语义在日志里看得清。 - -**See also**：[`ContactSensor`](../concepts/sensors.md#contactsensor)。 - -## 地形 - -```bash -uv run genelab play GeneLab-Terrain-Showcase-v0 --vis --steps 200 -``` - -一行 5 列的 sub-terrain 把 5 种内置地形铺到同一个场景里： - -| 列 | Sub-terrain | 可见特征 | -|---|---|---| -| 0 | `FlatPatchCfg` | z=0 平面参考 | -| 1 | `PyramidStairsCfg(step_width=0.4, step_height=-0.08)` | 同心下降阶梯 | -| 2 | `RandomRoughCfg(min=-0.08, max=0.08)` | 均匀抖动 | -| 3 | `SlopeCfg(slope=-0.25)` | 线性斜坡 | -| 4 | `WaveCfg(num_waves=2, amplitude=0.08)` | 正弦起伏 | - -G1 在 row 中点投放，保持默认姿态自由下落。一个朝下的 `GridPattern` ray-cast 挂在 -pelvis 上，把局部 heightfield 响应写进 `logs/showcase/terrain/terrain.log`。 - -**See also**：[Terrains](../concepts/terrains.md)。 - -## 课程 - -```bash -uv run genelab play GeneLab-Curriculum-Showcase-v0 --vis --steps 400 -``` - -16 个 G1 实例分布在一个 5×5 `RandomRoughCfg` 网格上，行号控制粗糙幅度。runner 每 30 -控制步把一半 env 瞬移 1.5 m，触发 auto-reset 时 `walked > distance_threshold` 分支让它们 -升级；剩下的 env 原地不动，逐步降级到 level 0。`logs/showcase/curriculum/levels.log` -每 30 步追加 per-env level 向量加一个行级直方图。 - -**See also**：[Manager 与 MDP term](../concepts/managers.md)、[Terrains](../concepts/terrains.md)。 - -## 执行器 - -```bash -uv run genelab play GeneLab-Actuator-Showcase-v0 --vis --steps 200 -``` - -Franka 改用 `IdealPDActuator` 驱动 7 个机械臂关节——`stiffness=400` / `damping=80` 数值 -与 asset zoo 默认相同，但走的是 `control_dofs_force` 力通道，不再走 Genesis 内建 PD。 -对 joint 1 发 0.8 rad 正弦目标；`logs/showcase/actuators/tracking.log` 每 20 步记录目标 vs -实际位置和关节速度，可以和 sensors showcase (默认 ImplicitPDActuator) 直接对比跟踪质量。 - -**See also**：[Actuators](../concepts/actuators.md)。 - -## 数据录制 - -```bash -uv run genelab play GeneLab-Recording-Showcase-v0 --vis --steps 400 -``` - -Franka 的 hand link 上挂一个 `IMUSensor`，env cfg 上配三个 `RecordingCfg` 数据流： - -- IMU `lin_acc_b` → `PyQtPlotCfg` 实时窗口 **+** `CSVFileCfg` 写到 - `logs/showcase/recording/lin_acc.csv`。 -- IMU `orientation` → `NPZFileCfg` 按 episode 落盘到 - `logs/showcase/recording/orientation_*.npz`（`save_on_reset=True`）。 -- 一个自定义 callable，返回 `env.robot_state.joint_pos[:, 0]` → `MPLPlotCfg` 实时窗口。 - -脚本 runner 只做一件事：用 0.5 rad / 4 s 的正弦摆动 joint 1。recorder 由 Genesis 自己的 -recorder 线程驱动，所以这个 showcase 不需要重载 `_dump`。 - -**See also**：[数据录制与绘图](../concepts/recording.md)。 - -!!! tip "Smoke-test 预算" - 单 env 默认 200 / 400 步，每个 showcase 都在 1 分钟内跑完。需要积累更长日志证据再 - 加大 `--steps`；并行 env 数量对 showcase 没有好处。 - -## Log 输出 -每个 showcase 写到 `logs/showcase//`，slug 依次是 `sensors`、`raycast`、`contact`、 -`terrain`、`curriculum`、`actuators`、`recording`。PNG / CSV / NPZ dump 和文本日志同目录， -比对运行时归档一个目录即可。 +用 `genelab info ` 查看每个 showcase task 的准确 override 路径。 -## See also +## 另见 - [传感器](../concepts/sensors.md) -- [Terrains](../concepts/terrains.md) -- [Actuators](../concepts/actuators.md) -- [数据录制与绘图](../concepts/recording.md) -- [Manager 与 MDP term](../concepts/managers.md) -- [场景与实体](../concepts/scene.md) +- [数据录制与实时绘图](../concepts/recording.md) +- [地形](../concepts/terrains.md) diff --git a/docs/examples/unitree-g1.en.md b/docs/examples/unitree-g1.en.md index be8f2954..25a38b9f 100644 --- a/docs/examples/unitree-g1.en.md +++ b/docs/examples/unitree-g1.en.md @@ -1,138 +1,50 @@ # Unitree G1 -`examples/unitree/` ships two PPO tasks for the Unitree G1 humanoid on flat ground. -Both tasks are ports of the equivalent `mjlab` recipes adapted to Genesis. +The Unitree G1 example is the advanced robot path after the inverted pendulum tutorial. It shows how +GeneLab scales to humanoid locomotion and motion imitation. ## Tasks -| Task id | Problem | -|---------|---------| -| `Genelab-Velocity-Flat-Unitree-G1-v0` | Velocity-tracking baseline — follow a commanded body-frame twist. | -| `Genelab-Tracking-Flat-Unitree-G1-v0` | Motion imitation — track a recorded clip per-body (BeyondMimic style). | +| Task id | Shows | +|---|---| +| `Genelab-Velocity-Flat-Unitree-G1-v0` | Velocity tracking on flat ground. | +| `Genelab-Tracking-Flat-Unitree-G1-v0` | Reference motion tracking. | -## Installation - -Pick the `torch-*` extra that matches the host. +## Install ```bash -uv sync --extra torch-cu128 uv pip install -e examples/unitree - uv run genelab list tasks -# -> Genelab-Velocity-Flat-Unitree-G1-v0 -# -> Genelab-Tracking-Flat-Unitree-G1-v0 ``` ## Velocity tracking ```bash +uv run genelab play Genelab-Velocity-Flat-Unitree-G1-v0 --vis --steps 500 uv run genelab train Genelab-Velocity-Flat-Unitree-G1-v0 \ - --num-envs 4096 --max-iterations 1500 - -uv run genelab play Genelab-Velocity-Flat-Unitree-G1-v0 \ - --checkpoint logs/rsl_rl/g1_velocity_flat//model_1500.pt --vis + --num_envs 4096 \ + --max_iterations 1500 ``` -## Motion imitation - -The tracking task consumes an NPZ motion clip (keys: `joint_pos`, `joint_vel`, -`body_pos_w`, `body_quat_w`, `body_lin_vel_w`, `body_ang_vel_w` — the schema mjlab's -`csv_to_npz` emits). The default clip is the LAFAN1 retargeted `dance1_subject2` NPZ -fetched on first use by `genelab.asset_zoo.unitree_g1_motions.g1_lafan1_dance1_subject2` -and cached under `.cache/assets/g1_lafan1_dance1_subject2//`. No manual download -required — the example runs out of the box. +Replay: ```bash -# Replay the reference clip frame-by-frame (no policy; robot snaps to motion each step). -# The right way to visually verify the env wiring before training. -uv run python -m genelab_unitree.replay_motion - -# Train. -uv run genelab train Genelab-Tracking-Flat-Unitree-G1-v0 \ - --num-envs 4096 --max-iterations 30000 - -# Replay the trained policy. -uv run genelab play Genelab-Tracking-Flat-Unitree-G1-v0 \ - --agent trained \ - --checkpoint logs/rsl_rl/g1_tracking_flat//model_30000.pt +uv run genelab play Genelab-Velocity-Flat-Unitree-G1-v0 \ + --checkpoint logs/rsl_rl/g1_velocity_flat//model_1500.pt ``` -`--agent` accepts `zero`, `random`, or `trained`. Without `--agent`, play defaults to -`trained` when `--checkpoint` is set, otherwise `zero`. `zero` and `random` only reset -the robot to the clip's first frame and then apply zero / random torques — the robot -will fall, not follow the motion. Use the `replay_motion` script above for pure -reference playback. - -### Swapping the clip - -The default `motion_file` is wired in `tracking_env_cfg.unitree_g1_tracking_env_cfg`. -Replace the `g1_lafan1_dance1_subject2()` call with any NPZ matching the schema above, -and update `motion_body_order` / `motion_joint_order` on the -`MotionCommandCfg` if the new clip uses a different body / joint axis ordering. The -[`genelab-assets`](https://github.com/KraHsu/genelab-assets) repo ships -`unitree_g1/motions/scripts/convert.sh`, a recipe that drives mjlab's `csv_to_npz` -forward-kinematics replay for arbitrary G1-retargeted CSVs. - -The bundled clip inherits its upstream license (CC BY-NC-ND 4.0 — non-commercial -research use, attribution required); see `unitree_g1/motions/LICENSE.NOTICE` in the -assets repo. - -## Long training runs - -For multi-hour training (the tracking task defaults to `max_iterations=30_000` — -several wall-clock hours on 8×H200), set PyTorch's allocator to use growable segments -**before** launching training. This dramatically reduces fragmentation-driven -slowdown over time. +## Motion imitation ```bash -export PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True +uv run python -m genelab_unitree.replay_motion +uv run genelab train Genelab-Tracking-Flat-Unitree-G1-v0 \ + --num_envs 4096 \ + --max_iterations 30000 ``` -If training time drifts upward across iterations anyway, capture a profiler trace by -re-running with `GENELAB_PROFILE=1`. The trace lands under `logs/torch_profile/` and -can be loaded with `tensorboard --logdir logs/torch_profile`. Take one trace at the -start and another several hours in to compare cumulative time per section. Optional -env vars: `GENELAB_PROFILE_OUT`, `GENELAB_PROFILE_WAIT`, `GENELAB_PROFILE_WARMUP`, -`GENELAB_PROFILE_ACTIVE`, `GENELAB_PROFILE_REPEAT` (see `src/genelab/rl/_profiler.py`). -Only rank 0 emits a trace under distributed launches. - -!!! tip "Smoke-test budget" - A 5–10 iteration run with `--num-envs 64 --max-iterations 5` is enough to validate - the env wiring end-to-end. The reward signal stays noisy at that scale; - convergence on the velocity task needs the 1500-iteration budget above. - -## Notes - -- The G1 MJCF and STL meshes are fetched on first use via - `genelab.asset_zoo.unitree_g1.UnitreeG1Cfg` from the `genelab-assets` repository and - cached under `.cache/`. They originate from Unitree's `mujoco_menagerie` release; - refer to that repository for licensing details. -- The default motion NPZ (`dance1_subject2`) is similarly fetched from - `genelab-assets` on first use. The clip stores bodies and joints in mjlab's MJCF - DFS order; the env config permutes both axes into Genesis's traversal order via - `motion_body_order` and `motion_joint_order` on `MotionCommandCfg`. -- The motion-imitation task drops mjlab's adaptive-bin failure sampling; only - `start` and `uniform` sampling modes are wired up. Self-collision penalties are - omitted because GeneLab has no contact-pair sensor abstraction yet — the env still - penalises action rate and joint-limit excursions. -- `policy` and `critic` observation groups differ in the tracking task: the critic - receives the privileged per-body pose / orientation features that the actor does - not see. - -## Logs - -Both tasks write to `logs/rsl_rl///` exactly like the inverted -pendulum examples: - -- `params/env.json` and `params/agent.json` — frozen configs at run time. -- `model_.pt` — checkpoints saved every `save_interval` iterations. -- TensorBoard event files alongside the checkpoints. +The default clip is fetched through the asset zoo and cached locally. ## See also -- [Inverted Pendulum](inverted-pendulum.md) -- [Wuji Hand](wuji-hand.md) — smaller fixed-trajectory replay example -- [Actuators](../concepts/actuators.md) +- [RL experiments](../best-practices/rl-experiments.md) - [Asset zoo](../concepts/asset_zoo.md) -- [Sensors](../concepts/sensors.md) -- [Play and Train CLI](../cli/play-train.md) diff --git a/docs/examples/unitree-g1.zh.md b/docs/examples/unitree-g1.zh.md index 08bd4145..949e1316 100644 --- a/docs/examples/unitree-g1.zh.md +++ b/docs/examples/unitree-g1.zh.md @@ -1,125 +1,49 @@ # Unitree G1 -`examples/unitree/` 在平地上提供 2 个 PPO 任务，机器人是 Unitree G1 人形。两个任务都 -对标 `mjlab` 的同名 recipe，移植到 Genesis 上。 +Unitree G1 示例是倒立摆教程后的进阶机器人路径。它展示 GeneLab 如何扩展到人形机器人 locomotion 和动作模仿。 -## 任务列表 +## 任务 -| Task id | 问题 | -|---------|---------| -| `Genelab-Velocity-Flat-Unitree-G1-v0` | 速度跟踪基线——跟踪 body 系的命令 twist。 | -| `Genelab-Tracking-Flat-Unitree-G1-v0` | 动作模仿——按 body 跟踪录制 clip (BeyondMimic 风格)。 | +| Task id | 展示内容 | +|---|---| +| `Genelab-Velocity-Flat-Unitree-G1-v0` | 平地速度跟踪。 | +| `Genelab-Tracking-Flat-Unitree-G1-v0` | 参考动作跟踪。 | ## 安装 -`torch-*` extra 按本机硬件挑。 - ```bash -uv sync --extra torch-cu128 uv pip install -e examples/unitree - uv run genelab list tasks -# -> Genelab-Velocity-Flat-Unitree-G1-v0 -# -> Genelab-Tracking-Flat-Unitree-G1-v0 ``` ## 速度跟踪 ```bash +uv run genelab play Genelab-Velocity-Flat-Unitree-G1-v0 --vis --steps 500 uv run genelab train Genelab-Velocity-Flat-Unitree-G1-v0 \ - --num-envs 4096 --max-iterations 1500 - -uv run genelab play Genelab-Velocity-Flat-Unitree-G1-v0 \ - --checkpoint logs/rsl_rl/g1_velocity_flat//model_1500.pt --vis + --num_envs 4096 \ + --max_iterations 1500 ``` -## 动作模仿 - -tracking 任务消费 NPZ motion clip (key：`joint_pos`、`joint_vel`、`body_pos_w`、 -`body_quat_w`、`body_lin_vel_w`、`body_ang_vel_w` —— mjlab `csv_to_npz` 产出的 schema)。 -默认 clip 是 LAFAN1 retargeted 的 `dance1_subject2` NPZ，首次使用时由 -`genelab.asset_zoo.unitree_g1_motions.g1_lafan1_dance1_subject2` 拉取并缓存到 -`.cache/assets/g1_lafan1_dance1_subject2//`，无需手动下载，example 开箱即用。 +回放： ```bash -# 逐帧回放参考 clip (不接策略；机器人每步贴到 motion 当前帧)。 -# 训练前验证 env 接线是否正确的标准方式。 -uv run python -m genelab_unitree.replay_motion - -# 训练。 -uv run genelab train Genelab-Tracking-Flat-Unitree-G1-v0 \ - --num-envs 4096 --max-iterations 30000 - -# 回放训练好的策略。 -uv run genelab play Genelab-Tracking-Flat-Unitree-G1-v0 \ - --agent trained \ - --checkpoint logs/rsl_rl/g1_tracking_flat//model_30000.pt +uv run genelab play Genelab-Velocity-Flat-Unitree-G1-v0 \ + --checkpoint logs/rsl_rl/g1_velocity_flat//model_1500.pt ``` -`--agent` 取 `zero`、`random` 或 `trained`。不写 `--agent` 时，`--checkpoint` 存在则默认 -`trained`，否则 `zero`。`zero` 和 `random` 都只在 reset 时把机器人摆到 clip 第 0 帧、 -之后施加零 / 随机力矩，机器人只会倒下，不会跟随动作。要做无策略的参考回放，用上面的 -`replay_motion` 脚本。 - -### 替换 clip - -默认 `motion_file` 在 `tracking_env_cfg.unitree_g1_tracking_env_cfg` 里硬编码。把 -`g1_lafan1_dance1_subject2()` 替换成符合上面 schema 的任意 NPZ 路径即可；若新 clip 的 -body / joint 轴顺序不同，记得同步更新 `MotionCommandCfg` 的 `motion_body_order` / -`motion_joint_order`。[`genelab-assets`](https://github.com/KraHsu/genelab-assets) 仓库 -里附带 `unitree_g1/motions/scripts/convert.sh`，把任意 G1 retargeted CSV 喂给 mjlab 的 -`csv_to_npz` 做一次正运动学回放。 - -捆绑 clip 继承上游许可 (CC BY-NC-ND 4.0 —— 仅供非商业研究，需署名)；详情见 assets 仓库 -里的 `unitree_g1/motions/LICENSE.NOTICE`。 - -## 长时间训练 - -多小时的训练 (tracking 任务默认 `max_iterations=30_000`，在 8×H200 上要数小时墙钟)， -启动训练**前**把 PyTorch 内存分配器切到可增长 segment，能显著减少碎片化导致的逐步降速。 +## 动作模仿 ```bash -export PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True +uv run python -m genelab_unitree.replay_motion +uv run genelab train Genelab-Tracking-Flat-Unitree-G1-v0 \ + --num_envs 4096 \ + --max_iterations 30000 ``` -如果训练时长仍然在 iteration 间漂移，重跑时加 `GENELAB_PROFILE=1` 捕一段 profiler。 -trace 落在 `logs/torch_profile/`，可用 `tensorboard --logdir logs/torch_profile` 看。 -在训练初期和数小时后各抓一段，对比 per-section 累计时间。可选 env var： -`GENELAB_PROFILE_OUT`、`GENELAB_PROFILE_WAIT`、`GENELAB_PROFILE_WARMUP`、 -`GENELAB_PROFILE_ACTIVE`、`GENELAB_PROFILE_REPEAT` (见 `src/genelab/rl/_profiler.py`)。 -分布式启动时只 rank 0 写 trace。 - -!!! tip "Smoke-test 预算" - `--num-envs 64 --max-iterations 5` 跑 5–10 个 iteration 就能验证 env 接线 OK。这种 - 规模下 reward 信号噪声大；速度任务真正收敛需要上面 1500 iteration 的预算。 - -## 备注 - -- G1 的 MJCF 与 STL mesh 在首次使用时由 `genelab.asset_zoo.unitree_g1.UnitreeG1Cfg` - 从 `genelab-assets` 仓库拉取并缓存到 `.cache/`。源自 Unitree 的 `mujoco_menagerie` - release，license 详见上游仓库。 -- 默认 motion NPZ (`dance1_subject2`) 同样从 `genelab-assets` 首次使用时拉取。clip - 内的 body / joint 轴按 mjlab MJCF 的 DFS 顺序存储；env config 通过 `MotionCommandCfg` - 的 `motion_body_order` / `motion_joint_order` 把两轴重排到 Genesis 的遍历顺序。 -- 动作模仿任务去掉了 mjlab 的 adaptive-bin 失败采样；只接了 `start` 与 `uniform` 两种 - 采样模式。自碰惩罚也省略，因为 GeneLab 还没有接触对 sensor 抽象——env 仍然惩罚动作 - 变化率与关节越界。 -- tracking 任务的 `policy` / `critic` observation group 不同：critic 看得到特权的 per-body - 位姿 / 朝向特征，actor 看不到。 - -## Log 输出 - -两个任务都写到 `logs/rsl_rl///`，与倒立摆任务一致： - -- `params/env.json` 与 `params/agent.json` — 运行时冻结下来的 cfg。 -- `model_.pt` — 每 `save_interval` 个 iteration 一份 checkpoint。 -- TensorBoard event 文件与 checkpoint 同目录。 +默认 clip 通过 asset zoo 获取并缓存到本地。 -## See also +## 另见 -- [倒立摆](inverted-pendulum.md) -- [五指手](wuji-hand.md) —— 更小规模的固定轨迹回放示例 -- [Actuators](../concepts/actuators.md) -- [Asset zoo](../concepts/asset_zoo.md) -- [传感器](../concepts/sensors.md) -- [play 与 train CLI](../cli/play-train.md) +- [运行 RL 实验](../best-practices/rl-experiments.md) +- [资产库](../concepts/asset_zoo.md) diff --git a/docs/examples/wuji-hand.en.md b/docs/examples/wuji-hand.en.md index a3c50b26..2359d1d5 100644 --- a/docs/examples/wuji-hand.en.md +++ b/docs/examples/wuji-hand.en.md @@ -1,109 +1,35 @@ # Wuji Hand -`examples/genelab_examples/src/genelab_examples/wuji_hand/` ships a play-only Genesis -demo of the Wuji five-finger dexterous hand replaying a fixed `.npy` trajectory. Like the -[Rubik's cube example](rubiks-cube.md) it is **not** an RL task — it builds a Genesis -`Scene` directly and PD-controls each joint toward the next trajectory frame. The point -is to show the minimum amount of GeneLab + Genesis glue needed to drive a robot from a -recorded motion clip. +The Wuji Hand example is a play-only articulated-hand playback task. It demonstrates asset packaging +and scripted joint trajectory playback. ## Task -| Task id | Problem | -|---------|---------| -| `GeneLab-Wuji-Hand-Playback-v0` | Load the right-side Wuji MJCF, map each `right_fingerN_jointM` to its Genesis DOF, then replay a 20-column trajectory frame-by-frame through `control_dofs_position`. | - -## Installation - -```bash -uv sync --extra torch-cu128 -uv pip install -e examples/genelab_examples - -uv run genelab list tasks | grep Wuji -# -> GeneLab-Wuji-Hand-Playback-v0 -``` - -Or run without installing: - -```bash -PYTHONPATH=examples/genelab_examples/src \ - uv run genelab --import genelab_examples.tasks list tasks +```text +GeneLab-Wuji-Hand-Playback-v0 ``` ## Run ```bash -# Default trajectory: a 20-DoF wave gesture bundled in the package. -uv run genelab play GeneLab-Wuji-Hand-Playback-v0 --vis --steps 600 - -# Disable periodic hard resets (let the trajectory run uninterrupted). -uv run genelab play GeneLab-Wuji-Hand-Playback-v0 --vis --env.reset_interval 0 - -# Swap to the left-side MJCF (only `right` is bundled by default). -uv run genelab play GeneLab-Wuji-Hand-Playback-v0 --vis --env.robot.side left +uv pip install -e examples/genelab_examples +uv run genelab play GeneLab-Wuji-Hand-Playback-v0 --vis --steps 500 ``` -`reset_interval` is the number of Genesis steps between periodic teleports back to the -joint-zero pose; `0` disables the teleport. The default `wave.npy` ships next to the -package source. - -## What this example demonstrates - -| GeneLab capability | Where it appears | Concept doc | -|---|---|---| -| Task registration of a play-only task | `tasks.py` | [Registry](../concepts/registry.md) | -| Robot registration backed by an on-disk MJCF asset directory | `robots.py`, `wuji_hand/assets.py:resolve_mjcf_path` | [Registry](../concepts/registry.md), [Asset zoo](../concepts/asset_zoo.md) | -| Config dataclass with nested `SimulationCfg` and a default `Path` field | `wuji_hand/config.py` | [Configs](../concepts/configs.md) | -| Joint-name → Genesis DOF-index mapping for trajectory replay | `wuji_hand/sim.py:build_joint_mapping`, `JointMapping` dataclass | n/a (example-specific helper) | -| Trajectory loading from `.npy` + per-joint clamping to DOF limits | `wuji_hand/assets.py:load_trajectory`, `wuji_hand/sim.py:trajectory_target` | Same pattern as the [Unitree G1 motion clip](unitree-g1.md#motion-imitation) | -| Per-joint PD gain and force-range setup at runtime | `wuji_hand/sim.py:apply_wuji_gains` | [Actuators](../concepts/actuators.md) — contrast with the configured-actuator pattern | - -## Code walkthrough - -The package is small (≈400 lines total) and each file owns one concern: - -- **`wuji_hand/config.py` (24 lines)** — `WujiEnvCfg` (extends `ManagerBasedEnvCfg`) and - `WujiRobotCfg`. Default `dt=0.01`, `steps=0` (infinite viewer loop), `reset_interval=500`, - `side="right"`. All overridable via dotted flags - (`--env.simulation.dt 0.005`, `--env.reset_interval 0`, `--env.robot.side left`). -- **`wuji_hand/assets.py` (74 lines)** — File-path resolution and trajectory loading. - - `wuji_joint_names(side)` returns the 20 joint names in the order the trajectory's - columns expect (`{side}_fingerN_jointM` for `N=1..5, M=1..4`). - - `resolve_mjcf_path(desc_dir, side)` tries three candidate paths so users can drop in - their own MJCF layout. - - `load_trajectory(path)` validates shape (2D, ≥20 columns) and returns `float32`. -- **`wuji_hand/sim.py` (280 lines)** — The playback loop. Worth reading in order: - 1. `WujiHandRunConfig` — flat run-time config with `__post_init__` validators. - 2. `build_joint_mapping(entity, side)` — for each trajectory column, look up the matching - Genesis joint by name and record its local DOF index. Missing joints are skipped - with a warning rather than failing. - 3. `apply_wuji_gains` — sets `kp=0.8`, `kv=0.04` on every mapped DOF and copies the - MJCF's declared force range onto Genesis's runtime range (only when both bounds are - finite — MJCFs with `forcerange="0 0"` or unset values are passed through). - 4. `run_wuji_hand(config)` — initialises Genesis, builds the scene (with rigid options - tuned for self-collision on a high-DoF hand), maps joints, then in the main loop - calls `entity.control_dofs_position(target, mapping.dof_indices)` once per Genesis - step using the clamped trajectory frame. -- **`envs.py:WujiHandPlaybackEnv.play`** — Thin wrapper that copies `WujiEnvCfg` fields - into a `WujiHandRunConfig` and calls `run_wuji_hand`. **No `ManagerBasedRlEnv`** — like - the Rubik's example, this task has no obs/action/reward graph. -- **`tasks.py`** — Registers `GeneLab-Wuji-Hand-Playback-v0` with `trainable=False`. - -## Smoke test +Useful overrides: ```bash -PYTHONPATH=examples/genelab_examples/src \ - uv run genelab --import genelab_examples.tasks play GeneLab-Wuji-Hand-Playback-v0 --steps 5 +uv run genelab play GeneLab-Wuji-Hand-Playback-v0 --env.reset_interval 0 +uv run genelab play GeneLab-Wuji-Hand-Playback-v0 --env.robot.side left ``` -A 5-step run validates MJCF resolution, joint mapping, and one trajectory frame. First -launch triggers Genesis kernel compilation. +## Shows + +- Articulated hand asset packaging. +- Playback task structure without an RL runner. +- Configurable reset/playback behavior. ## See also -- [Rubik's Cube](rubiks-cube.md) — sibling play-only demo packaged in the same extension. -- [Unitree G1](unitree-g1.md) — the larger motion-replay example (LAFAN1 NPZ, full - humanoid). -- [Registry](../concepts/registry.md) — how the task and robot are wired in. -- [Configs](../concepts/configs.md) — dataclass override mechanism behind the - `--env.robot.side left` flag. +- [Scene and entities](../concepts/scene.md) +- [Asset zoo](../concepts/asset_zoo.md) diff --git a/docs/examples/wuji-hand.zh.md b/docs/examples/wuji-hand.zh.md index 0ceb237c..71affb90 100644 --- a/docs/examples/wuji-hand.zh.md +++ b/docs/examples/wuji-hand.zh.md @@ -1,101 +1,34 @@ # 五指手 -`examples/genelab_examples/src/genelab_examples/wuji_hand/` 是一个 play-only 的 Genesis 演示， -让 Wuji 五指灵巧手回放一段固定的 `.npy` 轨迹。和[魔方示例](rubiks-cube.md)一样，它**不是** -RL 任务——直接构造 Genesis `Scene`，每个关节用 PD 控制器跟踪下一帧轨迹目标。它的意义在于 -展示用最少的 GeneLab + Genesis 胶水��码就能让机器人按预录动作走完。 +五指手示例是 play-only 灵巧手 playback task，展示资产打包和脚本化关节轨迹回放。 -## 任务列表 +## 任务 -| Task id | 问题 | -|---------|------| -| `GeneLab-Wuji-Hand-Playback-v0` | 加载右手 MJCF，将每个 `right_fingerN_jointM` 与 Genesis DOF 对齐，然后通过 `control_dofs_position` 逐帧重放一段 20 列轨迹。 | - -## 安装 - -```bash -uv sync --extra torch-cu128 -uv pip install -e examples/genelab_examples - -uv run genelab list tasks | grep Wuji -# -> GeneLab-Wuji-Hand-Playback-v0 -``` - -或免安装运行： - -```bash -PYTHONPATH=examples/genelab_examples/src \ - uv run genelab --import genelab_examples.tasks list tasks +```text +GeneLab-Wuji-Hand-Playback-v0 ``` ## 运行 ```bash -# 默认轨迹：包内自带的 20 自由度挥手动作。 -uv run genelab play GeneLab-Wuji-Hand-Playback-v0 --vis --steps 600 - -# 关闭周期性硬复位，让轨迹连续播放。 -uv run genelab play GeneLab-Wuji-Hand-Playback-v0 --vis --env.reset_interval 0 - -# 切到左手 MJCF（默认只带 right）。 -uv run genelab play GeneLab-Wuji-Hand-Playback-v0 --vis --env.robot.side left +uv pip install -e examples/genelab_examples +uv run genelab play GeneLab-Wuji-Hand-Playback-v0 --vis --steps 500 ``` -`reset_interval` 是每隔多少 Genesis 步把手重置回零位姿态的间隔，`0` 表示关闭。默认的 -`wave.npy` 与包源码同目录。 - -## 这个示例演示了什么 - -| GeneLab 能力 | 出现位置 | 概念文档 | -|---|---|---| -| 注册 play-only 任务 | `tasks.py` | [Registry](../concepts/registry.md) | -| 基于磁盘 MJCF 资产目录注册机器人 | `robots.py`、`wuji_hand/assets.py:resolve_mjcf_path` | [Registry](../concepts/registry.md)、[Asset zoo](../concepts/asset_zoo.md) | -| 嵌套 `SimulationCfg`、默认 `Path` 字段的 config dataclass | `wuji_hand/config.py` | [Configs](../concepts/configs.md) | -| 关节名 → Genesis DOF 下标的映射，用于轨迹重放 | `wuji_hand/sim.py:build_joint_mapping`、`JointMapping` dataclass | 无（示例自带的辅助器） | -| 从 `.npy` 加载轨迹 + 按 DOF 上下限逐关节裁剪 | `wuji_hand/assets.py:load_trajectory`、`wuji_hand/sim.py:trajectory_target` | 与 [Unitree G1](unitree-g1.md) 的动作模仿模式相同 | -| 运行时按需设置 PD 增益和力矩范围 | `wuji_hand/sim.py:apply_wuji_gains` | [Actuators](../concepts/actuators.md) —— 与"在配置里直接声明 actuator"模式形成对比 | - -## 代码走读 - -整个包约 400 行，每个文件只做一件事： - -- **`wuji_hand/config.py`（24 行）** —— `WujiEnvCfg`（继承 `ManagerBasedEnvCfg`）和 - `WujiRobotCfg`。默认 `dt=0.01`、`steps=0`（viewer 死循环）、`reset_interval=500`、 - `side="right"`。全部可以通过点号 flag 覆写（`--env.simulation.dt 0.005`、 - `--env.reset_interval 0`、`--env.robot.side left`）。 -- **`wuji_hand/assets.py`（74 行）** —— 路径解析与轨迹加载。 - - `wuji_joint_names(side)` 按轨迹列顺序返回 20 个关节名 - （`{side}_fingerN_jointM`，`N=1..5, M=1..4`）。 - - `resolve_mjcf_path(desc_dir, side)` 顺序尝试三种候选路径，方便用户接入自己的 MJCF 目录布局。 - - `load_trajectory(path)` 校验形状（2D、≥20 列），返回 `float32`。 -- **`wuji_hand/sim.py`（280 行）** —— 重放循环。建议按顺序读： - 1. `WujiHandRunConfig` —— 扁平的运行时配置，`__post_init__` 做基本校验。 - 2. `build_joint_mapping(entity, side)` —— 对每个轨迹列，按名字查找匹配的 Genesis - 关节并记录其本地 DOF 下标。找不到的关节只打 warning 跳过，不直接报错。 - 3. `apply_wuji_gains` —— 给映射上的每个 DOF 设 `kp=0.8`、`kv=0.04`，并把 MJCF - 声明的力矩范围复制到 Genesis 的运行时 force range（仅当上下限都有限时； - `forcerange="0 0"` 或未设值的 MJCF 不动）。 - 4. `run_wuji_hand(config)` —— 初始化 Genesis，按高自由度手的自碰撞需求调好 rigid - options，建场景，做完关节映射后进入主循环：用裁剪过的轨迹帧每步调一次 - `entity.control_dofs_position(target, mapping.dof_indices)`。 -- **`envs.py:WujiHandPlaybackEnv.play`** —— 薄包装，把 `WujiEnvCfg` 的字段拷到一个 - `WujiHandRunConfig`，再调 `run_wuji_hand`。**没有 `ManagerBasedRlEnv`**——和魔方示例 - 一样，这个任务没有 obs/action/reward 图。 -- **`tasks.py`** —— 把 `GeneLab-Wuji-Hand-Playback-v0` 注册为 `trainable=False` 的任务。 - -## Smoke test +常用 override： ```bash -PYTHONPATH=examples/genelab_examples/src \ - uv run genelab --import genelab_examples.tasks play GeneLab-Wuji-Hand-Playback-v0 --steps 5 +uv run genelab play GeneLab-Wuji-Hand-Playback-v0 --env.reset_interval 0 +uv run genelab play GeneLab-Wuji-Hand-Playback-v0 --env.robot.side left ``` -5 步运行足以验证 MJCF 解析、关节映射和一帧轨迹的下发。首次启动会触发 Genesis -kernel 编译。 +## 展示内容 + +- 灵巧手 articulation 资产打包。 +- 不带 RL runner 的 playback task 结构。 +- 可配置 reset/playback 行为。 -## See also +## 另见 -- [魔方](rubiks-cube.md) —— 同一个 extension 里的另一个 play-only 示例。 -- [Unitree G1](unitree-g1.md) —— 更大规模的动作回放示例（LAFAN1 NPZ、整人形机器人）。 -- [Registry](../concepts/registry.md) —— task 和 robot 是怎样接入注册表的。 -- [Configs](../concepts/configs.md) —— `--env.robot.side left` 走的 dataclass 覆写机制。 +- [场景与实体](../concepts/scene.md) +- [资产库](../concepts/asset_zoo.md) diff --git a/docs/getting-started/installation.en.md b/docs/getting-started/installation.en.md index 364ae1df..359806f4 100644 --- a/docs/getting-started/installation.en.md +++ b/docs/getting-started/installation.en.md @@ -1,77 +1,77 @@ # Installation -GeneLab requires **Python 3.12 or newer** and uses [`uv`](https://docs.astral.sh/uv/) for -dependency management. +This page is the focused installation checklist. For the full learning path, start with the +[tutorial](../tutorial.md). -## 1. Sync the environment +## Requirements + +| Requirement | Version | +|---|---| +| Python | 3.12 or newer | +| Dependency manager | `uv` | +| Simulator backend | `genesis-world>=0.4.7` | +| PyTorch | `torch>=2.8.0` through one `torch-*` extra | + +## Sync the environment From the repository root: ```bash -uv sync -uv run genelab --help +uv sync --extra torch-cpu +uv run genelab --version ``` -`uv sync` creates the project virtual environment, installs GeneLab from this checkout, and -installs the dependencies pinned by `uv.lock`. `uv run ...` runs commands inside that -environment. A bare `genelab` command works only after `.venv` is activated or GeneLab is -installed into the active Python environment. - -## 2. Pick exactly one PyTorch extra - -The `torch-*` extras are **mutually exclusive**: +Choose exactly one PyTorch extra: -| Extra | Hardware target | -|-------|----------------| +| Extra | Target | +|---|---| | `torch-cpu` | CPU-only or non-NVIDIA development machines. | -| `torch-cu126` | NVIDIA, CUDA 12.6 driver. | -| `torch-cu128` | NVIDIA, CUDA 12.8 driver. | -| `torch-cu130` | NVIDIA, CUDA 13.0 driver. | +| `torch-cu126` | NVIDIA driver compatible with CUDA 12.6 wheels. | +| `torch-cu128` | NVIDIA driver compatible with CUDA 12.8 wheels. | +| `torch-cu130` | NVIDIA driver compatible with CUDA 13.0 wheels. | + +Refresh an existing torch install: ```bash -uv sync --extra torch-cpu # one of the above +uv sync --reinstall-package torch --extra torch-cu128 ``` -!!! warning "PyTorch version requirement" - Genesis requires `torch>=2.8.0` — older builds emit a `'torch<2.8.0' is not supported` - warning at import time and may break Genesis runtime assumptions. All `torch-*` extras pin - `torch>=2.8.0`, so `uv sync` will pull a compatible wheel automatically. PyTorch only - publishes 2.8+ wheels on the `cpu`, `cu126`, `cu128`, and `cu130` indices; older CUDA - flavours (`cu118` / `cu121` / `cu124`) are intentionally not offered as extras. An older - `torch` already in the environment can be refreshed with - `uv sync --reinstall-package torch --extra torch-cuXXX`. - -Run `nvidia-smi` to confirm the driver version when unsure which CUDA build to use. - -## 3. Initialize project-local caches - -Genesis, Quadrants, and Matplotlib all require a writable cache directory. The CLI sets up both -the directory layout and the matching environment variables: +## Create local caches ```bash uv run genelab cache ``` -This sets `XDG_CACHE_HOME` and `MPLCONFIGDIR` under `.cache/`, so the simulator never writes to -the user's home directory. +The command creates writable project-local cache directories and points `XDG_CACHE_HOME` and +`MPLCONFIGDIR` at `.cache/`. -## 4. Verify +## Verify imports ```bash uv run python -c "import genelab; print(genelab.__version__)" -uv run python -c "from genelab.lab import ManagerBasedEnvCfg; print(ManagerBasedEnvCfg.__name__)" +uv run python -c "from genelab.lab import TaskCfg; print(TaskCfg.__name__)" +uv run python -c "import torch; print(torch.__version__, torch.version.cuda)" uv run python -c "import genesis; print(genesis.__version__)" -uv run pytest -uv run ruff check -uv run pyright ``` -Verify the selected PyTorch build: +## Install example extensions + +The core package is a framework; tasks are registered by extensions. ```bash -uv run python -c "import torch; print(torch.__version__, torch.version.cuda)" +uv pip install -e examples/inverted_pendulum +uv pip install -e examples/genelab_examples +uv run genelab list tasks +``` + +Install Unitree only when you want the larger humanoid examples: + +```bash +uv pip install -e examples/unitree ``` ## See also -- [Quickstart](quickstart.md) +- [Tutorial](../tutorial.md) +- [Debug Common Failures](../best-practices/debugging.md) +- [Build an Extension Project](../best-practices/extension-projects.md) diff --git a/docs/getting-started/installation.zh.md b/docs/getting-started/installation.zh.md index 4048b438..bea19509 100644 --- a/docs/getting-started/installation.zh.md +++ b/docs/getting-started/installation.zh.md @@ -1,71 +1,75 @@ # 安装 -GeneLab 需要 **Python 3.12 或更新版本**，使用 [`uv`](https://docs.astral.sh/uv/) 管理依赖。 +这页是专门的安装检查清单。完整学习路径请从[教程](../tutorial.md)开始。 -## 1. 同步环境 +## 要求 -在仓库根目录运行： +| 要求 | 版本 | +|---|---| +| Python | 3.12 或更新 | +| 依赖管理 | `uv` | +| 仿真后端 | `genesis-world>=0.4.7` | +| PyTorch | 通过一个 `torch-*` extra 安装 `torch>=2.8.0` | + +## 同步环境 + +在仓库根目录： ```bash -uv sync -uv run genelab --help +uv sync --extra torch-cpu +uv run genelab --version ``` -`uv sync` 会创建项目虚拟环境，从当前 checkout 安装 GeneLab，并安装 `uv.lock` 锁定的依赖。 -`uv run ...` 在该环境中运行命令。裸 `genelab` 命令只有在 `.venv` 已激活、或 GeneLab 已安装 -到当前 Python 环境后才可用。 +只能选择一个 PyTorch extra： -## 2. 挑选一个 PyTorch extra +| Extra | 目标 | +|---|---| +| `torch-cpu` | CPU-only 或非 NVIDIA 开发机器。 | +| `torch-cu126` | 兼容 CUDA 12.6 wheel 的 NVIDIA 驱动。 | +| `torch-cu128` | 兼容 CUDA 12.8 wheel 的 NVIDIA 驱动。 | +| `torch-cu130` | 兼容 CUDA 13.0 wheel 的 NVIDIA 驱动。 | -`torch-*` extras **互斥**： - -| Extra | 目标硬件 | -|-------|---------| -| `torch-cpu` | 仅 CPU 或非 NVIDIA 开发机器。 | -| `torch-cu126` | NVIDIA，CUDA 12.6 驱动。 | -| `torch-cu128` | NVIDIA，CUDA 12.8 驱动。 | -| `torch-cu130` | NVIDIA，CUDA 13.0 驱动。 | +刷新已有 torch： ```bash -uv sync --extra torch-cpu # 从上表挑一个 +uv sync --reinstall-package torch --extra torch-cu128 ``` -!!! warning "PyTorch 版本要求" - Genesis 要求 `torch>=2.8.0`，旧版本在导入时会报 `'torch<2.8.0' is not supported`，并可能 - 破坏 Genesis 的运行时假设。所有 `torch-*` extras 都固定 `torch>=2.8.0`，`uv sync` 会自动 - 拉取兼容 wheel。PyTorch 只在 `cpu`、`cu126`、`cu128`、`cu130` 索引提供 2.8+ wheel，因此 - 较旧的 CUDA flavour（`cu118` / `cu121` / `cu124`）不再以 extra 形式提供。环境里已存在 - 旧 torch 时，可用 `uv sync --reinstall-package torch --extra torch-cuXXX` 刷新。 - -不确定使用哪个 CUDA build 时，用 `nvidia-smi` 查看驱动版本。 - -## 3. 初始化项目本地缓存 - -Genesis、Quadrants、Matplotlib 都需要可写缓存目录。CLI 一条命令完成目录与环境变量配置： +## 创建本地缓存 ```bash uv run genelab cache ``` -它会把 `XDG_CACHE_HOME` 和 `MPLCONFIGDIR` 指向 `.cache/`，避免仿真器写到用户主目录。 +该命令创建项目本地可写缓存目录，并把 `XDG_CACHE_HOME` 和 `MPLCONFIGDIR` 指向 `.cache/`。 -## 4. 验证 +## 验证导入 ```bash uv run python -c "import genelab; print(genelab.__version__)" -uv run python -c "from genelab.lab import ManagerBasedEnvCfg; print(ManagerBasedEnvCfg.__name__)" +uv run python -c "from genelab.lab import TaskCfg; print(TaskCfg.__name__)" +uv run python -c "import torch; print(torch.__version__, torch.version.cuda)" uv run python -c "import genesis; print(genesis.__version__)" -uv run pytest -uv run ruff check -uv run pyright ``` -验证当前 PyTorch build： +## 安装示例扩展 + +核心包是框架；任务由扩展注册。 ```bash -uv run python -c "import torch; print(torch.__version__, torch.version.cuda)" +uv pip install -e examples/inverted_pendulum +uv pip install -e examples/genelab_examples +uv run genelab list tasks +``` + +只有需要更大的人形机器人示例时再安装 Unitree： + +```bash +uv pip install -e examples/unitree ``` -## See also +## 另见 -- [快速开始](quickstart.md) +- [教程](../tutorial.md) +- [调试常见问题](../best-practices/debugging.md) +- [构建扩展项目](../best-practices/extension-projects.md) diff --git a/docs/getting-started/quickstart.en.md b/docs/getting-started/quickstart.en.md index cabe4a0a..55ed3e6a 100644 --- a/docs/getting-started/quickstart.en.md +++ b/docs/getting-started/quickstart.en.md @@ -1,11 +1,8 @@ # Quickstart -## 1. Listing registered content +Use this page when the environment is already installed and you want the shortest command path. -The core `genelab` package ships empty registries — robots, environments, and tasks are -contributed by extension packages. The example extension under `examples/genelab_examples/` is -discovered automatically because its `pyproject.toml` declares a `genelab.extensions` entry -point. +## 1. List registered content ```bash uv run genelab list robots @@ -13,129 +10,55 @@ uv run genelab list envs uv run genelab list tasks ``` -An empty registry means no extension has been imported. Install one with `uv pip install`, or -pass `--import MODULE` on the command line. - -## 2. Running a task - -```bash -uv run genelab play -``` - -The CLI resolves `` against the `TASKS` registry, constructs its `TaskCfg`, applies any -command-line overrides, and runs the rollout in the configured Genesis backend. - -The three scene shortcuts: +If a task is missing, load its extension explicitly: ```bash -uv run genelab play --vis # enable visualization -uv run genelab play --steps 500 # cap the episode length -uv run genelab play --gpu 1 # pin to a single GPU +PYTHONPATH=examples/inverted_pendulum/src \ + uv run genelab --import genelab_inverted_pendulum.tasks list tasks ``` -Arbitrary overrides use the dotted `--a.b.c VALUE` syntax — strings are coerced to the type -declared on the target dataclass field: +## 2. Inspect a task ```bash -uv run genelab play \ - --env.simulation.dt 0.005 \ - --env.actions.scale 0.5 +uv run genelab info GeneLab-Inverted-Pendulum-v0 ``` -## 3. Training +Copy override paths from the printed config tree instead of guessing. -When a task ships an RL runner (`rsl_rl` and similar), training is launched with: +## 3. Play ```bash -uv run genelab train -uv run genelab train --gpus 4 # multi-GPU via torchrun -uv run genelab train --checkpoint path/to/model.pt +uv run genelab play GeneLab-Inverted-Pendulum-v0 --steps 64 +uv run genelab play GeneLab-Inverted-Pendulum-v0 --vis --steps 500 +uv run genelab play GeneLab-Inverted-Pendulum-v0 --agent random --steps 128 ``` -!!! warning "Distributed training" - `--gpus N` transparently dispatches through `torchrun` for distributed training; the target - task's runner must be `torchrun`-compatible. - -## 4. Scaffolding a project - -A standalone downstream package is generated with: - -```bash -uv run genelab project new my_robot_project -``` - -This produces `config.py`, `robots.py`, `envs.py`, `tasks.py`, and a `pyproject.toml` that -declares a `genelab.extensions` entry point — the same shape as the bundled example extension. - -## 5. Advanced: end-to-end RL on Unitree G1 { #unitree-g1 } - -The repository ships a production-grade RL example under `examples/unitree/` — two PPO tasks on -the Unitree G1 humanoid (velocity tracking and motion imitation), ported from mjlab and adapted -to Genesis. The following walks through install, training, and checkpoint replay in one -sitting. - -### 5.1 Install the extension - -The Unitree extension pulls in vendored MJCF + meshes (~19 MB). Pick the `torch-*` extra that -matches your hardware — see *Installation* for the full extras table. +## 4. Train ```bash -uv sync --extra torch-cu128 -uv pip install -e examples/unitree - -uv run genelab list tasks -# -> Genelab-Velocity-Flat-Unitree-G1-v0 -# -> Genelab-Tracking-Flat-Unitree-G1-v0 +uv run genelab train GeneLab-Inverted-Pendulum-v0 \ + --num_envs 64 \ + --max_iterations 2 ``` -### 5.2 Velocity tracking PPO - -The velocity-tracking task asks the G1 to follow a commanded body-frame twist. Train, then -replay the resulting checkpoint: +Longer run: ```bash -uv run genelab train Genelab-Velocity-Flat-Unitree-G1-v0 \ - --num-envs 4096 --max-iterations 1500 - -uv run genelab play Genelab-Velocity-Flat-Unitree-G1-v0 \ - --checkpoint logs/rsl_rl/g1_velocity_flat//model_1500.pt +uv run genelab train GeneLab-Inverted-Pendulum-v0 \ + --num_envs 4096 \ + --max_iterations 300 ``` -`--checkpoint` makes `play` route through the RL runner with `--agent trained` by default. - -### 5.3 Motion imitation - -The tracking task imitates a recorded clip per-body (BeyondMimic-style). The default clip -is the LAFAN1 retargeted `dance1_subject2` NPZ, fetched on first use from `genelab-assets` -via `genelab.asset_zoo.unitree_g1_motions.g1_lafan1_dance1_subject2()` and cached under -`.cache/`. No manual download required. +## 5. Scaffold a project ```bash -# Replay the reference clip frame-by-frame (no policy; robot snaps to motion each step). -uv run python -m genelab_unitree.replay_motion - -# Train. -uv run genelab train Genelab-Tracking-Flat-Unitree-G1-v0 \ - --num-envs 4096 --max-iterations 30000 - -# Replay trained policy. -uv run genelab play Genelab-Tracking-Flat-Unitree-G1-v0 \ - --agent trained \ - --checkpoint logs/rsl_rl/g1_tracking_flat//model_30000.pt +uv run genelab project new my_robot_project +uv pip install -e my_robot_project +uv run genelab list tasks ``` -### 5.4 `--agent` modes - -`--agent` selects the policy source for `play`: - -| Value | Policy source | -|-------|--------------| -| `zero` | Constant zero action — basic sanity check; robot resets then falls under gravity. | -| `random` | Uniform-random action sampling — visible perturbation around the reference pose. | -| `trained` | Load from `--checkpoint`. This is the default whenever `--checkpoint` is set. | - ## See also -- [Configs](../concepts/configs.md) -- [Extensions](../concepts/extensions.md) -- [Examples](../examples/overview.md) +- [Tutorial](../tutorial.md) +- [CLI Reference](../reference/cli.md) +- [Run RL Experiments](../best-practices/rl-experiments.md) diff --git a/docs/getting-started/quickstart.zh.md b/docs/getting-started/quickstart.zh.md index 69d581eb..58bc7ba8 100644 --- a/docs/getting-started/quickstart.zh.md +++ b/docs/getting-started/quickstart.zh.md @@ -1,10 +1,8 @@ # 快速开始 -## 1. 列出已注册内容 +环境已经安装好、只想走最短命令路径时，用这一页。 -核心 `genelab` 包自带空注册表 —— 机器人、环境、任务都由扩展包贡献。仓库内 -`examples/genelab_examples/` 通过 `pyproject.toml` 中的 `genelab.extensions` entry point -被自动发现。 +## 1. 列出已注册内容 ```bash uv run genelab list robots @@ -12,126 +10,55 @@ uv run genelab list envs uv run genelab list tasks ``` -注册表为空意味着没有扩展被导入。可通过 `uv pip install` 安装一个，或在命令行加 -`--import MODULE`。 - -## 2. 运行任务 - -```bash -uv run genelab play -``` - -CLI 会在 `TASKS` 注册表中解析 ``，构造其 `TaskCfg`，应用命令行 override，然后在 -Genesis 后端跑 rollout。 - -三个场景短标志： - -```bash -uv run genelab play --vis # 启用可视化 -uv run genelab play --steps 500 # 限制单 episode 步数 -uv run genelab play --gpu 1 # 锁定到单张 GPU -``` - -任意 override 使用 `--a.b.c VALUE` 点路径写法，字符串值按目标 dataclass 字段的类型注解自动 -转换： +如果缺少某个任务，显式加载对应扩展： ```bash -uv run genelab play \ - --env.simulation.dt 0.005 \ - --env.actions.scale 0.5 +PYTHONPATH=examples/inverted_pendulum/src \ + uv run genelab --import genelab_inverted_pendulum.tasks list tasks ``` -## 3. 训练 - -任务自带 RL runner（`rsl_rl` 等）时，以如下方式启动训练： +## 2. 查看任务 ```bash -uv run genelab train -uv run genelab train --gpus 4 # 多 GPU 通过 torchrun 启动 -uv run genelab train --checkpoint path/to/model.pt +uv run genelab info GeneLab-Inverted-Pendulum-v0 ``` -!!! warning "分布式训练" - `--gpus N` 会透明走 `torchrun` 启动分布式训练，前提是任务自身的 runner 支持 `torchrun`。 - -## 4. 新建下游项目 +从打印出的配置树复制 override 路径，不要手猜。 -生成一个独立的下游扩展包： +## 3. Play ```bash -uv run genelab project new my_robot_project +uv run genelab play GeneLab-Inverted-Pendulum-v0 --steps 64 +uv run genelab play GeneLab-Inverted-Pendulum-v0 --vis --steps 500 +uv run genelab play GeneLab-Inverted-Pendulum-v0 --agent random --steps 128 ``` -会生成 `config.py`、`robots.py`、`envs.py`、`tasks.py` 以及一份声明了 `genelab.extensions` -entry point 的 `pyproject.toml` —— 形态与仓库自带示例扩展一致。 - -## 5. 进阶：在 Unitree G1 上跑通完整 RL 流程 { #unitree-g1 } - -仓库在 `examples/unitree/` 下提供一个生产级 RL 示例：两个 Unitree G1 人形机器人的 PPO 任务 -（速度跟踪与动作模仿），从 mjlab 移植并适配到 Genesis。下面给出从安装到训练再到 checkpoint -回放的全部命令，无需跳页阅读。 - -### 5.1 安装扩展 - -unitree 扩展随包附带 MJCF 与网格资源（约 19 MB）。`torch-*` extra 需按硬件挑选 —— -对照表见 *安装*。 +## 4. Train ```bash -uv sync --extra torch-cu128 -uv pip install -e examples/unitree - -uv run genelab list tasks -# -> Genelab-Velocity-Flat-Unitree-G1-v0 -# -> Genelab-Tracking-Flat-Unitree-G1-v0 +uv run genelab train GeneLab-Inverted-Pendulum-v0 \ + --num_envs 64 \ + --max_iterations 2 ``` -### 5.2 速度跟踪 PPO - -速度跟踪任务要求 G1 跟随给定的 body-frame twist。训练后再回放训练好的 checkpoint： +更长训练： ```bash -uv run genelab train Genelab-Velocity-Flat-Unitree-G1-v0 \ - --num-envs 4096 --max-iterations 1500 - -uv run genelab play Genelab-Velocity-Flat-Unitree-G1-v0 \ - --checkpoint logs/rsl_rl/g1_velocity_flat//model_1500.pt +uv run genelab train GeneLab-Inverted-Pendulum-v0 \ + --num_envs 4096 \ + --max_iterations 300 ``` -`--checkpoint` 会让 `play` 走 RL runner，并把 `--agent` 默认设为 `trained`。 - -### 5.3 动作模仿 - -动作模仿任务按 body 跟踪一段录制的动作片段（BeyondMimic 风格）。默认 clip 是 LAFAN1 -retargeted 的 `dance1_subject2` NPZ，首次使用时由 -`genelab.asset_zoo.unitree_g1_motions.g1_lafan1_dance1_subject2()` 从 `genelab-assets` -拉取并缓存到 `.cache/`，无需手动下载。 +## 5. 生成项目 ```bash -# 逐帧回放参考 clip (不接策略；机器人每步贴到 motion 当前帧)。 -uv run python -m genelab_unitree.replay_motion - -# 训练。 -uv run genelab train Genelab-Tracking-Flat-Unitree-G1-v0 \ - --num-envs 4096 --max-iterations 30000 - -# 回放训练好的策略。 -uv run genelab play Genelab-Tracking-Flat-Unitree-G1-v0 \ - --agent trained \ - --checkpoint logs/rsl_rl/g1_tracking_flat//model_30000.pt +uv run genelab project new my_robot_project +uv pip install -e my_robot_project +uv run genelab list tasks ``` -### 5.4 `--agent` 三种模式 - -`--agent` 决定 `play` 的策略来源： - -| 取值 | 策略来源 | -|------|---------| -| `zero` | 恒零动作 —— 基本健康检查；机器人 reset 后在重力下倒掉。 | -| `random` | 均匀随机动作 —— 在参考姿态附近显示可见扰动。 | -| `trained` | 从 `--checkpoint` 加载。设置 `--checkpoint` 时即为默认。 | - -## See also +## 另见 -- [配置系统](../concepts/configs.md) -- [扩展加载](../concepts/extensions.md) -- [示例](../examples/overview.md) +- [教程](../tutorial.md) +- [CLI 参考](../reference/cli.md) +- [运行 RL 实验](../best-practices/rl-experiments.md) diff --git a/docs/index.en.md b/docs/index.en.md index 452b6227..20352bc5 100644 --- a/docs/index.en.md +++ b/docs/index.en.md @@ -17,7 +17,7 @@ package layout and CLI for downstream robotics projects.

-[Get started](getting-started/installation.md){ .md-button .md-button--primary } +[Start the tutorial](tutorial.md){ .md-button .md-button--primary } [View on GitHub](https://github.com/KraHsu/GeneLab){ .md-button }
@@ -45,21 +45,21 @@ package layout and CLI for downstream robotics projects.
-- 01 :material-download:{ .lg .middle } **Install** +- 01 :material-school-outline:{ .lg .middle } **Tutorial** --- - Set up `uv`, pick a `torch-*` extra, and verify the CLI. + Build the first robot experiment, then continue to Unitree G1. - [Installation →](getting-started/installation.md) + [Tutorial →](tutorial.md) -- 02 :material-rocket-launch-outline:{ .lg .middle } **Run** +- 02 :material-download:{ .lg .middle } **Install** --- - List registered tasks and play one in a Genesis viewer. + Set up `uv`, pick a `torch-*` extra, and verify the CLI. - [Quickstart →](getting-started/quickstart.md) + [Installation →](getting-started/installation.md) - 03 :material-console-line:{ .lg .middle } **CLI** @@ -79,13 +79,13 @@ package layout and CLI for downstream robotics projects.
-## Modules +## Abilities -
    -
  • genelab.registryRegistries, registration helpers, extension loading.
    • -
  • genelab.configsReusable dataclass configs (ManagerBasedEnvCfg, TaskCfg).
    • -
  • genelab.labPublic facade for registry + manager-based environment primitives.
    • -
  • genelab.cliTyper + Rich dispatcher; play / train / info / project new.
    • -
  • genelab.envs / robots / tasksCore registry helper namespaces.
    • -
  • genelab.actuator / scene / sensor / terrainsExtension namespaces for robotics research code.
    • +
      +
    • DiscoverRegister and inspect robots, environments, and tasks from installed or explicit extensions.
      • +
    • ComposeBuild Genesis scenes from robot assets, rigid objects, terrains, sensors, and recordings.
      • +
    • Shape MDPsDefine actions, commands, observations, rewards, terminations, events, curricula, and metrics as named terms.
      • +
    • RunUse one CLI for headless rollouts, viewer playback, config overrides, and task introspection.
      • +
    • TrainLaunch RSL-RL PPO runs, checkpoint replay, profiler traces, and multi-GPU training.
      • +
    • ExtendKeep downstream robot projects in independent Python packages with entry-point discovery.
    diff --git a/docs/index.zh.md b/docs/index.zh.md index c90d7740..9210c908 100644 --- a/docs/index.zh.md +++ b/docs/index.zh.md @@ -17,7 +17,7 @@ hide:

    -[开始使用](getting-started/installation.md){ .md-button .md-button--primary } +[开始教程](tutorial.md){ .md-button .md-button--primary } [查看 GitHub](https://github.com/KraHsu/GeneLab){ .md-button }
    @@ -45,21 +45,21 @@ hide:
    -- 01 :material-download:{ .lg .middle } **安装** +- 01 :material-school-outline:{ .lg .middle } **教程** --- - 准备 `uv`，挑选一个 `torch-*` extra，并验证 CLI。 + 构建第一个机器人实验，然后继续到 Unitree G1。 - [安装 →](getting-started/installation.md) + [教程 →](tutorial.md) -- 02 :material-rocket-launch-outline:{ .lg .middle } **运行** +- 02 :material-download:{ .lg .middle } **安装** --- - 列出已注册任务，并在 Genesis viewer 中 play 一个。 + 准备 `uv`，挑选一个 `torch-*` extra，并验证 CLI。 - [快速开始 →](getting-started/quickstart.md) + [安装 →](getting-started/installation.md) - 03 :material-console-line:{ .lg .middle } **CLI** @@ -79,13 +79,13 @@ hide:
    -## 模块速览 +## 能力速览 -
      -
    • genelab.registry注册表、注册 helper、扩展加载。
      • -
    • genelab.configs可复用的 dataclass 配置（ManagerBasedEnvCfg、TaskCfg）。
      • -
    • genelab.lab注册表与 manager-based 环境原语的公共 facade。
      • -
    • genelab.cliTyper + Rich 调度器；play / train / info / project new。
      • -
    • genelab.envs / robots / tasks核心注册 helper 命名空间。
      • -
    • genelab.actuator / scene / sensor / terrains面向机器人研究代码的扩展命名空间。
      • +
        +
      • 发现从已安装或显式导入的扩展中注册并检查机器人、环境和任务。
        • +
      • 组合用机器人资产、刚体、地形、传感器和录制配置构建 Genesis scene。
        • +
      • 塑造 MDP把 action、command、observation、reward、termination、event、curriculum、metric 定义成命名 term。
        • +
      • 运行用统一 CLI 做 headless rollout、viewer 回放、配置 override 和任务检查。
        • +
      • 训练启动 RSL-RL PPO、checkpoint 回放、profiler trace 和多 GPU 训练。
        • +
      • 扩展把下游机器人项目放在独立 Python 包中，并通过 entry point 自动发现。
      diff --git a/docs/reference/cli.en.md b/docs/reference/cli.en.md new file mode 100644 index 00000000..08b82eff --- /dev/null +++ b/docs/reference/cli.en.md @@ -0,0 +1,76 @@ +# CLI Reference + +GeneLab exposes one console script: + +```bash +uv run genelab [global options] [command options] +``` + +## Global options + +| Option | Description | +|---|---| +| `--version` | Print GeneLab version and exit. | +| `--import MODULE` | Import an extension module before dispatch. Repeatable. | +| `--no-entry-points` | Skip installed `genelab.extensions` entry points. Useful for reproducible debugging with explicit imports. | +| `--help` | Show help. | + +## Commands + +| Command | Purpose | +|---|---| +| `cache` | Create project-local cache directories. | +| `list robots` | List registered robots. | +| `list envs` | List registered environments. | +| `list tasks` | List registered tasks. | +| `info NAME` | Show detail for a registered task, env, or robot. | +| `play TASK ...` | Run a registered task. | +| `train TASK ...` | Train a task with a supported runner. | +| `prof open [DIR]` | Open TensorBoard for profiler traces. | +| `project new NAME` | Scaffold an external GeneLab extension project. | + +## Runtime flags + +These flags are accepted after the task id for `play` and `train`. + +| Flag | Description | +|---|---| +| `--vis`, `-v` | Enable the Genesis viewer. | +| `--gpu` | Use the Genesis GPU backend. | +| `--steps N` | In `play`, rollout steps. In `train`, short form for `--max_iterations N`. | +| `--dt X` | Override simulator timestep. | +| `--a.b.c VALUE` | Apply a dotted config override. | + +## Runner flags + +| Flag | Command | Description | +|---|---|---| +| `--num_envs N` | play/train | Total number of envs. In distributed train, divided by `--gpus`. | +| `--num_envs_per_gpu N` | play/train | Per-rank env count. Mutually exclusive with `--num_envs`. | +| `--agent zero|random|trained` | play | Select policy source. | +| `--checkpoint PATH` | play/train | Load a checkpoint. In play, defaults agent to `trained`. | +| `--seed N` | train | Override env and agent seed. | +| `--log_dir PATH` | train | Use a resolved log directory. | +| `--max_iterations N` | train | Override PPO iteration count. | +| `--gpus N` | train | Relaunch under `torchrun` with `N` local ranks. | + +## Profiler flags + +| Flag | Environment fallback | Default | +|---|---|---| +| `--prof` | `GENELAB_PROFILE=1` | off | +| `--prof-out PATH` | `GENELAB_PROFILE_OUT` | `logs/torch_profile` | +| `--prof-wait N` | `GENELAB_PROFILE_WAIT` | `10` | +| `--prof-warmup N` | `GENELAB_PROFILE_WARMUP` | `5` | +| `--prof-active N` | `GENELAB_PROFILE_ACTIVE` | `10` | +| `--prof-repeat N` | `GENELAB_PROFILE_REPEAT` | `2` | +| `--prof-record-shapes` | `GENELAB_PROFILE_RECORD_SHAPES=1` | off | +| `--prof-with-stack` | `GENELAB_PROFILE_WITH_STACK=1` | off | + +## Completion and interactive recovery + +Typer provides `--install-completion` and `--show-completion`. Completion loads installed entry +points but cannot see temporary `--import MODULE` extensions. + +When stdin is a TTY, GeneLab can prompt for missing task ids, unknown names, invalid `--agent` +values, and unknown override paths. In scripts and CI, errors are raised directly. diff --git a/docs/reference/cli.zh.md b/docs/reference/cli.zh.md new file mode 100644 index 00000000..a570234c --- /dev/null +++ b/docs/reference/cli.zh.md @@ -0,0 +1,76 @@ +# CLI 参考 + +GeneLab 暴露一个命令行脚本： + +```bash +uv run genelab [全局选项] <命令> [命令选项] +``` + +## 全局选项 + +| 选项 | 说明 | +|---|---| +| `--version` | 打印 GeneLab 版本并退出。 | +| `--import MODULE` | 派发命令前导入扩展模块，可重复。 | +| `--no-entry-points` | 跳过已安装的 `genelab.extensions` entry point。适合做完全显式的调试。 | +| `--help` | 显示帮助。 | + +## 命令 + +| 命令 | 用途 | +|---|---| +| `cache` | 创建项目本地缓存目录。 | +| `list robots` | 列出已注册机器人。 | +| `list envs` | 列出已注册环境。 | +| `list tasks` | 列出已注册任务。 | +| `info NAME` | 查看已注册 task、env 或 robot 的详情。 | +| `play TASK ...` | 运行已注册任务。 | +| `train TASK ...` | 用支持的 runner 训练任务。 | +| `prof open [DIR]` | 为 profiler trace 打开 TensorBoard。 | +| `project new NAME` | 生成外部 GeneLab 扩展项目骨架。 | + +## 运行时标志 + +这些标志放在 `play` 或 `train` 的 task id 后面。 + +| 标志 | 说明 | +|---|---| +| `--vis`、`-v` | 启用 Genesis viewer。 | +| `--gpu` | 使用 Genesis GPU 后端。 | +| `--steps N` | 在 `play` 中表示 rollout 步数；在 `train` 中是 `--max_iterations N` 的短写。 | +| `--dt X` | 覆盖仿真时间步。 | +| `--a.b.c VALUE` | 应用 dotted config override。 | + +## Runner 标志 + +| 标志 | 命令 | 说明 | +|---|---|---| +| `--num_envs N` | play/train | env 总数。分布式 train 中会除以 `--gpus`。 | +| `--num_envs_per_gpu N` | play/train | 每个 rank 的 env 数。与 `--num_envs` 互斥。 | +| `--agent zero|random|trained` | play | 选择策略来源。 | +| `--checkpoint PATH` | play/train | 加载 checkpoint。play 中会默认把 agent 设为 `trained`。 | +| `--seed N` | train | 覆盖 env 和 agent seed。 | +| `--log_dir PATH` | train | 使用已解析日志目录。 | +| `--max_iterations N` | train | 覆盖 PPO iteration 数。 | +| `--gpus N` | train | 用 `torchrun` 重新启动为 `N` 个本地 rank。 | + +## Profiler 标志 + +| 标志 | 环境变量兜底 | 默认 | +|---|---|---| +| `--prof` | `GENELAB_PROFILE=1` | 关闭 | +| `--prof-out PATH` | `GENELAB_PROFILE_OUT` | `logs/torch_profile` | +| `--prof-wait N` | `GENELAB_PROFILE_WAIT` | `10` | +| `--prof-warmup N` | `GENELAB_PROFILE_WARMUP` | `5` | +| `--prof-active N` | `GENELAB_PROFILE_ACTIVE` | `10` | +| `--prof-repeat N` | `GENELAB_PROFILE_REPEAT` | `2` | +| `--prof-record-shapes` | `GENELAB_PROFILE_RECORD_SHAPES=1` | 关闭 | +| `--prof-with-stack` | `GENELAB_PROFILE_WITH_STACK=1` | 关闭 | + +## 补全与交互式恢复 + +Typer 提供 `--install-completion` 和 `--show-completion`。补全会加载已安装 entry point， +但看不到临时 `--import MODULE` 扩展。 + +stdin 是 TTY 时，GeneLab 可对缺失 task id、未知名字、非法 `--agent`、未知 override 路径 +弹出选择器。脚本和 CI 中会直接抛出错误。 diff --git a/docs/reference/configuration.en.md b/docs/reference/configuration.en.md new file mode 100644 index 00000000..6915c21f --- /dev/null +++ b/docs/reference/configuration.en.md @@ -0,0 +1,74 @@ +# Configuration Reference + +GeneLab configuration is plain Python dataclasses. The CLI mutates these dataclasses before runtime +through dotted overrides; task code can construct or clone them directly. + +## Top-level task shape + +| Field | Type | Meaning | +|---|---|---| +| `TaskCfg.name` | `str` | Stable task id registered in `TASKS`. | +| `TaskCfg.env_name` | `str` | Name of the registered environment. | +| `TaskCfg.robot_name` | `str` | Name of the registered robot. | +| `TaskCfg.env` | `object` | Training/default environment config. Usually a `ManagerBasedRlEnvCfg` subclass. | +| `TaskCfg.play_env` | `object | None` | Optional play-mode config. `genelab play` prefers it when present. | +| `TaskCfg.agent` | `object | None` | Runner config. RSL-RL tasks use `RslRlOnPolicyRunnerCfg`. | +| `TaskCfg.trainable` | `bool` | Informational flag shown by registry views. | + +## Runtime configuration + +| Dataclass | Important fields | +|---|---| +| `SimulationCfg` | `vis`, `gpu`, `steps`, `dt`, `substeps`, `num_envs` | +| `InteractiveSceneCfg` | `env_spacing`, `sensors`, `mouse_interaction`, `entities`, `terrain`, `batch_render`, `recordings` | +| `ManagerBasedEnvCfg` | `simulation`, `scene`, and lightweight enabled flags for core manager groups | +| `ManagerBasedRlEnvCfg` | `decimation`, `episode_length_s`, `device`, `seed`, `robot`, `actions_cfg`, `observations_cfg`, `rewards_cfg`, `terminations_cfg`, `commands_cfg`, `events_cfg`, `curriculum_cfg`, `metrics_cfg`, `bridges_cfg` | + +## Override grammar + +Any `--a.b.c VALUE` after a `play` or `train` task id becomes an override. Hyphens in option names +are converted to underscores. + +```bash +uv run genelab play GeneLab-Inverted-Pendulum-v0 \ + --env.simulation.dt 0.005 \ + --env.rewards_cfg.pole_upright.weight 4.0 +``` + +The path is resolved against `TaskCfg`, so `env.*` points to `task.cfg.env` and `play_env.*` points +to `task.cfg.play_env`. + +## Shortcut aliases + +| CLI flag | Play target | Train target | +|---|---|---| +| `--vis` / `-v` | `play_env.simulation.vis=true` when `play_env` exists, else `env.simulation.vis=true` | `env.simulation.vis=true` | +| `--gpu` | `play_env.simulation.gpu=true` when `play_env` exists, else `env.simulation.gpu=true` | `env.simulation.gpu=true` | +| `--steps N` | `play_env.simulation.steps=N` when `play_env` exists, else `env.simulation.steps=N` | `--max_iterations N` | +| `--dt X` | `play_env.simulation.dt=X` when `play_env` exists, else `env.simulation.dt=X` | `env.simulation.dt=X` | + +## Type coercion + +`apply_overrides` reads dataclass annotations when available and converts strings to the target +type. + +| Target type | Accepted value | +|---|---| +| `bool` | `true`, `false`, `1`, `0`, `yes`, `no`, `on`, `off` | +| `int` | Decimal integer | +| `float` | Python float literal | +| `Path` | `pathlib.Path(value)` | +| `list[T]` | Comma-separated values coerced as `T` | +| `tuple[T, ...]` | Comma-separated values coerced as `T` | +| `None` | `none` or `null` | +| Other | String fallback | + +Unknown paths and invalid conversions fail before the simulator starts. + +## Discovering paths + +```bash +uv run genelab info GeneLab-Inverted-Pendulum-v0 +``` + +Copy paths from `Overridable cfg paths`. This is safer than guessing nested term names by hand. diff --git a/docs/reference/configuration.zh.md b/docs/reference/configuration.zh.md new file mode 100644 index 00000000..ea1d957c --- /dev/null +++ b/docs/reference/configuration.zh.md @@ -0,0 +1,73 @@ +# 配置参考 + +GeneLab 配置是普通 Python dataclass。CLI 在运行前通过 dotted override 修改这些 dataclass； +任务代码也可以直接构造或复制它们。 + +## 顶层任务结构 + +| 字段 | 类型 | 含义 | +|---|---|---| +| `TaskCfg.name` | `str` | 注册到 `TASKS` 的稳定 task id。 | +| `TaskCfg.env_name` | `str` | 已注册环境名。 | +| `TaskCfg.robot_name` | `str` | 已注册机器人名。 | +| `TaskCfg.env` | `object` | 默认/训练环境配置，通常是 `ManagerBasedRlEnvCfg` 子类。 | +| `TaskCfg.play_env` | `object | None` | 可选的 play 模式配置。`genelab play` 优先使用它。 | +| `TaskCfg.agent` | `object | None` | runner 配置。RSL-RL 任务使用 `RslRlOnPolicyRunnerCfg`。 | +| `TaskCfg.trainable` | `bool` | 注册表视图展示用标志。 | + +## 运行时配置 + +| Dataclass | 重要字段 | +|---|---| +| `SimulationCfg` | `vis`、`gpu`、`steps`、`dt`、`substeps`、`num_envs` | +| `InteractiveSceneCfg` | `env_spacing`、`sensors`、`mouse_interaction`、`entities`、`terrain`、`batch_render`、`recordings` | +| `ManagerBasedEnvCfg` | `simulation`、`scene` 和轻量 manager enabled flag | +| `ManagerBasedRlEnvCfg` | `decimation`、`episode_length_s`、`device`、`seed`、`robot`、`actions_cfg`、`observations_cfg`、`rewards_cfg`、`terminations_cfg`、`commands_cfg`、`events_cfg`、`curriculum_cfg`、`metrics_cfg`、`bridges_cfg` | + +## Override 语法 + +`play` 或 `train` 的 task id 后面，任何 `--a.b.c VALUE` 都会变成 override。选项名里的 +连字符会转换成下划线。 + +```bash +uv run genelab play GeneLab-Inverted-Pendulum-v0 \ + --env.simulation.dt 0.005 \ + --env.rewards_cfg.pole_upright.weight 4.0 +``` + +路径从 `TaskCfg` 开始解析，因此 `env.*` 指向 `task.cfg.env`，`play_env.*` 指向 +`task.cfg.play_env`。 + +## 短标志别名 + +| CLI 标志 | Play 目标 | Train 目标 | +|---|---|---| +| `--vis` / `-v` | 有 `play_env` 时为 `play_env.simulation.vis=true`，否则为 `env.simulation.vis=true` | `env.simulation.vis=true` | +| `--gpu` | 有 `play_env` 时为 `play_env.simulation.gpu=true`，否则为 `env.simulation.gpu=true` | `env.simulation.gpu=true` | +| `--steps N` | 有 `play_env` 时为 `play_env.simulation.steps=N`，否则为 `env.simulation.steps=N` | `--max_iterations N` | +| `--dt X` | 有 `play_env` 时为 `play_env.simulation.dt=X`，否则为 `env.simulation.dt=X` | `env.simulation.dt=X` | + +## 类型转换 + +`apply_overrides` 会读取 dataclass 注解，并把字符串转换到目标类型。 + +| 目标类型 | 接受值 | +|---|---| +| `bool` | `true`、`false`、`1`、`0`、`yes`、`no`、`on`、`off` | +| `int` | 十进制整数 | +| `float` | Python float 字面量 | +| `Path` | `pathlib.Path(value)` | +| `list[T]` | 逗号分隔，并按 `T` 转换 | +| `tuple[T, ...]` | 逗号分隔，并按 `T` 转换 | +| `None` | `none` 或 `null` | +| 其他 | 字符串兜底 | + +未知路径和无法转换的值会在仿真启动前报错。 + +## 发现可覆盖路径 + +```bash +uv run genelab info GeneLab-Inverted-Pendulum-v0 +``` + +从 `Overridable cfg paths` 里复制路径，比手写嵌套 term 名更可靠。 diff --git a/docs/reference/module-map.en.md b/docs/reference/module-map.en.md new file mode 100644 index 00000000..78700c41 --- /dev/null +++ b/docs/reference/module-map.en.md @@ -0,0 +1,61 @@ +# Module Map + +This reference maps GeneLab's public modules to the part of the system they own. Use it when you +know what you want to change but need the right import path. + +## Public facade + +| Module | Use it for | Typical imports | +|---|---|---| +| `genelab.lab` | Stable user-facing facade for configs, registries, scene objects, sensors, actuators, terrains, and extension loading. | `TaskCfg`, `ManagerBasedEnvCfg`, `ArticulationCfg`, `SensorCfg`, `register_*` via registries | +| `genelab` | Lightweight package root. It lazily re-exports a few top-level types without importing torch. | `__version__`, `TaskCfg`, `ManagerBasedEnvCfg` | + +Prefer `genelab.lab` in notebooks, downstream tasks, and examples unless a lower-level module is +documented as the owner of the API you need. + +## Discovery and dispatch + +| Module | Responsibility | +|---|---| +| `genelab.registry` | `ROBOTS`, `ENVS`, `TASKS`, `RegistryEntry`, registration helpers, entry-point loading, explicit extension imports. | +| `genelab.cli` | Typer/Rich CLI: `cache`, `list`, `info`, `play`, `train`, `prof`, `project new`. | +| `genelab.cache` | Creates project-local cache directories for Genesis, Quadrants, and Matplotlib. | + +## Configuration and MDP + +| Module | Responsibility | +|---|---| +| `genelab.configs` | Core dataclasses: `SimulationCfg`, `InteractiveSceneCfg`, `ManagerBasedEnvCfg`, `TaskCfg`, and `apply_overrides`. | +| `genelab.envs.manager_based_rl_env` | Genesis-backed `ManagerBasedRlEnv` and `ManagerBasedRlEnvCfg`. | +| `genelab.managers` | Manager and term cfg types for actions, commands, observations, rewards, terminations, events, curricula, and metrics. | +| `genelab.mdp` | Reusable MDP functions for observations, rewards, terminations, events, curricula, metrics, noise, actions, commands, and domain randomization. | + +## Simulation objects + +| Module | Responsibility | +|---|---| +| `genelab.scene` | `InteractiveScene`, the owner of the live Genesis scene. | +| `genelab.entity` | `Articulation`, `RigidObject`, and their config dataclasses. | +| `genelab.actuator` | Implicit PD, ideal PD, and DC motor actuator models. | +| `genelab.sensor` | Base sensor API and built-in body velocity, IMU, camera, contact, ray-cast, frame transformer, terrain height, self-contact, and angular momentum sensors. | +| `genelab.terrains` | Sub-terrain configs, terrain grid generation, and terrain import into Genesis. | +| `genelab.asset_zoo` | Curated robot asset configs and motion asset fetchers. This is bundled content, not the core abstraction boundary. | + +## Training and experiment I/O + +| Module | Responsibility | +|---|---| +| `genelab.rl` | RSL-RL configs, VecEnv wrapper, `train_task`, `play_task`, distributed helpers, and profiler integration. | +| `genelab.recording` | Declarative recording configs for plots, files, and camera video. | +| `genelab.bridges` | Runtime input/control bridges, including keyboard twist and DearPyGui twist controls. | +| `genelab.utils` | Math and asset download helpers used by higher-level modules. | + +## Choosing the right layer + +| Goal | Start here | +|---|---| +| Register a new robot, env, or task | `genelab.registry`, then [Build an Extension Project](../best-practices/extension-projects.md) | +| Add observations, rewards, or terminations | `genelab.managers`, `genelab.mdp`, then [Managers and MDP terms](../concepts/managers.md) | +| Add a sensor | `genelab.sensor`, then [Sensors](../concepts/sensors.md) | +| Change training or replay | `genelab.rl`, then [Run RL Experiments](../best-practices/rl-experiments.md) | +| Look up signatures and defaults | [API Reference](../api/reference.md) | diff --git a/docs/reference/module-map.zh.md b/docs/reference/module-map.zh.md new file mode 100644 index 00000000..0e73eb74 --- /dev/null +++ b/docs/reference/module-map.zh.md @@ -0,0 +1,59 @@ +# 模块地图 + +这是一页查阅型参考，用来把 GeneLab 的公开模块映射到它们负责的系统边界。你知道要改什么、但不确定从哪里导入时，先看这里。 + +## 公共 facade + +| 模块 | 用途 | 常见导入 | +|---|---|---| +| `genelab.lab` | 稳定的用户入口，汇总配置、注册表、scene 对象、传感器、执行器、地形和扩展加载。 | `TaskCfg`、`ManagerBasedEnvCfg`、`ArticulationCfg`、`SensorCfg` | +| `genelab` | 轻量包根。懒加载少量顶层类型，避免导入包根时立刻拉起 torch。 | `__version__`、`TaskCfg`、`ManagerBasedEnvCfg` | + +下游任务、notebook 和示例优先从 `genelab.lab` 导入；只有需要明确的低层能力时再进入对应模块。 + +## 发现与调度 + +| 模块 | 职责 | +|---|---| +| `genelab.registry` | `ROBOTS`、`ENVS`、`TASKS`、`RegistryEntry`、注册 helper、entry point 加载、显式扩展导入。 | +| `genelab.cli` | Typer/Rich 命令行：`cache`、`list`、`info`、`play`、`train`、`prof`、`project new`。 | +| `genelab.cache` | 创建 Genesis、Quadrants、Matplotlib 使用的项目本地缓存目录。 | + +## 配置与 MDP + +| 模块 | 职责 | +|---|---| +| `genelab.configs` | 核心 dataclass：`SimulationCfg`、`InteractiveSceneCfg`、`ManagerBasedEnvCfg`、`TaskCfg` 和 `apply_overrides`。 | +| `genelab.envs.manager_based_rl_env` | Genesis 后端的 `ManagerBasedRlEnv` 与 `ManagerBasedRlEnvCfg`。 | +| `genelab.managers` | action、command、observation、reward、termination、event、curriculum、metric 的 manager 与 term cfg。 | +| `genelab.mdp` | 可复用的 observation、reward、termination、event、curriculum、metric、noise、action、command、domain randomization 函数。 | + +## 仿真对象 + +| 模块 | 职责 | +|---|---| +| `genelab.scene` | `InteractiveScene`，持有 live Genesis scene。 | +| `genelab.entity` | `Articulation`、`RigidObject` 及其配置。 | +| `genelab.actuator` | 隐式 PD、理想 PD、DC motor 执行器模型。 | +| `genelab.sensor` | 传感器基类与 body velocity、IMU、camera、contact、ray-cast、frame transformer、terrain height、self-contact、angular momentum 传感器。 | +| `genelab.terrains` | 子地形配置、地形网格生成、导入 Genesis。 | +| `genelab.asset_zoo` | 预置机器人资产配置与 motion 资产下载 helper。它是随包内容，不是核心抽象边界。 | + +## 训练与实验 I/O + +| 模块 | 职责 | +|---|---| +| `genelab.rl` | RSL-RL 配置、VecEnv wrapper、`train_task`、`play_task`、分布式 helper、profiler 集成。 | +| `genelab.recording` | 声明式录制配置：实时曲线、文件、相机视频。 | +| `genelab.bridges` | 运行时输入/控制桥，包括键盘 twist 和 DearPyGui twist 控制。 | +| `genelab.utils` | 数学与资产下载 helper。 | + +## 选层规则 + +| 目标 | 起点 | +|---|---| +| 注册新机器人、环境、任务 | `genelab.registry`，再看 [构建扩展项目](../best-practices/extension-projects.md) | +| 增加 observation、reward、termination | `genelab.managers`、`genelab.mdp`，再看 [Manager 与 MDP term](../concepts/managers.md) | +| 增加传感器 | `genelab.sensor`，再看 [传感器](../concepts/sensors.md) | +| 修改训练或回放 | `genelab.rl`，再看 [运行 RL 实验](../best-practices/rl-experiments.md) | +| 查签名与默认值 | [API 参考](../api/reference.md) | diff --git a/docs/stylesheets/extra.css b/docs/stylesheets/extra.css index 6ef54baa..003bcaa8 100644 --- a/docs/stylesheets/extra.css +++ b/docs/stylesheets/extra.css @@ -447,8 +447,8 @@ letter-spacing: 0.02em; } -/* Modules at a glance — 2-column technical layout */ -.gl-modules { +/* Abilities at a glance — 2-column technical layout */ +.gl-abilities { display: grid; grid-template-columns: repeat(auto-fit, minmax(18rem, 1fr)); gap: 0.4rem 1.6rem; @@ -457,7 +457,7 @@ list-style: none; } -.gl-modules li { +.gl-abilities li { display: grid; grid-template-columns: minmax(8.5rem, max-content) 1fr; gap: 1rem; @@ -468,7 +468,7 @@ line-height: 1.55; } -.gl-modules li code { +.gl-abilities li code { font-size: 0.78rem; background: transparent; padding: 0; @@ -476,10 +476,10 @@ white-space: nowrap; } -[data-md-color-scheme="slate"] .gl-modules li code { +[data-md-color-scheme="slate"] .gl-abilities li code { color: var(--md-primary-fg-color--light); } -.gl-modules li span { +.gl-abilities li span { color: var(--md-default-fg-color--light); } diff --git a/docs/tutorial.en.md b/docs/tutorial.en.md new file mode 100644 index 00000000..eca24d95 --- /dev/null +++ b/docs/tutorial.en.md @@ -0,0 +1,310 @@ +--- +hide: + - toc +--- + +# Build Your First Robot Experiment with GeneLab + +This tutorial starts from a clean checkout and walks through the smallest useful GeneLab loop: +install the environment, inspect the registries, run the inverted pendulum task, understand how a +task reaches the Genesis backend, override configuration, launch a short training run, and scaffold +your own downstream extension project. It then points to the Unitree G1 tasks as the next step for a +full robot workflow. By the end, you should understand the shape of GeneLab and know where to +continue into detailed module, API, and best-practice documentation. + +The intended reader knows Python and the command line, but is new to GeneLab. You do not need to +understand every internal module before starting; each step has a concrete result to verify. + +## What You Will Build + +You will run the bundled `GeneLab-Inverted-Pendulum-v0` task. It is small, but it covers the core +GeneLab path: + +```text +CLI +└── TASKS registry + └── TaskCfg + ├── env / play_env: ManagerBasedRlEnvCfg + ├── robot: ArticulationCfg + ├── manager terms: actions, observations, rewards, terminations, events + └── agent: RslRlOnPolicyRunnerCfg + └── Genesis-backed ManagerBasedRlEnv +``` + +Larger tasks such as Unitree G1, terrain curricula, sensors, and recording use the same route with a +larger robot, richer manager terms, and a more complete runner configuration. + +## 1. Prepare the Environment + +From the repository root, sync dependencies: + +```bash +uv sync --extra torch-cpu +uv run genelab --version +``` + +If you have an NVIDIA GPU, replace `torch-cpu` with the matching `torch-cu126`, `torch-cu128`, or +`torch-cu130` extra. Pick exactly one `torch-*` extra. + +Expected result: + +```text +genelab 0.1.0 +``` + +Create project-local cache directories used by Genesis and plotting backends: + +```bash +uv run genelab cache +``` + +If dependency sync fails or torch is wrong, read [Installation](getting-started/installation.md). + +## 2. Inspect What GeneLab Knows + +The core `genelab` package provides the framework. Robots, environments, and tasks are registered by +the asset zoo or extension packages. List the three registries: + +```bash +uv run genelab list robots +uv run genelab list envs +uv run genelab list tasks +``` + +You should see bundled asset-zoo robots and any example tasks discovered through entry points. If +the task list is empty, explicitly import the inverted pendulum extension: + +```bash +uv run genelab --import genelab_inverted_pendulum.tasks list tasks +``` + +The registries are the first architecture boundary: + +| Registry | Contents | Typical consumer | +|---|---|---| +| `ROBOTS` | Robot or asset config factories | Env configs, CLI `list robots` | +| `ENVS` | Environment factories | CLI `list envs`, downstream projects | +| `TASKS` | Task factories carrying env, play env, and agent cfg | `genelab play`, `genelab train` | + +## 3. Run the First Task + +Start with a short rollout to verify construction and stepping: + +```bash +uv run genelab play GeneLab-Inverted-Pendulum-v0 --steps 32 +``` + +On a local desktop, open the Genesis viewer: + +```bash +uv run genelab play GeneLab-Inverted-Pendulum-v0 --vis --steps 500 +``` + +This step follows this path: + +1. The CLI loads entry-point extensions and explicit `--import` modules. +2. The CLI resolves `GeneLab-Inverted-Pendulum-v0` in `TASKS`. +3. The task returns a `TaskCfg`; when present, `play_env` is preferred for viewer-friendly runs. +4. `ManagerBasedRlEnv` builds the Genesis scene, robot articulation, sensors, and managers. +5. The play loop produces actions and calls `env.step(action)`. + +## 4. Inspect and Override Task Configuration + +Use `info` to inspect task metadata and overridable paths: + +```bash +uv run genelab info GeneLab-Inverted-Pendulum-v0 +``` + +Look for `Overridable cfg paths`. Every CLI `--a.b.c VALUE` is applied to a dataclass field under +`TaskCfg`, with type coercion handled by `apply_overrides`. + +Change the timestep and rollout length: + +```bash +uv run genelab play GeneLab-Inverted-Pendulum-v0 \ + --steps 128 \ + --env.simulation.dt 0.005 +``` + +Common shortcut flags are rewritten to config paths. In `play`, they target `play_env` when the task +defines one; otherwise they target `env`. + +| Shortcut | Play target with `play_env` | Train target | +|---|---|---| +| `--vis` | `play_env.simulation.vis=true` | `env.simulation.vis=true` | +| `--gpu` | `play_env.simulation.gpu=true` | `env.simulation.gpu=true` | +| `--steps N` | `play_env.simulation.steps=N` | `--max_iterations N` | +| `--dt X` | `play_env.simulation.dt=X` | `env.simulation.dt=X` | + +## 5. Read the Task Source Against the Architecture + +The inverted pendulum registration entry point is +`examples/inverted_pendulum/src/genelab_inverted_pendulum/tasks.py`. It does three things: + +```python +register_robot("inverted-pendulum", get_inverted_pendulum_robot_cfg, ...) +register_env("inverted-pendulum-env", lambda: ManagerBasedRlEnv(...), ...) +register_task("GeneLab-Inverted-Pendulum-v0", InvertedPendulumTask, ...) +``` + +The environment config lives in +`examples/inverted_pendulum/src/genelab_inverted_pendulum/single/env_cfg.py`. It shows GeneLab's +manager-based MDP shape: + +```python +actions_cfg={ + "cart": JointPositionActionCfg(...) +} +observations_cfg={ + "policy": ObservationGroupCfg(terms={...}), + "critic": ObservationGroupCfg(terms={...}), +} +rewards_cfg={ + "alive": RewardTermCfg(...), + "pole_upright": RewardTermCfg(...), +} +terminations_cfg={ + "time_out": TerminationTermCfg(...), + "pole_fell": TerminationTermCfg(...), +} +events_cfg={ + "reset_joints": EventTermCfg(mode="reset", ...) +} +``` + +This is the main GeneLab capability: RL environments are assembled from replaceable terms instead +of hiding everything inside one large `step()` function. + +## 6. Launch a Short Training Run + +The inverted pendulum task ships with an RSL-RL runner config. Run a tiny training job to verify the +RL pipeline can construct the env, wrap it as a VecEnv, and write logs/checkpoints: + +```bash +uv run genelab train GeneLab-Inverted-Pendulum-v0 \ + --num_envs 64 \ + --max_iterations 2 +``` + +After it finishes, inspect the log root: + +```bash +ls logs/rsl_rl +``` + +The main process writes: + +```text +params/env.json +params/agent.json +model_*.pt +``` + +Replay a trained policy by passing a checkpoint: + +```bash +uv run genelab play GeneLab-Inverted-Pendulum-v0 \ + --agent trained \ + --checkpoint logs/rsl_rl///model_*.pt +``` + +For environment health checks, use `--agent zero` or `--agent random`. + +## 7. Create Your Own Extension Project + +GeneLab expects real projects to live in their own Python packages rather than under +`src/genelab/`. Generate a scaffold: + +```bash +uv run genelab project new my_robot_project +``` + +The scaffold contains: + +```text +my_robot_project/ +├── pyproject.toml +└── src/my_robot_project/ + ├── config.py + ├── envs.py + ├── robots.py + └── tasks.py +``` + +The key is the entry point in `pyproject.toml`: + +```toml +[project.entry-points."genelab.extensions"] +my_robot_project = "my_robot_project.tasks:register" +``` + +Install it as editable and the CLI discovers it automatically: + +```bash +uv pip install -e my_robot_project +uv run genelab list tasks +``` + +Without installing, import it explicitly: + +```bash +PYTHONPATH=my_robot_project/src \ + uv run genelab --import my_robot_project.tasks list tasks +``` + +## 8. Continue to Unitree G1 + +The Unitree G1 example uses the same architecture with a humanoid robot, larger action and +observation spaces, command sampling, RSL-RL training, and motion imitation. + +Install the extension: + +```bash +uv pip install -e examples/unitree +uv run genelab list tasks +``` + +You should see: + +```text +Genelab-Velocity-Flat-Unitree-G1-v0 +Genelab-Tracking-Flat-Unitree-G1-v0 +``` + +Run a visual smoke test before training: + +```bash +uv run genelab play Genelab-Velocity-Flat-Unitree-G1-v0 --vis --steps 500 +``` + +The two Unitree tasks demonstrate the advanced side of GeneLab: + +| Task | Demonstrates | +|---|---| +| `Genelab-Velocity-Flat-Unitree-G1-v0` | Velocity commands, humanoid action scaling, PPO training, checkpoint replay. | +| `Genelab-Tracking-Flat-Unitree-G1-v0` | Motion-command loading, reference replay, body pose tracking rewards. | + +Read [Unitree G1](examples/unitree-g1.md) for the full training and replay commands. + +## 9. GeneLab Capability Map + +You have now walked through the smallest useful loop and seen where the larger robot example fits. +GeneLab can be understood in four layers: + +| Layer | Modules | Role | +|---|---|---| +| Discovery and dispatch | `genelab.registry`, `genelab.cli` | Register robots/envs/tasks; discover, run, and train from the CLI | +| Configuration and MDP | `genelab.configs`, `genelab.managers`, `genelab.mdp` | Dataclass configs, overrides, action/obs/reward/termination/event/curriculum terms | +| Simulation objects | `genelab.scene`, `genelab.entity`, `genelab.actuator`, `genelab.sensor`, `genelab.terrains` | Genesis scenes, articulations, rigid objects, actuators, sensors, terrains | +| Training and experiments | `genelab.rl`, `genelab.recording`, `genelab.bridges` | RSL-RL training/playback, recording, live plots, keyboard or GUI control | + +## Next Steps + +- For more commands, read [CLI overview](cli/overview.md) and [Play and Train](cli/play-train.md). +- For architecture, read [Module map](reference/module-map.md) and + [Managers and MDP terms](concepts/managers.md). +- To write your own task, read [Task design](best-practices/task-design.md) and + [Extensions](concepts/extensions.md). +- To look up APIs, read [API Reference](api/reference.md). +- For complete demos, read [Examples](examples/overview.md). diff --git a/docs/tutorial.zh.md b/docs/tutorial.zh.md new file mode 100644 index 00000000..e2c7f4e8 --- /dev/null +++ b/docs/tutorial.zh.md @@ -0,0 +1,307 @@ +--- +hide: + - toc +--- + +# 用 GeneLab 构建第一个机器人实验 + +本教程从一个干净 checkout 开始，带你完成 GeneLab 的最小闭环：安装环境、查看注册表、运行 +倒立摆、理解任务如何接入 Genesis 后端、修改配置、启动一次短训练，最后创建一个自己的下游 +扩展项目。教程末尾会把同一套结构延伸到 Unitree G1，作为完整机器人工作流的进阶入口。完成后， +你应该能说清 GeneLab 的整体架构，并知道下一步去哪里查模块、API 和最佳实践。 + +目标读者是熟悉 Python 与命令行、但第一次接触 GeneLab 的机器人或强化学习开发者。教程不要求 +你先理解所有内部模块；每一步都会给出可验证结果。 + +## 你会构建什么 + +你会运行仓库自带的 `GeneLab-Inverted-Pendulum-v0` 任务。这个任务很小，但覆盖了 GeneLab +的核心路径： + +```text +CLI +└── TASKS registry + └── TaskCfg + ├── env / play_env: ManagerBasedRlEnvCfg + ├── robot: ArticulationCfg + ├── manager terms: actions, observations, rewards, terminations, events + └── agent: RslRlOnPolicyRunnerCfg + └── Genesis-backed ManagerBasedRlEnv +``` + +后续复杂任务，例如 Unitree G1、地形课程、传感器和数据录制，本质上是在这条路径上增加更大的 +机器人、更丰富的 manager term，以及更完整的 runner 配置。 + +## 1. 准备环境 + +在仓库根目录同步依赖： + +```bash +uv sync --extra torch-cpu +uv run genelab --version +``` + +如果你有 NVIDIA GPU，把 `torch-cpu` 换成与你驱动匹配的 `torch-cu126`、`torch-cu128` 或 +`torch-cu130`。每次只能选一个 `torch-*` extra。 + +你应该看到类似输出： + +```text +genelab 0.1.0 +``` + +初始化 Genesis 和绘图库需要的项目本地缓存： + +```bash +uv run genelab cache +``` + +如果安装失败或 torch 版本不对，先看 [安装](getting-started/installation.md)。 + +## 2. 查看 GeneLab 当前知道什么 + +GeneLab 的核心包只提供框架。机器人、环境和任务由内置资产库或扩展包注册进来。先列出三张 +注册表： + +```bash +uv run genelab list robots +uv run genelab list envs +uv run genelab list tasks +``` + +你应该能看到内置资产库的机器人，以及已通过 entry point 发现的示例任务。若任务列表为空， +说明示例扩展没有被安装或发现；可以显式导入倒立摆扩展： + +```bash +uv run genelab --import genelab_inverted_pendulum.tasks list tasks +``` + +注册表是 GeneLab 的第一层架构边界： + +| 注册表 | 放什么 | 典型消费者 | +|---|---|---| +| `ROBOTS` | 机器人或资产配置 factory | 环境配置、CLI `list robots` | +| `ENVS` | 可构造环境的 factory | CLI `list envs`、下游项目 | +| `TASKS` | 包含 env、play env、agent 的任务 factory | `genelab play`、`genelab train` | + +## 3. 运行第一个任务 + +先用短 rollout 验证任务可以构造并走完几步： + +```bash +uv run genelab play GeneLab-Inverted-Pendulum-v0 --steps 32 +``` + +如果你在本地桌面环境中运行，可以打开 Genesis viewer： + +```bash +uv run genelab play GeneLab-Inverted-Pendulum-v0 --vis --steps 500 +``` + +这一步经过的路径是： + +1. CLI 加载 entry-point 扩展和显式 `--import` 扩展。 +2. CLI 在 `TASKS` 中查找 `GeneLab-Inverted-Pendulum-v0`。 +3. task 返回 `TaskCfg`，其中 `play_env` 是可视化友好的环境配置。 +4. `ManagerBasedRlEnv` 构造 Genesis scene、机器人 articulation、传感器和各类 manager。 +5. play loop 产生动作并调用 `env.step(action)`。 + +## 4. 看清一个任务的可改配置 + +用 `info` 查看任务元信息和可覆盖路径： + +```bash +uv run genelab info GeneLab-Inverted-Pendulum-v0 +``` + +重点看输出里的 `Overridable cfg paths`。CLI 的每个 `--a.b.c VALUE` 都会落到 +`TaskCfg` 的某个 dataclass 字段上，并由 `apply_overrides` 自动转换类型。 + +试着改仿真步长和 episode 步数： + +```bash +uv run genelab play GeneLab-Inverted-Pendulum-v0 \ + --steps 128 \ + --env.simulation.dt 0.005 +``` + +常用短标志会被改写为配置路径。`play` 中，如果 task 定义了 `play_env`，短标志会优先作用到 +`play_env`；否则才作用到 `env`。 + +| 短标志 | 有 `play_env` 时的 play 目标 | train 目标 | +|---|---|---| +| `--vis` | `play_env.simulation.vis=true` | `env.simulation.vis=true` | +| `--gpu` | `play_env.simulation.gpu=true` | `env.simulation.gpu=true` | +| `--steps N` | `play_env.simulation.steps=N` | `--max_iterations N` | +| `--dt X` | `play_env.simulation.dt=X` | `env.simulation.dt=X` | + +## 5. 打开任务源码并对照架构 + +倒立摆的注册入口在 `examples/inverted_pendulum/src/genelab_inverted_pendulum/tasks.py`。 +它做三件事： + +```python +register_robot("inverted-pendulum", get_inverted_pendulum_robot_cfg, ...) +register_env("inverted-pendulum-env", lambda: ManagerBasedRlEnv(...), ...) +register_task("GeneLab-Inverted-Pendulum-v0", InvertedPendulumTask, ...) +``` + +环境配置在 +`examples/inverted_pendulum/src/genelab_inverted_pendulum/single/env_cfg.py`。这里能看到 +GeneLab 的 manager-based MDP 形状： + +```python +actions_cfg={ + "cart": JointPositionActionCfg(...) +} +observations_cfg={ + "policy": ObservationGroupCfg(terms={...}), + "critic": ObservationGroupCfg(terms={...}), +} +rewards_cfg={ + "alive": RewardTermCfg(...), + "pole_upright": RewardTermCfg(...), +} +terminations_cfg={ + "time_out": TerminationTermCfg(...), + "pole_fell": TerminationTermCfg(...), +} +events_cfg={ + "reset_joints": EventTermCfg(mode="reset", ...) +} +``` + +这一层是 GeneLab 最重要的能力：把强化学习环境拆成可替换的 term，而不是把所有逻辑塞进一个 +巨大的 `step()` 函数。 + +## 6. 启动一次短训练 + +倒立摆任务自带 RSL-RL runner 配置。先跑一个很短的训练，验证 RL 管线可以构造 env、包装 +VecEnv、写日志和 checkpoint： + +```bash +uv run genelab train GeneLab-Inverted-Pendulum-v0 \ + --num_envs 64 \ + --max_iterations 2 +``` + +训练结束后，查看日志目录： + +```bash +ls logs/rsl_rl +``` + +你会看到按实验名和时间戳组织的目录。主进程会写入： + +```text +params/env.json +params/agent.json +model_*.pt +``` + +回放训练好的策略时传入 checkpoint： + +```bash +uv run genelab play GeneLab-Inverted-Pendulum-v0 \ + --agent trained \ + --checkpoint logs/rsl_rl///model_*.pt +``` + +如果只是检查动作边界或环境稳定性，也可以用 `--agent zero` 或 `--agent random`。 + +## 7. 创建自己的扩展项目 + +GeneLab 推荐把真实项目放在独立 Python 包里，而不是直接改 `src/genelab/`。生成一个项目骨架： + +```bash +uv run genelab project new my_robot_project +``` + +骨架会包含： + +```text +my_robot_project/ +├── pyproject.toml +└── src/my_robot_project/ + ├── config.py + ├── envs.py + ├── robots.py + └── tasks.py +``` + +关键是 `pyproject.toml` 里的 entry point： + +```toml +[project.entry-points."genelab.extensions"] +my_robot_project = "my_robot_project.tasks:register" +``` + +安装为 editable 包之后，CLI 会自动发现它： + +```bash +uv pip install -e my_robot_project +uv run genelab list tasks +``` + +如果暂时不安装，也可以显式导入： + +```bash +PYTHONPATH=my_robot_project/src \ + uv run genelab --import my_robot_project.tasks list tasks +``` + +## 8. 继续到 Unitree G1 + +Unitree G1 示例使用同一套架构，但换成了人形机器人、更大的 action/observation 空间、 +命令采样、RSL-RL 训练和动作模仿。 + +安装扩展： + +```bash +uv pip install -e examples/unitree +uv run genelab list tasks +``` + +你应该能看到： + +```text +Genelab-Velocity-Flat-Unitree-G1-v0 +Genelab-Tracking-Flat-Unitree-G1-v0 +``` + +训练前先做可视化 smoke test： + +```bash +uv run genelab play Genelab-Velocity-Flat-Unitree-G1-v0 --vis --steps 500 +``` + +两个 Unitree 任务展示 GeneLab 的进阶能力： + +| Task | 展示能力 | +|---|---| +| `Genelab-Velocity-Flat-Unitree-G1-v0` | 速度命令、人形机器人 action scale、PPO 训练、checkpoint 回放。 | +| `Genelab-Tracking-Flat-Unitree-G1-v0` | motion command 加载、参考动作回放、body pose tracking reward。 | + +完整训练与回放命令见 [Unitree G1](examples/unitree-g1.md)。 + +## 9. GeneLab 能力地图 + +到这里，你已经走完最小闭环，也看到了更大机器人示例的位置。GeneLab 的能力可以按四层理解： + +| 层级 | 模块 | 作用 | +|---|---|---| +| 发现与调度 | `genelab.registry`、`genelab.cli` | 注册机器人、环境、任务；用 CLI 发现、运行、训练 | +| 配置与 MDP | `genelab.configs`、`genelab.managers`、`genelab.mdp` | dataclass 配置、override、action/obs/reward/termination/event/curriculum term | +| 仿真对象 | `genelab.scene`、`genelab.entity`、`genelab.actuator`、`genelab.sensor`、`genelab.terrains` | Genesis scene、articulation、刚体、执行器、传感器、地形 | +| 训练与实验 | `genelab.rl`、`genelab.recording`、`genelab.bridges` | RSL-RL 训练/回放、数据录制、实时绘图、键盘或 GUI 控制 | + +## 下一步 + +按目标继续阅读： + +- 想跑更多命令：看 [CLI 总览](cli/overview.md) 和 [play 与 train](cli/play-train.md)。 +- 想理解架构：看 [模块地图](reference/module-map.md)、[Manager 与 MDP term](concepts/managers.md)。 +- 想写自己的任务：看 [设计任务配置](best-practices/task-design.md) 和 + [扩展加载](concepts/extensions.md)。 +- 想查 API：看 [API 参考](api/reference.md)。 +- 想看完整示例：看 [示例](examples/overview.md)。 diff --git a/mkdocs.yml b/mkdocs.yml index 65199a21..83b625cc 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -2,6 +2,7 @@ site_name: GeneLab site_description: Isaac Lab-inspired API for RL and robotics research powered by Genesis. site_url: https://krahsu.github.io/GeneLab/ site_author: GeneLab contributors +dev_addr: 127.0.0.1:8001 repo_name: KraHsu/GeneLab repo_url: https://github.com/KraHsu/GeneLab @@ -71,6 +72,8 @@ plugins: lang: - en - zh + - autorefs: + resolve_closest: true - include-markdown - i18n: docs_structure: suffix @@ -87,9 +90,19 @@ plugins: site_name: GeneLab nav_translations: Home: 首页 + Tutorial: 教程 Getting Started: 快速上手 Installation: 安装 Quickstart: 快速开始 + How-to Guides: 最佳实践 + Design a Task: 设计任务 + Build an Extension Project: 构建扩展项目 + Run RL Experiments: 运行 RL 实验 + Debug Common Failures: 调试常见问题 + Reference: 参考 + Module Map: 模块地图 + Configuration Reference: 配置参考 + CLI Reference: CLI 参考 CLI: 命令行 Overview: 总览 'Discovery: list and info': '发现：list 与 info' @@ -162,9 +175,20 @@ markdown_extensions: nav: - Home: index.md + - Tutorial: tutorial.md - Getting Started: - Installation: getting-started/installation.md - Quickstart: getting-started/quickstart.md + - How-to Guides: + - Design a Task: best-practices/task-design.md + - Build an Extension Project: best-practices/extension-projects.md + - Run RL Experiments: best-practices/rl-experiments.md + - Debug Common Failures: best-practices/debugging.md + - Reference: + - Module Map: reference/module-map.md + - Configuration Reference: reference/configuration.md + - CLI Reference: reference/cli.md + - API Reference: api/reference.md - CLI: - Overview: cli/overview.md - 'Discovery: list and info': cli/list-info.md @@ -191,5 +215,4 @@ nav: - Showcase: examples/showcase.md - "Rubik's Cube": examples/rubiks-cube.md - Wuji Hand: examples/wuji-hand.md - - API Reference: api/reference.md - Contributing: contributing.md