Skip to content

Add skrl as a second RL backend behind a backend abstraction - #77

Merged
KraHsu merged 2 commits into
devfrom
feat/skrl-rl-backend
May 19, 2026
Merged

KraHsu merged 2 commits into
devfrom
feat/skrl-rl-backend

Conversation

@KraHsu

@KraHsu KraHsu commented May 19, 2026

Copy link
Copy Markdown
Owner

Context

GeneLab was hard-wired to a single RL library — RSL-RL (PPO only). This PR adds skrl as a second RL backend behind a generic backend abstraction: RSL-RL and skrl are both selectable behind train_task / play_task, chosen automatically by the type of TaskCfg.agent. RSL-RL stays the default and its behavior is unchanged.

Changes

Backend abstraction (src/genelab/rl/backends/)

  • base.py — Backend protocol + TrainContext / PlayContext / ProfileArgs.
  • __init__.py — registry keyed by agent-cfg type; select_backend(agent_cfg).
  • rsl_rl.py — the existing RSL-RL / OnPolicyRunner path, moved into RslRlBackend verbatim.
  • skrl.py — SkrlBackend: PPO, A2C, SAC, TD3, DDPG via skrl agents + SequentialTrainer.

skrl glue

  • skrl_config.py — one SkrlAgentCfg dataclass covering all five algorithms.
  • skrl_wrapper.py — GenelabSkrlWrapper adapts ManagerBasedRlEnv to skrl (guarded import).
  • skrl_models.py — per-algorithm MLP model factory on skrl's mixin API.

Wiring

  • runner.py — train_task / play_task become thin, backend-agnostic dispatchers; shared helpers stay shared.
  • cli/__init__.py — _dispatch_train routes via select_backend; multi-GPU stays RSL-RL only.
  • pyproject.toml — optional skrl extra (skrl>=1.4.0,<2).
  • Franka example — new GeneLab-Franka-Pick-And-Place-skrl-v0 task; docs updated (EN/ZH).

Note: skrl trains in timesteps, so --max-iterations maps to the skrl timestep budget.

Verification

  • 336 existing tests pass (RSL-RL regression intact); new test_skrl_pipeline.py (9 tests) passes.
  • ruff check and pyright (strict) both clean.
  • End-to-end on the Franka task with Genesis (CPU): skrl PPO train, trained-policy replay, and SAC off-policy train all run, producing checkpoints + TensorBoard logs.

🤖 Generated with Claude Code

KraHsu and others added 2 commits May 19, 2026 22:08
GeneLab was hard-wired to RSL-RL (PPO only). Introduce a generic RL
backend abstraction so RSL-RL and skrl are both selectable behind
train_task / play_task, chosen by the type of TaskCfg.agent.

- genelab.rl.backends: Backend protocol + Train/PlayContext, a registry
  keyed by agent-cfg type, RslRlBackend (existing path moved verbatim),
  and SkrlBackend (PPO, A2C, SAC, TD3, DDPG).
- skrl glue: SkrlAgentCfg (one dataclass for all algorithms),
  GenelabSkrlWrapper, and an MLP model factory on skrl's mixin API.
- runner.py train_task/play_task become backend-agnostic dispatchers;
  shared helpers (bridges, play loop, log dirs) stay shared.
- CLI _dispatch_train routes via select_backend; multi-GPU stays
  RSL-RL only. skrl is an optional extra (skrl>=1.4.0,<2).
- Franka example gains a GeneLab-Franka-Pick-And-Place-skrl-v0 task;
  docs (EN/ZH) updated. New test_skrl_pipeline.py.

RSL-RL behavior is unchanged; existing tests pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@KraHsu
KraHsu merged commit 5ffb3e8 into dev May 19, 2026
3 checks passed
@KraHsu
KraHsu deleted the feat/skrl-rl-backend branch May 19, 2026 14:16
KraHsu added a commit that referenced this pull request Jun 5, 2026
Add skrl as a second RL backend behind a backend abstraction
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant