You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit c71c858
Browse filesBrowse the repository at this point in the historyBrowse files
feat(examples): add "recall the target" RNN memory showcase
Demonstrates when recurrence decisively beats a feedforward MLP — and, by being
the only such task here, makes the honest point that POMDP stabilisation alone
does not need an RNN.
Task: each episode a random cart target is flashed in the observation for the
first 5 steps then removed; the cart must be driven to the (now hidden) target.
A memoryless MLP cannot recall it and drifts to centre; an LSTM stores it in its
hidden state and tracks it. Measured post-cue tracking error: LSTM 0.013 vs
MLP 0.78 (targets uniform in [-0.8, 0.8]).
- mdp.py: reset_cart_target / cart_target_cue (masked after cue_steps) /
cart_target_tracking_l2, with a per-env target buffer stashed on the env.
- single/memory_env_cfg.py: reuses the base env, adds the masked cue obs,
replaces cart-centring with target tracking, removes pushes, and shortens
episodes to ~100 steps.
- single/rnn_cfg.py: LSTM + MLP PPO configs. num_steps_per_env=100 so truncated
BPTT spans the whole cue->reach episode (the key trainability detail; a 24-step
window leaves the LSTM unable to learn the recall).
- tasks.py: registers GeneLab-Inverted-Pendulum-Memory-{Rnn,Mlp}-v0 via a shared
_MemoryTask base; single/double/skrl tasks unchanged.
- docs + README: memory showcase section with the real numbers and the
trainability note; EN/ZH in parity.
- tests: registration, RNN-vs-MLP cfg (RNNModel vs MLPModel, num_steps_per_env),
and the masked-cue/tracking-reward env wiring — all headless.
Builds on the recurrent support in dev/recurrent. Trained + evaluated end-to-end
on GPU (the numbers above); example tests + ruff green.
0 commit comments