This is an implementation of the Hierarchical Actor Critic (HAC) algorithm described in the paper, Learning Multi-Level Hierarchies with Hindsight (ICLR 2019), in PyTorch for OpenAI gym environments. The algorithm learns to reach a goal state by dividing the task into short horizon intermediate goals (subgoals).
- All the hyperparameters are contained in the
train.py
file. - To train a new network run
train.py
- To test a preTrained network run
test.py
- For a detailed explanation of offsets and bounds, refer to issue #2
- For hyperparameters used for preTraining the pendulum policy refer to issue #3
- The code is implemented as described in the appendix section of the paper and the Official repository, i.e. without target networks and with bounded Q-values.
- The Actor and Critic networks have 2 hidded layers of size 64.
Please use this bibtex if you want to cite this repository in your publications :
@misc{pytorch_hac,
author = {Barhate, Nikhil},
title = {PyTorch Implementation of Hierarchical Actor-Critic},
year = {2021},
publisher = {GitHub},
journal = {GitHub repository},
howpublished = {\url{https://github.com/nikhilbarhate99/Hierarchical-Actor-Critic-HAC-PyTorch}},
}
- Python 3.6
- PyTorch
- OpenAI gym
(2 levels, H = 20, 200 episodes) | (3 levels, H = 5, 200 episodes) |
---|---|
(2 levels, H = 20, 200 episodes) |
---|
- Official Paper and Code (TensorFlow)