This repository provides a PyTorch implementation of the Hierarchical Actor-Critic (HAC) algorithm, designed for goal-reaching tasks in OpenAI gym environments. It addresses complex, multi-stage decision-making by decomposing tasks into hierarchical subgoals, benefiting researchers and practitioners in reinforcement learning.

How It Works

The implementation follows the HAC paper's appendix, utilizing a hierarchical structure where higher-level actors set subgoals for lower-level actors. It omits target networks and employs bounded Q-values, with both actor and critic networks featuring two hidden layers of 64 units. This approach simplifies training and improves stability for multi-level goal achievement.

Quick Start & Requirements

• Install via pip install -r requirements.txt (implicitly, as no direct install command is given).
• Requires Python 3.6 and PyTorch.
• Tested with OpenAI gym environments.

Highlighted Details

• Implements HAC as described in "Learning Multi-Level Hierarchies with Hindsight" (ICLR 2019).
• Actor and Critic networks use 2x64 hidden layers.
• Code follows the paper's appendix, omitting target networks and using bounded Q-values.
• Includes results for MountainCarContinuous-v0 with 2 and 3 levels of hierarchy.

Maintenance & Community

• Maintained by Nikhil Barhate.
• No community links (Discord, Slack) or roadmap are provided in the README.

Licensing & Compatibility

• The repository does not explicitly state a license.
• Compatibility for commercial use or closed-source linking is undetermined.

Limitations & Caveats

The project requires Python 3.6, which is outdated. The README lacks explicit licensing information, posing a potential blocker for commercial adoption. No community channels or detailed documentation beyond the README are linked.