This repository provides code for the "Meta-Learning Shared Hierarchies" paper, enabling researchers and practitioners to explore meta-learning for hierarchical reinforcement learning. It offers a framework for learning shared hierarchical policies across a distribution of tasks, potentially leading to faster adaptation and improved sample efficiency.

How It Works

The MLSH approach focuses on learning shared hierarchical policies by leveraging a meta-learning framework. It trains a higher-level policy that selects sub-policies, which are then trained to solve specific sub-tasks. This hierarchical decomposition allows for more efficient exploration and learning of complex behaviors, with the meta-learning aspect enabling rapid adaptation to new, related tasks.

Quick Start & Requirements

• Install by adding mlsh/gym and mlsh/rl-algs to PYTHONPATH.
• Install custom environments: cd test_envs && pip install -e .
• Run experiments: python main.py --task AntBandits-v1 --num_subs 2 --macro_duration 1000 --num_rollouts 2000 --warmup_time 20 --train_time 30 --replay False
• View trained agents: python main.py [...] --replay True --continue_iter [your iteration]
• Requires environments implementing randomizeCorrect().
• Supports multi-core execution via mpirun.

Highlighted Details

• Implements the Meta-Learning Shared Hierarchies (MLSH) algorithm.
• Designed for environments compatible with the OpenAI Gym interface and randomizeCorrect() function.
• Includes example environments within the envs/ directory.
• Supports distributed execution for faster training.

Maintenance & Community

The project is marked as "Archive" and no updates are expected.

Licensing & Compatibility

The repository does not explicitly state a license.

Limitations & Caveats

The project is archived and will not receive further updates. Compatibility is limited to Gym environments that implement the randomizeCorrect() function.