ALF (Agent Learning Framework) is a flexible and extensible reinforcement learning framework built on PyTorch, designed for researchers and engineers implementing complex RL algorithms. It offers a wide array of algorithms, from on-policy and off-policy methods to intrinsic motivation and imitation learning, aiming to simplify the development and experimentation of novel RL approaches.

How It Works

ALF is built on PyTorch, emphasizing modularity and ease of implementation for complex RL algorithms. It supports a broad spectrum of RL techniques, including on-policy (A2C, PPO, PPG), off-policy (DDQN, DDPG, SAC, HER), intrinsic motivation (ICM, RND, DIAYN), model-based (MuZero), and offline RL (BC, IQL) methods. This comprehensive library allows users to easily swap components and experiment with different algorithmic combinations.

Quick Start & Requirements

• Installation: pip install -e . after cloning the repository and activating a virtual environment.
• Prerequisites: Python 3.10-3.12, CUDA 11.8 (tested on Ubuntu 22.04/24.04), python3-dev, libboost-all-dev, ninja-build, swig.
• Docker: docker run --gpus all -it horizonrobotics/cuda:11.8.0-py3.11-torch2.2-ubuntu22.04 /bin/bash
• Documentation: https://alf.readthedocs.io/

Highlighted Details

• Supports over 20 distinct RL algorithms, including recent ones like RLPD and Causal BC.
• Includes implementations for model-based RL (MuZero) and intrinsic motivation techniques.
• Provides examples for Atari, MuJoCo, MetaDrive, and Super Mario environments.
• Offers robust support for distributed training (single-node multi-GPU and multi-node multi-GPU).

Maintenance & Community

The project is actively developed by Horizon Robotics and contributors. A contribution guideline is available.

Licensing & Compatibility

The repository does not explicitly state a license in the README. Compatibility for commercial use or closed-source linking is not specified.

Limitations & Caveats

The tutorial is still under construction and some chapters are unfinished. Some older .gin configuration files may be invalid or not updated to the latest PyTorch version.