This repository provides the official codebase for Reinforcement Learning with Augmented Data (RAD), a method designed to improve sample efficiency in image-based reinforcement learning tasks. It is targeted at researchers and practitioners in deep reinforcement learning, offering a robust framework for experimenting with various data augmentation strategies.

How It Works

RAD enhances standard reinforcement learning algorithms by incorporating data augmentation techniques directly into the training loop. It leverages contrastive learning principles, specifically a variant of CURL, to learn representations from augmented observations. This approach allows the agent to generalize better and learn from fewer interactions with the environment, particularly when dealing with high-dimensional visual inputs.

Quick Start & Requirements

• Install dependencies via conda env create -f conda_env.yml.
• Requires a CUDA-enabled GPU.
• Example training command: CUDA_VISIBLE_DEVICES=0 python train.py --domain_name cartpole --task_name swingup --encoder_type pixel --work_dir ./tmp/cartpole --action_repeat 8 --num_eval_episodes 10 --pre_transform_image_size 100 --image_size 84 --agent rad_sac --frame_stack 3 --data_augs flip --seed 23 --critic_lr 1e-3 --actor_lr 1e-3 --eval_freq 10000 --batch_size 128 --num_train_steps 200000
• Data augmentation chains can be specified with a hyphen, e.g., --data_augs crop-rotate-flip.
• Visualize progress with TensorBoard: tensorboard --logdir log --port 6006.

Highlighted Details

• Implements various data augmentations (flip, crop, rotate, etc.) that can be chained.
• Supports image-based observations with configurable preprocessing.
• Based on the RAD-SAC algorithm.
• Includes scripts for ProcGen and OpenAI Gym experiments.

Maintenance & Community

The project is associated with Michael Laskin, Kimin Lee, Adam Stooke, Lerrel Pinto, Pieter Abbeel, and Aravind Srinivas. Further community or roadmap information is not detailed in the README.

Licensing & Compatibility

The repository's license is not explicitly stated in the provided README. Users should verify licensing for commercial or closed-source integration.

Limitations & Caveats

The README focuses on setup and training for specific tasks; broader environment support or advanced configuration options may require deeper inspection of the codebase. The project was forked from CURL, implying potential dependencies or architectural similarities.