This repository provides an implementation of Augmented Random Search (ARS), a reinforcement learning algorithm for training linear policies in continuous control tasks. It is designed for researchers and practitioners familiar with RL and seeking a competitive baseline or a starting point for exploring policy gradient methods.

How It Works

ARS is a gradient-free optimization method that explores the policy parameter space by sampling random directions and updating the policy based on the performance improvement observed along those directions. This approach simplifies the implementation and avoids the complexities of gradient estimation, making it robust and efficient for certain continuous control problems.

Quick Start & Requirements

• Install: pip install ray
• Prerequisites: Python 3, OpenAI Gym 0.9.3, mujoco-py 0.5.7, MuJoCo Pro 13.1.
• Setup: Start Ray with ray start --head --redis-port=6379 --num-workers=X (where X is the number of CPUs). Set export MKL_NUM_THREADS=1 for optimal performance.
• Training: python code/ars.py --env_name <environment_name>
• Rendering: python code/run_policy.py <policy_path> <environment_name> --render
• Docs: OpenAI Gym, Ray

Highlighted Details

• Implements the ARS algorithm as described in the paper "Simple random search provides a competitive approach to reinforcement learning."
• Supports parallel computation across multiple CPUs using the Ray library.
• Allows customization of hyperparameters, environments, and random seeds.

Maintenance & Community

The project is maintained by modestyachts. No specific community channels or roadmap information are provided in the README.

Licensing & Compatibility

The README does not explicitly state a license. Compatibility for commercial use or closed-source linking is not specified.

Limitations & Caveats

The implementation has specific, older version requirements for dependencies (OpenAI Gym 0.9.3, mujoco-py 0.5.7, MuJoCo Pro 13.1), which may pose challenges for setup and integration with current RL ecosystems.