OmniSafe is a comprehensive, modular Python framework for accelerating research in Safe Reinforcement Learning (SafeRL). It provides a standardized benchmark and an out-of-the-box toolkit for implementing, training, and evaluating SafeRL algorithms, targeting researchers and developers in the field.

How It Works

OmniSafe employs a highly modular design with Adapter and Wrapper components to facilitate seamless integration of diverse SafeRL algorithms and environments. It leverages torch.distributed for high-performance parallel computing, enabling environment-level asynchronous parallelism and agent asynchronous learning to expedite training and improve stability.

Quick Start & Requirements

• Install: pip install omnisafe
• Prerequisites: Python 3.8+, PyTorch 1.10+. Officially tested on Linux (Python 3.8-3.10) and macOS (M1/M2). Windows support is community-contributed.
• Setup: Installation via PyPI is straightforward. For advanced use or development, cloning the repository and installing from source (pip install -e .) within a conda environment is recommended.
• Resources: Colab notebooks are available for quick exploration.

Highlighted Details

• Implements over 30 SafeRL algorithms across On-Policy, Off-Policy, Model-Based, and Offline categories, including recent papers from top conferences (NeurIPS, ICML, ICLR, AAAI).
• Supports a wide range of environments, including Safety-Gymnasium and custom environments via a flexible interface.
• Provides a Command Line Interface (CLI) for benchmarking, evaluation, and training, simplifying experimental workflows.
• Includes extensive tutorials and documentation, with community contributions for multi-language support.

Maintenance & Community

The project is primarily developed by the SafeRL research team at PKU-Alignment, led by Prof. Yaodong Yang. Community support is available via GitHub issues.

Licensing & Compatibility

Released under the Apache License 2.0, permitting commercial use and integration with closed-source projects.

Limitations & Caveats

While robust, official support for Windows is not provided. The framework is actively developed, and users should refer to the changelog for potential breaking changes.