This C++ framework provides a comprehensive solution for representing and solving Markov Decision Processes (MDPs) and Partially Observable Markov Decision Processes (POMDPs), targeting AI researchers and practitioners. It offers Python bindings for ease of use and extensibility, enabling efficient implementation and experimentation with various reinforcement learning and planning algorithms.

How It Works

The AI-Toolbox is built in C++ for performance and offers a clean, extensible interface. It supports defining models directly in code or by parsing Cassandra's POMDP format. A key advantage is its ability to integrate with native Python generative models, allowing seamless use of environments like OpenAI Gym. The framework includes a wide array of utilities for combinatorics, polytopes, linear programming, sampling, belief updating, and more, facilitating complex AI problem-solving.

Quick Start & Requirements

• Install: Build from source using CMake.
• Prerequisites: CMake >= 3.12, Boost >= 1.67, Eigen 3.4, lp_solve library, C++20 support (g++-10+).
• Setup: Compilation involves cmake .. and make within a build directory. Python bindings can be enabled with -DMAKE_PYTHON=1.
• Docs: Documentation and tutorials are available.

Highlighted Details

• Supports a wide range of algorithms for MDPs, POMDPs, Bandits, and Factored/Joint Multi-Agent systems.
• Includes parsers for Cassandra's POMDP format.
• Offers Python bindings with extensive utility functions.
• C++ core allows for custom implementations and performance optimization.

Maintenance & Community

The project was published in the Journal of Machine Learning Research (JMLR) in 2020 by Bargiacchi, Roijers, and Nowé. Further community engagement details are not explicitly provided in the README.

Licensing & Compatibility

The project is licensed under the GNU General Public License v3.0 (GPL-3.0-or-later). This license is copyleft, meaning derivative works must also be open-sourced under the same license, potentially restricting commercial use or linking with closed-source software.

Limitations & Caveats

Factored/Joint Multi-Agent Bandits and MDPs are not yet available in the Python bindings. The GPL-3.0 license imposes significant restrictions on commercial use and integration with proprietary software. Building from source requires specific development tools and libraries.