Tensorforce is a modular, TensorFlow-based library for applied reinforcement learning, designed for researchers and practitioners. It offers a flexible, component-based architecture that separates RL algorithms from environment interactions, enabling easier experimentation and deployment of custom RL agents.

How It Works

Tensorforce utilizes a component-based design, allowing users to mix and match various network layers, memory types, policy distributions, and optimizers. All RL logic is implemented within TensorFlow, creating portable computation graphs. This approach facilitates the integration of diverse RL algorithms and simplifies model deployment across different platforms.

Quick Start & Requirements

• Installation: pip3 install tensorforce or pip3 install -e . from a cloned repository.
• Prerequisites: Python 3. TensorFlow is a core dependency. Environment adapters may require additional packages (e.g., ale, gym, retro, vizdoom, carla). GPU usage is not always beneficial for RL and depends on the configuration.
• Documentation: https://github.com/tensorforce/tensorforce

Highlighted Details

• Supports a wide range of network layers (convolutional, recurrent, etc.) and architectures.
• Offers various memory types (batch buffer, replay memory) and policy distributions (Bernoulli, Gaussian, etc.).
• Includes multiple training objectives (policy gradient, value approximation) and optimizers (Adam, evolutionary).
• Provides environment adapters for popular simulators like OpenAI Gym, CARLA, and ViZDoom.

Maintenance & Community

The project is not maintained any longer. It was primarily developed by Alexander Kuhnle, with earlier contributions from Michael Schaarschmidt and Kai Fricke. Parallel execution functionality was contributed by Jean Rabault and Vincent Belus. Community contact is available via Gitter.

Licensing & Compatibility

The project is available under a permissive license, allowing for commercial use and integration with closed-source applications.

Limitations & Caveats

Tensorflow installation on M1 Macs requires a workaround. The README explicitly states the project is "not maintained any longer," indicating a lack of ongoing development or support.