This repository provides a minimal PyTorch implementation of the Decision Transformer algorithm for reinforcement learning, targeting OpenAI Gym mujoco control tasks. It offers a simplified GPT-based approach, optimized data handling, and includes a Colab notebook for easy experimentation and visualization, making it suitable for researchers and practitioners looking to understand or adapt Decision Transformer for their own RL projects.

How It Works

The implementation utilizes a simple GPT architecture (causal transformer) to model sequential decision-making. It leverages PyTorch's Dataset and DataLoader classes for efficient data management and removes redundant computations for rewards-to-go and state normalization, aiming for faster training cycles compared to the official implementation.

Quick Start & Requirements

• Install: Follow mujoco-py installation instructions. Install D4RL repo and run python3 data/download_d4rl_datasets.py to format and save datasets in the data directory.
• Prerequisites: mujoco-py, d4rl, PyTorch, Python 3.x. GPU recommended for training.
• Resources: Colab notebook provided for easy setup and testing. Training results are provided for 20k updates, with comparisons to the official implementation's 100k updates.
• Links: Colab Notebook

Highlighted Details

• Minimal GPT implementation for Decision Transformer.
• Optimized data handling using PyTorch's Dataset and DataLoader.
• Results provided for 20k updates on HalfCheetah, Hopper, and Walker2d environments, with variance noted.
• Includes scripts for training, testing, and plotting results.

Maintenance & Community

• The repository is authored by Nikhil Barhate.
• No specific community channels (Discord/Slack) or roadmap are mentioned in the README.

Licensing & Compatibility

• The repository does not explicitly state a license in the provided README.
• Compatibility for commercial use or closed-source linking is not specified.

Limitations & Caveats

The provided results are based on 20k training updates and are not directly comparable to the official implementation's 100k updates, serving only as rough reference points. The README does not specify a license, which may impact commercial adoption.