This repository curates a comprehensive and continually updated collection of research papers on Decision Transformers (DT), a novel approach to offline Reinforcement Learning (RL). It targets researchers and engineers seeking to understand and advance the state-of-the-art in sequence modeling for decision-making tasks, offering a structured overview of DT's evolution and applications.

How It Works

Decision Transformer reframes offline Reinforcement Learning as a conditional sequence modeling problem, leveraging the power of causal transformer architectures. Instead of traditional RL methods that rely on bootstrapping value functions, DT conditions a transformer model on desired future returns, past states, and actions to autoregressively generate future actions. This approach bypasses the need for explicit value estimation, avoids short-sighted behaviors often induced by reward discounting, and benefits from the scalability and multi-modal adaptability characteristic of transformer models prevalent in natural language processing and computer vision.

Quick Start & Requirements

This repository serves as a curated, continually updated list of resources for Decision Transformer (DT) research. It does not provide a direct software installation or execution guide. Instead, it links to papers, many of which specify their own experimental environments (e.g., D4RL, MuJoCo, Atari, SMAC) and potential prerequisites like GPUs or specific libraries, which are detailed within individual paper entries.

Highlighted Details

• The repository meticulously categorizes Decision Transformer research papers by conference (e.g., NeurIPS, ICML, ICLR, IROS) and topic, including surveys and recent arXiv preprints.
• Each entry typically includes paper titles, links, authors, publishers, keywords, code availability (where applicable), and the experimental environments used.
• A significant focus is placed on advancements in "In-Context Reinforcement Learning" (ICRL), where transformers learn policies directly from prompts without gradient updates.
• Various DT adaptations are highlighted, such as those incorporating state-space models (MetaMamba), graph neural networks, Q-learning, multi-agent systems, and applications in robotics, natural language understanding, and network intrusion detection.

Maintenance & Community

The repository encourages community contributions to expand its curated list. While it invites users to "follow and star" and provides a general pointer for contribution instructions, it does not list specific community channels (e.g., Discord, Slack) or active maintainer details.

Licensing & Compatibility

Awesome Decision Transformer is released under the Apache 2.0 license. This permissive license generally allows for commercial use and integration into closed-source projects.

Limitations & Caveats

As a curated list, the repository itself has no functional limitations. However, the research it documents indicates that Decision Transformers may struggle in highly stochastic environments. Furthermore, the effectiveness and optimal use cases for Decision Transformers compared to traditional RL methods are still active areas of investigation, suggesting they are not a universal solution for all offline RL problems.