Summary

This open-source curriculum addresses the gap between foundational Reinforcement Learning (RL) and cutting-edge AI systems like LLM alignment, RLVR, and multi-modal agents. It targets ML engineers, researchers, and LLM practitioners seeking a practical, code-first understanding of RL, enabling them to build and debug advanced AI agents.

How It Works

The project employs a "practice-first" methodology, grounding abstract RL concepts in runnable code, intuitive training phenomena, and debugging insights before formal mathematical exposition. It progresses from classic control problems (CartPole) through core RL algorithms (DQN, PPO), into LLM post-training (RLHF, DPO, GRPO), RLVR, Agentic RL (tool use), and extends to VLM RL and embodied intelligence.

Quick Start & Requirements

• Docs: Clone the repo, npm install, then npm run dev to serve locally.
• Code: Navigate to code/, create and activate a Python virtual environment (python -m venv .venv, source .venv/bin/activate), then pip install -r requirements.txt (or chapter-specific requirements).
• Prerequisites: Python, PyTorch proficiency, basic ML math. Node.js >= 18.0.0 for docs. GPU support is highly recommended for advanced examples and is actively sought by contributors.
• Links: Online Docs

Highlighted Details

• Hands-on, chapter-aligned Python code examples for core RL algorithms and advanced topics.
• Emphasis on debugging RL failures (reward hacking, KL drift, OOM) as integral to learning.
• Covers modern RL applications: LLM alignment (RLHF, DPO, GRPO), RLVR, Agentic RL (tool-use, code agents), VLM RL, and Embodied RL.
• Features code maps linking formulas to implementations and visualizations of training metrics with diagnostic explanations.

Maintenance & Community

The project is actively maintained by WalkingLabs and contributors, with a clear roadmap for ongoing development. Community interaction is facilitated via a WeChat Group. Contributions are welcomed for improving clarity, accuracy, and reproducibility.

Licensing & Compatibility

Released under the CC BY-NC-SA 4.0 license. This license permits non-commercial sharing and modification with attribution and a share-alike clause for derivatives. Commercial use or integration into proprietary systems is restricted.

Limitations & Caveats

The curriculum is AI-assisted and undergoing active iteration, meaning potential factual or code errors may exist. Some chapters are marked as unstable. The project actively seeks GPU contributions, suggesting that running advanced examples may require significant hardware resources.