This repository provides the code for the book "Build a Reasoning Model (From Scratch)," aimed at educating users on the inner workings of reasoning Large Language Models (LLMs). It offers a hands-on, step-by-step approach to developing a functional reasoning model, mirroring techniques used in advanced LLMs, making it ideal for engineers and researchers seeking a deep understanding of LLM reasoning capabilities.

How It Works

The project guides users through enhancing a pre-trained base LLM (specifically Qwen3) with reasoning abilities. It covers methods such as inference-time scaling, reinforcement learning, and distillation, implemented incrementally in code. This practical, code-centric methodology demystifies complex LLM reasoning concepts, allowing users to build and experiment with their own models.

Quick Start & Requirements

• Installation: Clone the repository using git clone --depth 1 https://github.com/rasbt/reasoning-from-scratch.git. Chapter 2 offers detailed guidance on Python and package management.
• Prerequisites: Python, with specific recommendations in setup_tips.
• Hardware: Consumer hardware (CPU/GPU) is generally sufficient for most chapters. GPU is recommended for Chapters 5 and 6 for replicating results.
• Resources: Links to the book on Manning and Amazon are provided.

Highlighted Details

• Companion code for the book "Build a Reasoning Model (From Scratch)."
• Focuses on practical implementation of reasoning techniques like inference-time scaling, reinforcement learning, and distillation.
• Utilizes the Qwen3 LLM as a base model for enhancements.
• Includes exercises and solutions within chapter subfolders.

Maintenance & Community

Feedback and questions are welcomed via Manning Discussion Forum or GitHub Discussions. Contributions to the main chapter code are not accepted to maintain consistency with the printed book.

Licensing & Compatibility

The repository's license is not explicitly stated in the provided README. Compatibility for commercial use or closed-source linking is not detailed.

Limitations & Caveats

The README indicates that Chapters 3-7 are "TBA" (To Be Announced), suggesting that the full scope of the code and content may still be under development or not yet released. The repository's primary purpose is educational, mirroring a printed book, which may limit its applicability as a production-ready library.