This repository provides executable Python lectures for Stanford's CS336: Language Models from Scratch course. It allows students to step through lecture content as code, inspect variables, and run code samples directly within an IDE like VSCode, offering a more interactive and debuggable learning experience than traditional slide decks.

How It Works

Lectures are implemented as Python scripts. Each script is designed to be executed line-by-line or in segments, mirroring the progression of a lecture. This approach enables users to interactively explore code, debug logic, and inspect intermediate states of variables, providing a deeper understanding of the underlying concepts in language models.

Quick Start & Requirements

• Install: pip install -r requirements.txt
• Prerequisites: Python 3.12, git clone the repository.
• Optional Configuration: OPENAI_API_KEY, TOGETHER_API_KEY, WANDB_API_KEY environment variables.
• GPU: While code runs on CPU, many parts depend on GPU (A100, H100 tested).
• Usage: Run python lecture_XX.py or step through in VSCode (F5, F11, F10). View logs in view.html.

Highlighted Details

• Executable lectures offer interactive code execution and debugging.
• Supports stepping through code, inspecting variables, and running samples.
• Tested on Python 3.12, CPUs, A100s, and H100s.
• Includes an execution log viewer (view.html) for non-renderable images.

Maintenance & Community

No specific community links or maintenance details are provided in the README.

Licensing & Compatibility

The repository's license is not specified in the README.

Limitations & Caveats

The view.html component for viewing logs and images is described as "a bit clunky." While core functionality runs on CPU, many lecture components require GPU access.