Andrej Karpathy

A multi-LLM collaborative framework for enhanced question answering

Starred by

Chip Huyen(Author of "AI Engineering", "Designing Machine Learning Systems"),

Jack Lukic(Author of Semantic UI),

Travis Fischer(Founder of Agentic),

Anton Bukov(Cofounder of 1inch Network), and

1 more.

llm-council by karpathy

3.3%

13k

Query multiple LLMs simultaneously, then have them review and rank each other's outputs.
A designated "Chairman" LLM synthesizes peer-reviewed responses into a final, consolidated answer.
Built with FastAPI and React, leveraging OpenRouter for API aggregation and model management.
Facilitates side-by-side LLM evaluation and exploration for complex query resolution.

Created 1 month ago

Self-hosted EPUB reader for LLM interaction

Starred by

Shyamal Anadkat(Research Scientist at OpenAI) and

Travis Fischer(Founder of Agentic).

reader3 by karpathy

1.0%

Enables chapter-by-chapter reading of EPUBs for easy copy-pasting into LLMs.
Lightweight application for accessing and interacting with digital book content.
Simple Python tool using uv for straightforward execution and server setup.
Facilitates collaborative reading by integrating book content with large language models.

Created 1 month ago

Minimalist repo for training/finetuning GPT models

Starred by

Tobi Lutke(Cofounder of Shopify),

Daniel Gross(Cofounder of Safe Superintelligence),

Chip Huyen(Author of "AI Engineering", "Designing Machine Learning Systems"),

Andrew Kane(Author of pgvector), and

44 more.

nanoGPT by karpathy

0.4%

52k

Trains GPT-2 (124M) on OpenWebText using PyTorch DDP.
Includes scripts for training, finetuning, and sampling.
Supports single GPU, multi-GPU, and CPU training.
Uses PyTorch 2.0 torch.compile for potential speedup.

Created 3 years ago

Flatten GitHub repos into a single HTML page

rendergit by karpathy

0.5%

Browse code with syntax highlighting and sidebar navigation for easy exploration.
Copy entire codebases as CXML text for LLM analysis and code review.
Renders markdown and filters out binaries/large files for cleaner output.
Provides a responsive, search-friendly interface for quick code access.

Created 4 months ago

Updated 4 months ago

Starred by

Peter Norvig(Author of "Artificial Intelligence: A Modern Approach"; Research Director at Google),

Alexey Milovidov(Cofounder of Clickhouse),

Didier Lopes(Founder of OpenBB),

Yaowei Zheng(Author of LLaMA-Factory), and

27 more.

llm.c by karpathy

LLM training in pure C/CUDA, no PyTorch needed

29k

Replicates GPT-2/3 pretraining; parallel PyTorch reference.
Faster than PyTorch; simple CPU fp32 reference (~1,000 lines).
Includes CUDA kernels, Flash Attention via cuDNN.
Supports multi-GPU/node training via MPI/NCCL.

Educational resource for neural network development, from basics to advanced models

Updated 6 months ago

Starred by

Zack Li(Cofounder of Nexa AI),

Didier Lopes(Founder of OpenBB),

Elvis Saravia(Founder of DAIR.AI),

Gabriel Almeida(Cofounder of Langflow), and

6 more.

nn-zero-to-hero by karpathy

1.0%

20k

Introduces backpropagation and training via micrograd, using Python and calculus.
Progresses to language modeling with torch.Tensor, training, and loss evaluation.
Explores MLPs, activations, gradients, BatchNorm, and deep network diagnostics.
Builds a GPT from scratch, covering tokenizers and the "Attention is All You Need" paper.

Created 3 years ago

Starred by

Aravind Srinivas(Cofounder of Perplexity),

Nat Friedman(Former CEO of GitHub),

George Hotz(Author of tinygrad; Founder of the tiny corp, comma.ai),

Lilian Weng(Cofounder of Thinking Machines Lab), and

31 more.

minGPT by karpathy

Minimal PyTorch re-implementation for GPT training and inference

23k

Transformer model definition in ~300 lines of code.
Byte Pair Encoder translates text to integer sequences.
Includes GPT-independent PyTorch boilerplate training code.
Demos for addition, character-level language modeling, and text generation.

Created 5 years ago

Starred by

Carol Willing(Core Contributor to CPython, Jupyter) and

Eugene Yan(AI Scientist at AWS).

build-nanogpt by karpathy

Educational resource for building nanoGPT from scratch

Walks through GPT-2 (124M) reproduction via step-by-step Git commits.
Includes a video lecture explaining each commit and code component.
Focuses on language model pretraining, not chat finetuning.

Minimal Llama 3.1 implementation for training, finetuning, and inference

Starred by

Shizhe Diao(Author of LMFlow; Research Scientist at NVIDIA) and

Simon Willison(Coauthor of Django).

nano-llama31 by karpathy

Reproduces reference output, ensuring a bug-free PyTorch adaptation.
Includes early draft of finetuning on Tiny Stories dataset.
Focuses on the 8B base model of Llama 3.1.

Starred by

Vincent Weisser(Cofounder of Prime Intellect),

Robin Huang(Cofounder of Comfy Org),

Jared Palmer(SVP at GitHub; Founder of Turborepo; Author of Formik, TSDX),

Russ d'Sa(Cofounder of LiveKit), and

16 more.

LLM101n by karpathy

Educational resource for building a Storyteller AI LLM

36k

End-to-end construction of a ChatGPT-like web app from scratch.
Covers basics to advanced topics in Python, C, and CUDA.
Includes tokenization, optimization, and distributed training.
Explores finetuning (SFT, RLHF) and multimodal (VQVAE, diffusion).

Minimal BPE encoder/decoder for LLM tokenization

Starred by

Jeff Hammerbacher(Cofounder of Cloudera),

Carol Willing(Core Contributor to CPython, Jupyter),

Chip Huyen(Author of "AI Engineering", "Designing Machine Learning Systems"),

Shizhe Diao(Author of LMFlow; Research Scientist at NVIDIA), and

12 more.

minbpe by karpathy

0.1%

10k

Trains tokenizer vocab and merges on given text corpus.
Encodes text to tokens and decodes tokens back to text.
Implements basic, regex-based, and GPT-4-compatible tokenizers.
Includes special token handling, mirroring tiktoken's GPT-4 behavior.

Character-level language model for generating text

Starred by

Omar Sanseviero(DevRel at Google DeepMind)

Deshraj Yadav(Cofounder of Mem0)

Nathan Lambert(Research Scientist at AI2)

makemore by karpathy

0.6%

Autoregressive model creates new "things" from a text file of training examples.
Supports bigram, MLP, CNN, RNN, LSTM, GRU, and Transformer architectures.
Generates samples or trains models; PyTorch is the only dependency.

Created 3 years ago