Sebastian Raschka

Author of "Build a Large Language Model (From Scratch)"

Authored Projects (7)

Starred by

Peter Norvig

Peter Norvig(Author of "Artificial Intelligence: A Modern Approach"; Research Director at Google),

Gabriel Almeida

Gabriel Almeida(Cofounder of Langflow),

Bojan Tunguz

Bojan Tunguz(AI Scientist; Formerly at NVIDIA),

Li Jiang

Li Jiang(Coauthor of AutoGen; Engineer at Microsoft), and

11 more.

LLMs-from-scratch by rasbt

Educational resource for LLM construction in PyTorch

GPT-like LLM implementation from scratch, step-by-step.
Covers pretraining and finetuning techniques.
Includes code for loading pretrained weights.
Contains bonus material on PEFT with LoRA.

Created 2 years ago

Updated 1 day ago

Starred by

Shizhe Diao

Shizhe Diao(Author of LMFlow; Research Scientist at NVIDIA).

reasoning-from-scratch by rasbt

Build a reasoning LLM from scratch

Implement LLM reasoning capabilities step-by-step using PyTorch.
Explore inference-time scaling, reinforcement learning, and distillation techniques.
Focuses on educational purposes, mirroring advanced model development.
Works with pre-trained models like Qwen3 for hands-on learning.

Created 1 year ago

Updated 4 days ago

mini-coding-agent by rasbt

Local coding agent framework for development and explanation

Enables local agent loops with workspace context, structured tools, and session memory.
Integrates Ollama backend with tool validation and approval gates for safe execution.
Features prompt caching, context reduction, and bounded subagent delegation for efficiency.
Supports interactive commands and session resumption in a readable, minimal implementation.

Created 1 week ago

Updated 4 days ago

llm-architecture-gallery by rasbt

Source data for LLM architecture visualization

Source data for an interactive gallery showcasing diverse LLM architectures.
Organizes model details, including image paths, dates, and fact sheets.
Provides structured YAML data for easy integration and display of model information.
Lightweight export of gallery metadata, not the full website implementation.

Created 3 weeks ago

Updated 1 week ago

MachineLearning-QandAI-book by rasbt

Supplementary materials for an ML/AI Q&A book

Covers multi-GPU training paradigms and transformer finetuning.
Explores encoder/decoder LLM differences and vision transformer concepts.
Includes notebooks on data augmentation and model evaluation metrics.
Discusses production, deployment, and predictive performance topics.

Created 3 years ago

Updated 3 months ago

Starred by

Yaowei Zheng

Yaowei Zheng(Author of LLaMA-Factory).

LLM-workshop-2024 by rasbt

Coding workshop for understanding LLM implementation and usage

Build GPT-like LLM from scratch in PyTorch, covering data pipeline and core architecture.
Pretrain the LLM on a small text sample.
Finetune LLMs using open-source libraries and pretrained weights (Llama, Phi, Gemma, Mistral).
Uses LitGPT library for training and finetuning.

Created 1 year ago

Updated 1 year ago

scipy2023-deeplearning by rasbt

PyTorch deep learning workshop material

Multi-GPU training paradigms for deep neural networks.
Fine-tuning large language models (transformers).

Created 2 years ago

Updated 2 years ago

Starred Projects (25)

chainlit by Chainlit

Framework for rapid conversational AI app development

pgarbacki:

wesm:

jordn:

hammer:

Created 3 years ago

Updated 2 days ago

reasoning-from-scratch by rasbt

Build a reasoning LLM from scratch

shizhediao:

Created 1 year ago

Updated 4 days ago

optillm by algorithmicsuperintelligence

Optimizing inference proxy for LLMs

chiphuyen:

casper-hansen:

marcklingen:

dguido:

Created 1 year ago

Updated 3 weeks ago

fastmcp by PrefectHQ

Pythonic SDK for building Model Context Protocol (MCP) servers/clients

hahnbeelee:

didierrlopes:

shyamal-anadkat:

hiyouga:

Created 1 year ago

Updated 1 day ago

verifiers by PrimeIntellect-ai

RL for LLMs in verifiable environments

karpathy:

willccbb:

jeffchuber:

hammer:

Created 1 year ago

Updated 2 days ago

LLMs-from-scratch by rasbt

Educational resource for LLM construction in PyTorch

norvig:

ogabrielluiz:

tunguz:

thinkall:

Created 2 years ago

Updated 1 day ago

KBLaM by microsoft

Knowledge augmentation research paper

Created 1 year ago

Updated 3 days ago

DeepSeek-R1 by deepseek-ai

Reasoning models research paper

shimmyshimmer:

transitive-bullshit:

omarsar:

hiyouga:

Created 1 year ago

Updated 9 months ago

SimpleTuner by bghira

Fine-tuning kit for diffusion models

chiphuyen:

osanseviero:

patrickvonplaten:

Created 2 years ago

Updated 1 day ago

ml-retreat by hesamsheikh

ML learning journal for advanced topics

Created 1 year ago

Updated 7 months ago

litgpt by Lightning-AI

LLM SDK for pretraining, finetuning, and deploying 20+ high-performance LLMs

karpathy:

stefanv:

luiscape:

omarsar:

Created 2 years ago

Updated 1 day ago

ml-roadmap by loganthorneloe

ML learning roadmap for software engineers

omarsar:

Created 2 years ago

Updated 2 weeks ago

phoenix by Arize-ai

AI observability platform for experimentation, evaluation, and troubleshooting

shimmyshimmer:

simonw:

thomwolf:

wsxiaoys:

Created 3 years ago

Updated 1 day ago

lightning-thunder by Lightning-AI

PyTorch compiler for model optimization via source-to-source transformation

lantiga:

williamFalcon:

csarofeen:

hammer:

Created 2 years ago

Updated 5 days ago

lectures by gpu-mode

Lecture series for GPU-accelerated computing

cournape:

stas00:

danielhanchen:

xiezhq-hermann:

Created 2 years ago

Updated 2 months ago

direct-preference-optimization by eric-mitchell

Reference implementation for Direct Preference Optimization (DPO)

chiphuyen:

hiyouga:

huybery:

hammer:

Created 2 years ago

Updated 1 year ago

ml-engineering by stas00

Open book for LLM/VLM training engineers

tobi:

WoosukKwon:

ppwwyyxx:

froystig:

Created 5 years ago

Updated 3 weeks ago

neurips_llm_efficiency_challenge by llm-efficiency-challenge

Competition toolkit for efficient LLM inference on a single GPU

hiyouga:

jph00:

artidoro:

Created 3 years ago

Updated 2 years ago

LLaMA-Cult-and-More by shm007g

LLM resource list for models, datasets, training, and evaluation

shizhediao:

eugeneyan:

Created 3 years ago

Updated 2 years ago

nanoT5 by PiotrNawrot

PyTorch code for T5 pre-training and fine-tuning on a single GPU

victortaelin:

hammer:

ebursztein:

Created 3 years ago

Updated 1 year ago

pytorch-grad-cam by jacobgil

PyTorch SDK for explainable AI in computer vision

hammer:

rwightman:

Jiayi-Pan:

JohannesHa:

Created 8 years ago

Updated 1 week ago

mint by dpressel

Minimal PyTorch library for Transformer tutorials

Created 4 years ago

Updated 3 years ago

deep-significance by Kaleidophon

SDK for statistical significance testing of deep neural networks

Created 5 years ago

Updated 1 year ago

distiller by IntelLabs

Neural network compression research toolkit

xiezhq-hermann:

hanlint:

luiscape:

apsdehal:

Created 8 years ago

Updated 3 years ago

umap by lmcinnes

Dimension reduction technique for visualization and general non-linear reduction

transitive-bullshit:

lewtun:

ankane:

patrick-kidger:

Created 8 years ago

Updated 3 days ago

Feedback? Help us improve.