Sebastian Raschka

Author of "Build a Large Language Model (From Scratch)"

Authored Projects (5)

reasoning-from-scratch by rasbt

Build a reasoning LLM from scratch

Implement LLM reasoning capabilities step-by-step using PyTorch.
Explore inference-time scaling, reinforcement learning, and distillation techniques.
Focuses on educational purposes, mirroring advanced model development.
Works with pre-trained models like Qwen3 for hands-on learning.

Created 6 months ago

Updated 21 hours ago

Starred by

Peter Norvig

Peter Norvig(Author of "Artificial Intelligence: A Modern Approach"; Research Director at Google),

Gabriel Almeida

Gabriel Almeida(Cofounder of Langflow),

Bojan Tunguz

Bojan Tunguz(AI Scientist; Formerly at NVIDIA),

Li Jiang

Li Jiang(Coauthor of AutoGen; Engineer at Microsoft), and

9 more.

LLMs-from-scratch by rasbt

Educational resource for LLM construction in PyTorch

GPT-like LLM implementation from scratch, step-by-step.
Covers pretraining and finetuning techniques.
Includes code for loading pretrained weights.
Contains bonus material on PEFT with LoRA.

Created 2 years ago

Updated 1 day ago

MachineLearning-QandAI-book by rasbt

Supplementary materials for an ML/AI Q&A book

Covers multi-GPU training paradigms and transformer finetuning.
Explores encoder/decoder LLM differences and vision transformer concepts.
Includes notebooks on data augmentation and model evaluation metrics.
Discusses production, deployment, and predictive performance topics.

Created 2 years ago

Updated 1 month ago

Starred by

Yaowei Zheng

Yaowei Zheng(Author of LLaMA-Factory).

LLM-workshop-2024 by rasbt

Coding workshop for understanding LLM implementation and usage

Build GPT-like LLM from scratch in PyTorch, covering data pipeline and core architecture.
Pretrain the LLM on a small text sample.
Finetune LLMs using open-source libraries and pretrained weights (Llama, Phi, Gemma, Mistral).
Uses LitGPT library for training and finetuning.

Created 1 year ago

Updated 8 months ago

scipy2023-deeplearning by rasbt

PyTorch deep learning workshop material

Multi-GPU training paradigms for deep neural networks.
Fine-tuning large language models (transformers).

Created 2 years ago

Updated 2 years ago

Starred Projects (21)

verifiers by willccbb

RL for LLMs in verifiable environments

karpathy:

jeffchuber:

hammer:

didierrlopes:

Created 7 months ago

Updated 21 hours ago

LLMs-from-scratch by rasbt

Educational resource for LLM construction in PyTorch

norvig:

ogabrielluiz:

tunguz:

thinkall:

Created 2 years ago

Updated 1 day ago

reasoning-from-scratch by rasbt

Build a reasoning LLM from scratch

Created 6 months ago

Updated 21 hours ago

KBLaM by microsoft

Knowledge augmentation research paper

Created 1 year ago

Updated 1 week ago

DeepSeek-R1 by deepseek-ai

Reasoning models research paper

shimmyshimmer:

transitive-bullshit:

omarsar:

hiyouga:

Created 8 months ago

Updated 2 months ago

SimpleTuner by bghira

Fine-tuning kit for diffusion models

chiphuyen:

osanseviero:

patrickvonplaten:

Created 2 years ago

Updated 18 hours ago

ml-retreat by hesamsheikh

ML learning journal for advanced topics

Created 1 year ago

Updated 1 week ago

litgpt by Lightning-AI

LLM SDK for pretraining, finetuning, and deploying 20+ high-performance LLMs

karpathy:

stefanv:

luiscape:

omarsar:

Created 2 years ago

Updated 5 days ago

essential-ml-roadmap by loganthorneloe

ML learning roadmap for software engineers

omarsar:

Created 1 year ago

Updated 1 day ago

phoenix by Arize-ai

AI observability platform for experimentation, evaluation, and troubleshooting

shimmyshimmer:

simonw:

thomwolf:

wsxiaoys:

Created 2 years ago

Updated 15 hours ago

lightning-thunder by Lightning-AI

PyTorch compiler for model optimization via source-to-source transformation

lantiga:

williamFalcon:

csarofeen:

hammer:

Created 1 year ago

Updated 13 hours ago

lectures by gpu-mode

Lecture series for GPU-accelerated computing

cournape:

stas00:

danielhanchen:

xiezhq-hermann:

Created 1 year ago

Updated 4 days ago

direct-preference-optimization by eric-mitchell

Reference implementation for Direct Preference Optimization (DPO)

chiphuyen:

hiyouga:

huybery:

hammer:

Created 2 years ago

Updated 1 year ago

ml-engineering by stas00

Open book for LLM/VLM training engineers

tobi:

froystig:

jrk:

casper-hansen:

Created 5 years ago

Updated 23 hours ago

neurips_llm_efficiency_challenge by llm-efficiency-challenge

Competition toolkit for efficient LLM inference on a single GPU

hiyouga:

jph00:

artidoro:

Created 2 years ago

Updated 1 year ago

LLaMA-Cult-and-More by shm007g

LLM resource list for models, datasets, training, and evaluation

shizhediao:

eugeneyan:

Created 2 years ago

Updated 2 years ago

nanoT5 by PiotrNawrot

PyTorch code for T5 pre-training and fine-tuning on a single GPU

victortaelin:

hammer:

ebursztein:

Created 2 years ago

Updated 1 year ago

pytorch-grad-cam by jacobgil

PyTorch SDK for explainable AI in computer vision

hammer:

rwightman:

Jiayi-Pan:

JohannesHa:

Created 8 years ago

Updated 5 months ago

mint by dpressel

Minimal PyTorch library for Transformer tutorials

Created 3 years ago

Updated 3 years ago

deep-significance by Kaleidophon

SDK for statistical significance testing of deep neural networks

Created 4 years ago

Updated 1 year ago

umap by lmcinnes

Dimension reduction technique for visualization and general non-linear reduction

transitive-bullshit:

lewtun:

ankane:

patrick-kidger:

Created 8 years ago

Updated 4 days ago

Feedback? Help us improve.