Sebastian Raschka

Author of "Build a Large Language Model (From Scratch)"

Authored Projects (7)

Starred by

Peter Norvig

Peter Norvig(Author of "Artificial Intelligence: A Modern Approach"; Research Director at Google),

Didier Lopes

Didier Lopes(Founder of OpenBB),

Gabriel Almeida

Gabriel Almeida(Cofounder of Langflow),

Bojan Tunguz

Bojan Tunguz(AI Scientist; Formerly at NVIDIA), and

12 more.

LLMs-from-scratch by rasbt

Educational resource for LLM construction in PyTorch

GPT-like LLM implementation from scratch, step-by-step.
Covers pretraining and finetuning techniques.
Includes code for loading pretrained weights.
Contains bonus material on PEFT with LoRA.

Created 3 years ago

Updated 2 weeks ago

Starred by

Shizhe Diao

Shizhe Diao(Author of LMFlow; Research Scientist at NVIDIA).

reasoning-from-scratch by rasbt

Build a reasoning LLM from scratch

Implement LLM reasoning capabilities step-by-step using PyTorch.
Explore inference-time scaling, reinforcement learning, and distillation techniques.
Focuses on educational purposes, mirroring advanced model development.
Works with pre-trained models like Qwen3 for hands-on learning.

Created 1 year ago

Updated 3 weeks ago

llm-architecture-gallery by rasbt

Source data for LLM architecture visualization

Source data for an interactive gallery showcasing diverse LLM architectures.
Organizes model details, including image paths, dates, and fact sheets.
Provides structured YAML data for easy integration and display of model information.
Lightweight export of gallery metadata, not the full website implementation.

Created 5 months ago

Updated 1 month ago

mini-coding-agent by rasbt

Local coding agent framework for development and explanation

Enables local agent loops with workspace context, structured tools, and session memory.
Integrates Ollama backend with tool validation and approval gates for safe execution.
Features prompt caching, context reduction, and bounded subagent delegation for efficiency.
Supports interactive commands and session resumption in a readable, minimal implementation.

Created 4 months ago

Updated 4 months ago

MachineLearning-QandAI-book by rasbt

Supplementary materials for an ML/AI Q&A book

Covers multi-GPU training paradigms and transformer finetuning.
Explores encoder/decoder LLM differences and vision transformer concepts.
Includes notebooks on data augmentation and model evaluation metrics.
Discusses production, deployment, and predictive performance topics.

Created 3 years ago

Updated 8 months ago

Starred by

Yaowei Zheng

Yaowei Zheng(Author of LLaMA-Factory).

LLM-workshop-2024 by rasbt

Coding workshop for understanding LLM implementation and usage

Build GPT-like LLM from scratch in PyTorch, covering data pipeline and core architecture.
Pretrain the LLM on a small text sample.
Finetune LLMs using open-source libraries and pretrained weights (Llama, Phi, Gemma, Mistral).
Uses LitGPT library for training and finetuning.

Created 2 years ago

Updated 1 year ago

scipy2023-deeplearning by rasbt

PyTorch deep learning workshop material

Multi-GPU training paradigms for deep neural networks.
Fine-tuning large language models (transformers).

Created 3 years ago

Updated 2 years ago

Starred Projects (28)

qwen-code by QwenLM

AI coding agent for complex codebases

hammer:

ekzhu:

developit:

JustinLin610:

Created 1 year ago

Updated 16 hours ago

chainlit by Chainlit

Framework for rapid conversational AI app development

pgarbacki:

wesm:

jordn:

hammer:

Created 3 years ago

Updated 3 weeks ago

reasoning-from-scratch by rasbt

Build a reasoning LLM from scratch

shizhediao:

Created 1 year ago

Updated 3 weeks ago

optillm by algorithmicsuperintelligence

Optimizing inference proxy for LLMs

chiphuyen:

casper-hansen:

marcklingen:

dguido:

Created 2 years ago

Updated 1 month ago

fastmcp by PrefectHQ

Pythonic SDK for building Model Context Protocol (MCP) servers/clients

hahnbeelee:

didierrlopes:

shyamal-anadkat:

hiyouga:

Created 1 year ago

Updated 21 hours ago

verifiers by PrimeIntellect-ai

RL for LLMs in verifiable environments

karpathy:

willccbb:

jeffchuber:

hammer:

Created 1 year ago

Updated 20 hours ago

LLMs-from-scratch by rasbt

Educational resource for LLM construction in PyTorch

norvig:

didierrlopes:

ogabrielluiz:

tunguz:

Created 3 years ago

Updated 2 weeks ago

KBLaM by microsoft

Knowledge augmentation research paper

Created 2 years ago

Updated 1 month ago

ExcalidrawZ by chocoford

Native drawing app with AI assistance and cross-platform sync

t3dotgg:

Created 3 years ago

Updated 1 week ago

DeepSeek-R1 by deepseek-ai

Reasoning models research paper

shimmyshimmer:

transitive-bullshit:

omarsar:

hiyouga:

Created 1 year ago

Updated 1 year ago

SimpleTuner by bghira

Fine-tuning kit for diffusion models

chiphuyen:

osanseviero:

patrickvonplaten:

Created 3 years ago

Updated 1 day ago

ml-retreat by hesamsheikh

ML learning journal for advanced topics

Created 1 year ago

Updated 11 months ago

litgpt by Lightning-AI

LLM SDK for pretraining, finetuning, and deploying 20+ high-performance LLMs

karpathy:

stefanv:

luiscape:

omarsar:

Created 3 years ago

Updated 1 month ago

ai-for-software-engineers by loganthorneloe

ML learning roadmap for software engineers

omarsar:

Created 2 years ago

Updated 1 month ago

marimo by marimo-team

AI-native reactive notebooks for reproducible Python development

tobi:

bcherny:

didierrlopes:

ekzhang:

Created 3 years ago

Updated 18 hours ago

phoenix by Arize-ai

AI observability platform for experimentation, evaluation, and troubleshooting

shimmyshimmer:

simonw:

thomwolf:

wsxiaoys:

Created 3 years ago

Updated 17 hours ago

lightning-thunder by Lightning-AI

PyTorch compiler for model optimization via source-to-source transformation

lantiga:

williamFalcon:

csarofeen:

hammer:

Created 2 years ago

Updated 3 weeks ago

lectures by gpu-mode

Lecture series for GPU-accelerated computing

cournape:

stas00:

danielhanchen:

xiezhq-hermann:

Created 2 years ago

Updated 2 months ago

direct-preference-optimization by eric-mitchell

Reference implementation for Direct Preference Optimization (DPO)

chiphuyen:

hiyouga:

huybery:

hammer:

Created 3 years ago

Updated 2 years ago

ml-engineering by stas00

Open book for LLM/VLM training engineers

tobi:

stas00:

WoosukKwon:

ppwwyyxx:

Created 6 years ago

Updated 2 days ago

neurips_llm_efficiency_challenge by llm-efficiency-challenge

Competition toolkit for efficient LLM inference on a single GPU

hiyouga:

jph00:

artidoro:

Created 3 years ago

Updated 2 years ago

LLaMA-Cult-and-More by shm007g

LLM resource list for models, datasets, training, and evaluation

shizhediao:

eugeneyan:

Created 3 years ago

Updated 3 years ago

nanoT5 by PiotrNawrot

PyTorch code for T5 pre-training and fine-tuning on a single GPU

victortaelin:

hammer:

ebursztein:

Created 3 years ago

Updated 2 years ago

pytorch-grad-cam by jacobgil

PyTorch SDK for explainable AI in computer vision

hammer:

rwightman:

Jiayi-Pan:

JohannesHa:

Created 9 years ago

Updated 1 month ago

mint by dpressel

Minimal PyTorch library for Transformer tutorials

Created 4 years ago

Updated 4 years ago

deep-significance by Kaleidophon

SDK for statistical significance testing of deep neural networks

Created 5 years ago

Updated 2 years ago

distiller by IntelLabs

Neural network compression research toolkit

xiezhq-hermann:

hanlint:

luiscape:

apsdehal:

Created 8 years ago

Updated 3 years ago

umap by lmcinnes

Dimension reduction technique for visualization and general non-linear reduction

transitive-bullshit:

lewtun:

ankane:

patrick-kidger:

Created 9 years ago

Updated 2 weeks ago

Feedback? Help us improve.