Jiayi Pan

Author of SWE-Gym; MTS at xAI

Authored Projects (1)

Starred by

George Hotz

George Hotz(Author of tinygrad; Founder of the tiny corp, comma.ai),

Andrej Karpathy

Andrej Karpathy(Founder of Eureka Labs; Formerly at Tesla, OpenAI; Author of CS 231n),

Vincent Weisser

Vincent Weisser(Cofounder of Prime Intellect),

Chip Huyen

Chip Huyen(Author of "AI Engineering", "Designing Machine Learning Systems"), and

8 more.

TinyZero by Jiayi-Pan

Minimal reproduction of DeepSeek R1 Zero for countdown/multiplication tasks

Uses reinforcement learning to develop self-verification and search abilities.
Built upon veRL framework; supports Qwen2.5 series base models.
Training scripts use vLLM with XFORMERS attention backend.

Created 9 months ago

Updated 6 months ago

Starred Projects (117)

l0 by cmriat

Scalable pipeline for training general-purpose agents

Created 4 months ago

Updated 4 months ago

terminal-bench by laude-institute

Benchmark for LLM agents in real terminal environments

ogabrielluiz:

transitive-bullshit:

vincentweisser:

hammer:

Created 9 months ago

Updated 1 day ago

SkyRL by NovaSky-AI

RL training pipeline for multi-turn tool use LLMs, optimized for real-world tasks

lewtun:

hiyouga:

WoosukKwon:

zhuohan123:

Created 6 months ago

Updated 14 hours ago

elimination_game by lechmazur

LLM benchmark for social reasoning, strategy, and deception

Created 8 months ago

Updated 2 months ago

frontier-evals by openai

Code repo for OpenAI preparedness evals

willccbb:

swyxio:

shizhediao:

JustinLin610:

Created 7 months ago

Updated 4 days ago

YuE by multimodal-art-projection

Open-source tool for generating full songs from lyrics

wsxiaoys:

snowzurfer:

Created 9 months ago

Updated 5 months ago

RL4VLM by RL4VLM

Research paper for fine-tuning VLMs as decision-making agents via RL

Created 1 year ago

Updated 10 months ago

LLM-Training-Puzzles by srush

Hands-on puzzles for large language model training

albertfgu:

willccbb:

hiyouga:

patrickvonplaten:

Created 2 years ago

Updated 1 year ago

TinyZero by Jiayi-Pan

Minimal reproduction of DeepSeek R1 Zero for countdown/multiplication tasks

geohot:

karpathy:

vincentweisser:

chiphuyen:

Created 9 months ago

Updated 6 months ago

DeepSeek-R1 by deepseek-ai

Reasoning models research paper

shimmyshimmer:

rasbt:

transitive-bullshit:

omarsar:

Created 9 months ago

Updated 4 months ago

Kimi-k1.5 by MoonshotAI

Research paper on scaling reinforcement learning with LLMs

didierrlopes:

lewtun:

shizhediao:

vincentweisser:

Created 9 months ago

Updated 8 months ago

levanter by marin-community

Framework for training foundation models with JAX

pcmoritz:

thomwolf:

chiphuyen:

Edward-Sun:

Created 3 years ago

Updated 12 hours ago

PRIME by PRIME-RL

Scalable RL solution for advanced reasoning of language models

vincentweisser:

lewtun:

omarsar:

philschmid:

Created 10 months ago

Updated 7 months ago

flash-attention by Dao-AILab

Fast, memory-efficient attention implementation

karpathy:

zhiyuan8:

alexchen4ai:

gakonst:

Created 3 years ago

Updated 23 hours ago

ring-flash-attention by zhuzilin

FlashAttention extension for ring attention

willccbb:

pgarbacki:

CodeCreator:

MishaLaskin:

Created 1 year ago

Updated 1 month ago

DeepSeek-V3 by deepseek-ai

MoE language model research paper with 671B total parameters

tobi:

shimmyshimmer:

jiamings:

syrusakbary:

Created 10 months ago

Updated 2 months ago

SWE-Gym by SWE-Gym

Environment for training software engineering agents

vincentweisser:

hammer:

shizhediao:

winglian:

Created 1 year ago

Updated 3 months ago

skypilot by skypilot-org

Framework for cloud AI/batch jobs, unifying execution across diverse infrastructure

karpathy:

tobi:

amin3141:

luiscape:

Created 4 years ago

Updated 9 hours ago

moatless-tools by aorwall

CLI tool for LLM-based code editing experiments on large codebases

paul-gauthier:

Created 2 years ago

Updated 2 months ago

Qwen3-Coder by QwenLM

Code LLM for code completion, generation, and assistant use cases

hugs:

victortaelin:

luiscape:

cournape:

Created 1 year ago

Updated 3 months ago

Liger-Kernel by linkedin

Triton kernels for efficient LLM training

karpathy:

pgarbacki:

WoosukKwon:

merrymercy:

Created 1 year ago

Updated 9 hours ago

OpenHands by OpenHands

AI platform for software development agents

truell20:

pgarbacki:

khou22:

yiranwu0:

Created 1 year ago

Updated 11 hours ago

WebRL by THUDM

Framework for training LLM web agents using reinforcement learning

omarsar:

Created 1 year ago

Updated 5 months ago

verl by volcengine

RL training library for LLMs

WoosukKwon:

hammer:

yiranwu0:

luiscape:

Created 1 year ago

Updated 9 hours ago

swarm by openai

Multi-agent orchestration framework for lightweight agent coordination

vincentweisser:

rodrigosnader:

achowdhery:

xiezhq-hermann:

Created 1 year ago

Updated 7 months ago

mle-bench by openai

Benchmark for evaluating AI agents on machine learning engineering tasks

JohannesHa:

thinkall:

casper-hansen:

shizhediao:

Created 1 year ago

Updated 2 weeks ago

litellm by BerriAI

SDK/proxy for calling 100+ LLM APIs using the OpenAI format

tobi:

mxcl:

chiphuyen:

amin3141:

Created 2 years ago

Updated 10 hours ago

NeMo-Aligner by NVIDIA

Toolkit for efficient model alignment

winglian:

thomwolf:

lewtun:

shizhediao:

Created 2 years ago

Updated 4 weeks ago

Pai-Megatron-Patch by alibaba

Training toolkit for LLMs & VLMs using Megatron

zhyncs:

Created 2 years ago

Updated 4 days ago

EasyDeL by erfanzar

Framework for streamlined JAX model training/serving on TPU/GPU

hammer:

mattjj:

Created 2 years ago

Updated 1 day ago

ngram by EurekaLabsAI

N-gram language model for character-level name generation

Created 1 year ago

Updated 1 year ago

LLaMA-Factory by hiyouga

Unified fine-tuning tool for 100+ LLMs & VLMs (ACL 2024)

patrickvonplaten:

alexchen4ai:

teetone:

lysandrejik:

Created 2 years ago

Updated 9 hours ago

self-correction-llm-papers by teacherpeterpan

Collection of research papers for self-correcting LLMs with automated feedback

Created 2 years ago

Updated 1 year ago

sglang by sgl-project

Fast serving framework for LLMs and vision language models

samlambert:

ebursztein:

philschmid:

pankajroark:

Created 1 year ago

Updated 9 hours ago

digirl by DigiRL-agent

Research code for device-control agents via autonomous reinforcement learning

thomwolf:

Created 1 year ago

Updated 8 months ago

LLaVA-NeXT by LLaVA-VL

Multimodal model for image, video, and 3D understanding

merrymercy:

Created 1 year ago

Updated 1 month ago

DeepSeek-Coder-V2 by deepseek-ai

Open-source code language model comparable to GPT4-Turbo

didierrlopes:

chiphuyen:

pgarbacki:

guoday:

Created 1 year ago

Updated 1 year ago

spinningup by openai

Educational resource for learning deep reinforcement learning

pgarbacki:

nirga:

codekansas:

spartee:

Created 7 years ago

Updated 1 year ago

unified-io-2 by allenai

Unified-IO 2 code for training, inference, and demo

shizhediao:

jwyang:

natolambert:

teknium1:

Created 1 year ago

Updated 1 year ago

xla by pytorch

PyTorch on XLA devices

aravindsrinivas:

WoosukKwon:

comaniac:

ankane:

Created 7 years ago

Updated 1 day ago

DeepSeek-V2 by deepseek-ai

MoE language model for research/API use

gakonst:

jiamings:

zhyncs:

Created 1 year ago

Updated 1 year ago

torchtitan by pytorch

PyTorch platform for generative AI model training research

karpathy:

pgarbacki:

lewtun:

zhuohan123:

Created 1 year ago

Updated 10 hours ago

SWE-bench by SWE-bench

Benchmark for evaluating LLMs on real-world GitHub issues

pgarbacki:

shizhediao:

transitive-bullshit:

aksh-at:

Created 2 years ago

Updated 3 weeks ago

prismatic-vlms by TRI-ML

VLM codebase for training visually-conditioned language models

hiyouga:

hammer:

Created 1 year ago

Updated 1 year ago

BrowserGym by ServiceNow

Gym environment for web task automation research

shizhediao:

gregpr07:

Created 1 year ago

Updated 1 day ago

HighPerfLLMs2024 by rwitten

Jax course for high-performance LLM construction

froystig:

hammer:

mattjj:

Created 1 year ago

Updated 1 year ago

test by hendrycks

Research paper for measuring multitask language understanding

zhiyuan8:

simonw:

chiphuyen:

osanseviero:

Created 5 years ago

Updated 2 years ago

unsloth by unslothai

Finetuning tool for LLMs, targeting speed and memory efficiency

tobi:

karpathy:

danielhanchen:

shimmyshimmer:

Created 1 year ago

Updated 9 hours ago

prm800k by openai

Dataset of LLM solutions to math problems with step-level correctness labels

hiyouga:

jph00:

pgarbacki:

transitive-bullshit:

Created 2 years ago

Updated 2 years ago

LWM by LargeWorldModel

Multimodal autoregressive model for long-context video/text

mateiz:

chiphuyen:

pgarbacki:

omarsar:

Created 1 year ago

Updated 1 year ago

vector-quantize-pytorch by lucidrains

PyTorch library for vector quantization techniques

yang-song:

codekansas:

Created 5 years ago

Updated 2 days ago

flax by google

NN library for JAX, designed for flexibility in neural network research

charliermarsh:

codekansas:

jaredpalmer:

soldni:

Created 5 years ago

Updated 16 hours ago

TinyLlama by jzhang38

Tiny pretraining project for a 1.1B Llama model

karpathy:

geohot:

tobi:

chiphuyen:

Created 2 years ago

Updated 1 year ago

webdataset by webdataset

High-performance I/O system for large deep learning problems, strong PyTorch support

soldni:

chiphuyen:

hammer:

theophilegervet:

Created 6 years ago

Updated 4 months ago

webarena by web-arena-x

Web environment for autonomous agent development

Ying1123:

transitive-bullshit:

Created 2 years ago

Updated 1 month ago

audiocraft by facebookresearch

PyTorch library for audio processing and generation research

hiyouga:

jn2clark:

calvinfo:

jrk:

Created 2 years ago

Updated 7 months ago

encodec by facebookresearch

Neural audio codec for high-fidelity compression research

theophilegervet:

osanseviero:

aangelopoulos:

parano:

Created 3 years ago

Updated 1 year ago

weak-to-strong by openai

Weak-to-strong generalization research paper implementation

jaredpalmer:

Edward-Sun:

vincentweisser:

omarsar:

Created 1 year ago

Updated 1 year ago

OpenRLHF by OpenRLHF

RLHF framework for scalable training of large language models

beyang:

parano:

vincentweisser:

binarybana:

Created 2 years ago

Updated 3 days ago

EasyLM by young-geng

LLM training/finetuning framework in JAX/Flax

chiphuyen:

shizhediao:

zhuohan123:

hammer:

Created 2 years ago

Updated 1 year ago

tpu-starter by ayaka14732

TPU guide for JAX-based ML workflows on Google Cloud

philschmid:

infwinston:

osanseviero:

Created 3 years ago

Updated 1 year ago

taming-transformers by CompVis

Image synthesis research paper using transformers

karpathy:

codekansas:

stellaathena:

patrickvonplaten:

Created 4 years ago

Updated 1 year ago

bitsandbytes by bitsandbytes-foundation

PyTorch library for k-bit quantization, enabling accessible LLMs

tjbck:

alexchen4ai:

danielhanchen:

shimmyshimmer:

Created 4 years ago

Updated 19 hours ago

llamafile by mozilla-ai

Single-file LLM distribution and runtime via `llama.cpp` and Cosmopolitan Libc

karpathy:

anildash:

mholt:

hahnbeelee:

Created 2 years ago

Updated 22 hours ago

generative-models by Stability-AI

Generative models SDK for video, image, and 3D synthesis research

hugs:

luiscape:

pgarbacki:

osanseviero:

Created 2 years ago

Updated 19 hours ago

vllm by vllm-project

LLM serving engine for high-throughput, memory-efficient inference

karpathy:

clmnt:

tobi:

danielhanchen:

Created 2 years ago

Updated 9 hours ago

Qwen-VL by QwenLM

Vision-language model for multimodal understanding, localization, and text reading

pgarbacki:

omarsar:

huybery:

jinze1994:

Created 2 years ago

Updated 1 year ago

Mind2Web by OSU-NLP-Group

Web agent benchmark dataset and models (NeurIPS'23 Spotlight paper)

ValentaTomas:

transitive-bullshit:

Created 2 years ago

Updated 7 months ago

awesome-llm-powered-agent by hyp1231

Curated list of LLM-powered agent resources (papers, repos, blogs)

gregpr07:

omarsar:

huybery:

Created 2 years ago

Updated 6 months ago

SEED by AILab-CVC

Multimodal LLM research paper with visual tokenization

Created 2 years ago

Updated 1 year ago

MetaGPT by FoundationAgents

Multi-agent framework for collaborative AI software development

pathak22:

omarsar:

guypod:

didierrlopes:

Created 2 years ago

Updated 1 month ago

alpa by alpa-projects

Auto-parallelization framework for large-scale neural network training and serving

chiphuyen:

transitive-bullshit:

soldni:

hammer:

Created 4 years ago

Updated 1 year ago

DeepSpeed by deepspeedai

Deep learning optimization library for distributed training and inference

aravindsrinivas:

ValentaTomas:

winglian:

stas00:

Created 5 years ago

Updated 22 hours ago

FastChat by lm-sys

Open platform for training, serving, and evaluating LLM-based chatbots

zjasper666:

aangelopoulos:

osanseviero:

natolambert:

Created 2 years ago

Updated 5 months ago

llm-foundry by mosaicml

LLM training code for Databricks foundation models

hiyouga:

vincentweisser:

hammer:

Sanger2000:

Created 2 years ago

Updated 1 week ago

lightning-hydra-template by ashleve

ML experimentation template using PyTorch Lightning + Hydra

claforte:

chuanli11:

hammer:

tridao:

Created 5 years ago

Updated 1 year ago

llm-numbers by ray-project

LLM developer's reference for key numbers

cournape:

chiphuyen:

Ying1123:

pirroh:

Created 2 years ago

Updated 1 year ago

rl by pytorch

PyTorch library for reinforcement learning research

evhub:

natolambert:

Created 3 years ago

Updated 23 hours ago

Gymnasium by Farama-Foundation

Python API standard for single-agent reinforcement learning environments

vincentweisser:

hiyouga:

thomwolf:

chiphuyen:

Created 3 years ago

Updated 6 days ago

pythia by EleutherAI

LLM suite for interpretability, learning dynamics, ethics, and transparency research

hammer:

osanseviero:

calvinfo:

vincentweisser:

Created 3 years ago

Updated 4 months ago

AutoGPT by Significant-Gravitas

AI agent platform for building, deploying, and running autonomous workflows

lilianweng:

julien-c:

deshraj:

sxyu:

Created 2 years ago

Updated 9 hours ago

obsidian-smart-connections by brianpetro

Obsidian plugin for AI-assisted note-taking and knowledge graph creation

tobi:

stammy:

osanseviero:

paul-gauthier:

Created 2 years ago

Updated 20 hours ago

peft by huggingface

Parameter-efficient fine-tuning (PEFT) library

tobi:

gakonst:

chiphuyen:

Ying1123:

Created 2 years ago

Updated 1 day ago

universe by openai

Deprecated platform for measuring/training AI across diverse applications

codekansas:

junxiaosong:

evhub:

steve8708:

Created 9 years ago

Updated 7 years ago

allenact by allenai

Open-source framework for embodied AI research

jwyang:

Created 5 years ago

Updated 2 months ago

stable-diffusion-webui by AUTOMATIC1111

Web UI for Stable Diffusion

clmnt:

k06a:

vincentweisser:

tjbck:

Created 3 years ago

Updated 3 weeks ago

fromage by kohjingyu

Multimodal model for grounding language models to images

binarybana:

transitive-bullshit:

Edward-Sun:

osanseviero:

Created 2 years ago

Updated 2 years ago

whisperX by m-bain

ASR tool for accurate, batched, word-level Whisper transcriptions

gakonst:

paul-gauthier:

nicolaerusan:

KyleAMathews:

Created 2 years ago

Updated 2 weeks ago

x-transformers by lucidrains

Transformer library with extensive experimental features

chiphuyen:

hammer:

chenlin9:

WoosukKwon:

Created 5 years ago

Updated 2 days ago

unilm by microsoft

Foundation models for language, vision, speech, and multimodal tasks

jph00:

AlexCheema:

chiphuyen:

osanseviero:

Created 6 years ago

Updated 4 months ago

petals by bigscience-workshop

Run LLMs at home, BitTorrent-style

osanseviero:

chiphuyen:

vincentweisser:

ishaan-jaff:

Created 3 years ago

Updated 1 year ago

ray by ray-project

AI compute engine for scaling Python and AI applications

beyang:

hsbt:

gregpr07:

hiyouga:

Created 9 years ago

Updated 10 hours ago

langchain by langchain-ai

Framework for building LLM-powered applications

karpathy:

MagMueller:

gregpr07:

willingc:

Created 3 years ago

Updated 12 hours ago

Awesome-LLM-Robotics by GT-RIPL

Curated list of papers using LLMs/multimodal models for robotics/RL

vnivargi:

natolambert:

huybery:

Created 3 years ago

Updated 2 weeks ago

pytorch-image-models by huggingface

PyTorch image model collection with training, eval, and inference scripts

clmnt:

karpathy:

osanseviero:

thomwolf:

Created 6 years ago

Updated 3 days ago

clearml by clearml

MLOps suite for experiment tracking, automation, and data management

jiamings:

rodrigosnader:

hammer:

luiscape:

Created 6 years ago

Updated 2 days ago

diffusers by huggingface

PyTorch/Flax library for diffusion model research and applications

karpathy:

clmnt:

vincentweisser:

hammer:

Created 3 years ago

Updated 9 hours ago

stablediffusion by Stability-AI

Latent diffusion model for high-resolution image synthesis

robinjhuang:

chiphuyen:

vincentweisser:

ogabrielluiz:

Created 2 years ago

Updated 4 months ago

Megatron-LM by NVIDIA

Framework for training transformer models at scale

jiamings:

gravicle:

alexchen4ai:

parasj:

Created 6 years ago

Updated 9 hours ago

stable-diffusion by CompVis

Latent text-to-image diffusion model

gaearon:

patrickvonplaten:

codekansas:

ValentaTomas:

Created 3 years ago

Updated 1 year ago

OFA by OFA-Sys

Unified sequence-to-sequence model for cross-modality, vision, and language tasks

Edward-Sun:

hammer:

shizhediao:

jinze1994:

Created 3 years ago

Updated 1 year ago

afterglow by Deep-Symmetry

Live-coding lighting controller for algorithmic light shows

Created 10 years ago

Updated 7 months ago

fairseq by facebookresearch

Sequence modeling toolkit for translation, language modeling, and text generation research

lilianweng:

aravindsrinivas:

tjbck:

pathak22:

Created 8 years ago

Updated 1 month ago

Video-Pre-Training by openai

Minecraft agent learning via video pretraining research paper

agermanidis:

JohannesHa:

Created 3 years ago

Updated 2 months ago

X-VLM by zengyan-97

Vision-language model for multi-grained alignment (ICML 2022 paper)

shizhediao:

Created 4 years ago

Updated 2 years ago

lxmert by airsplay

PyTorch code for cross-modality representation learning via Transformers

aravindsrinivas:

thomwolf:

Created 6 years ago

Updated 3 years ago

pytorch-grad-cam by jacobgil

PyTorch SDK for explainable AI in computer vision

hammer:

rwightman:

rasbt:

JohannesHa:

Created 8 years ago

Updated 7 months ago

metaseq by facebookresearch

Codebase for large-scale transformer model development and deployment

chiphuyen:

gakonst:

xiezhq-hermann:

Ying1123:

Created 3 years ago

Updated 1 year ago

disco-diffusion by alembics

AI art and animation notebook

chenlin9:

taranjeet:

patrickvonplaten:

soumith:

Created 3 years ago

Updated 2 years ago

METER by zdou0830

Multimodal framework for vision-and-language transformer research

shizhediao:

JustinLin610:

Created 4 years ago

Updated 3 years ago

glide-text2im by openai

Text-conditional image synthesis model from research paper

ShengjiaZhao:

Edward-Sun:

patrickvonplaten:

JustinLin610:

Created 3 years ago

Updated 1 year ago

transformers by huggingface

ML library for pretrained model inference and training

clmnt:

lilianweng:

karpathy:

tjbck:

Created 7 years ago

Updated 9 hours ago

CausalNLP_Papers by zhijing-jin

NLP resource for causality research papers

Created 4 years ago

Updated 5 months ago

CLIP by openai

Image-text matching model for zero-shot prediction

aravindsrinivas:

transitive-bullshit:

Edward-Sun:

mholt:

Created 4 years ago

Updated 1 year ago

paper-reading by mli

Deep learning paper readings

parano:

omarsar:

Ying1123:

merrymercy:

Created 4 years ago

Updated 7 months ago

best-of-ml-python by lukasmasuch

Curated list of Python ML libraries, ranked by quality score

vnivargi:

tunguz:

omarsar:

evhub:

Created 5 years ago

Updated 12 hours ago

gpt-3 by openai

Research paper on large language model few-shot learning

aravindsrinivas:

shizhediao:

evhub:

kentcdodds:

Created 5 years ago

Updated 5 years ago

Feedback? Help us improve.