Yineng Zhang

Inference Lead for SGLang at LMSYS

Starred Projects (89)

agents.md by openai

A format for guiding coding agents

transitive-bullshit:

romainhuet:

Created 1 week ago

Updated 1 week ago

axolotl by axolotl-ai-cloud

CLI tool for streamlined post-training of AI models

tobi:

patrickvonplaten:

amiruci:

luiscape:

Created 2 years ago

Updated 12 hours ago

DiffSynth-Studio by modelscope

Open-source project for diffusion model exploration

rodrigosnader:

doriandarko:

Created 1 year ago

Updated 18 hours ago

coze-studio by coze-dev

AI agent development platform with visual tools

Created 2 months ago

Updated 11 hours ago

Step3 by stepfun-ai

Multimodal reasoning model for efficient vision-language tasks

Created 1 month ago

Updated 2 weeks ago

SpecForge by sgl-project

Train speculative decoding models for faster inference

xiezhq-hermann:

Ying1123:

Created 2 months ago

Updated 10 hours ago

chatterbox by resemble-ai

Open-source TTS model

dguido:

joewalnes:

abidlabs:

jiamings:

Created 4 months ago

Updated 3 weeks ago

Kimi-K2 by MoonshotAI

State-of-the-art MoE language model

merrymercy:

shizhediao:

vincentweisser:

hiyouga:

Created 1 month ago

Updated 1 day ago

gemini-cli by google-gemini

AI agent for terminal workflows

lucidrains:

khou22:

koraykv:

amin3141:

Created 4 months ago

Updated 11 hours ago

trae-agent by bytedance

LLM-powered CLI for software engineering tasks

yiranwu0:

syrusakbary:

transitive-bullshit:

omarsar:

Created 2 months ago

Updated 10 hours ago

Math-Verify by huggingface

Math evaluator for LLM outputs in mathematical tasks

shizhediao:

thomwolf:

hiyouga:

lewtun:

Created 7 months ago

Updated 1 month ago

NeMo-Skills by NVIDIA

LLM skill-improvement pipelines for synthetic data generation, training, and evaluation

winglian:

Created 1 year ago

Updated 18 hours ago

RL-Factory by Simple-Efficient

RL post-training framework for agentic learning

Created 3 months ago

Updated 17 hours ago

minions by HazyResearch

Communication protocol for cost-efficient LLM collaboration

JohannesHa:

w4nderlust:

jmorganca:

mchiang0610:

Created 6 months ago

Updated 18 hours ago

slime by THUDM

LLM post-training framework for RL scaling

casper-hansen:

simonw:

merrymercy:

hiyouga:

Created 2 months ago

Updated 12 hours ago

Hunyuan3D-2.1 by Tencent-Hunyuan

Image to 3D asset generation with PBR materials

zhuohan123:

Created 2 months ago

Updated 1 day ago

dots.llm1 by rednote-hilab

MoE model for research

Created 3 months ago

Updated 1 week ago

ROLL by alibaba

RL library for large language models

yiranwu0:

pcmoritz:

Created 3 months ago

Updated 2 days ago

cosmos-reason1 by nvidia-cosmos

Multimodal LLMs for physical common sense and embodied decisions

Created 5 months ago

Updated 2 days ago

torchtune by pytorch

PyTorch library for LLM post-training and experimentation

lewtun:

patrickvonplaten:

JustinLin610:

shizhediao:

Created 1 year ago

Updated 1 day ago

unsloth by unslothai

Finetuning tool for LLMs, targeting speed and memory efficiency

tobi:

karpathy:

gregpr07:

jeffchuber:

Created 1 year ago

Updated 1 day ago

RL by NVIDIA-NeMo

Scalable toolkit for efficient model reinforcement learning

hammer:

pcmoritz:

willccbb:

Created 5 months ago

Updated 12 hours ago

VeOmni by ByteDance-Seed

Framework for scaling multimodal model training across accelerators

JustinLin610:

Created 5 months ago

Updated 2 days ago

MiMo by XiaomiMiMo

LLM for reasoning, pre-trained and post-trained for math/code tasks

alexchen4ai:

WoosukKwon:

omarsar:

Created 4 months ago

Updated 2 months ago

dia by nari-labs

TTS model for ultra-realistic dialogue generation

shimmyshimmer:

alexchen4ai:

omarsar:

abidlabs:

Created 4 months ago

Updated 1 month ago

SpargeAttn by thu-ml

Training-free sparse attention for model inference acceleration

Created 6 months ago

Updated 2 weeks ago

profile-data by deepseek-ai

Profiling data for computation-communication overlap analysis

Created 6 months ago

Updated 5 months ago

aider by Aider-AI

AI pair programming in your terminal

bcherny:

nat:

hugs:

yiranwu0:

Created 2 years ago

Updated 2 weeks ago

Triton-distributed by ByteDance-Seed

Distributed compiler for computation-communication overlapping, based on Triton

antiagainst:

Created 4 months ago

Updated 3 days ago

AReaL by inclusionAI

Distributed RL system for LLM reasoning

yiranwu0:

casper-hansen:

merrymercy:

Created 6 months ago

Updated 1 day ago

dynamo by ai-dynamo

Inference framework for distributed generative AI model serving

willingc:

chiphuyen:

luiscape:

WoosukKwon:

Created 5 months ago

Updated 12 hours ago

QwQ by QwenLM

Reasoning model for complex problem-solving, based on Qwen2.5

shimmyshimmer:

huybery:

JustinLin610:

Created 5 months ago

Updated 5 months ago

3FS by deepseek-ai

Distributed file system for AI training/inference workloads

joewalnes:

wesm:

alexey-milovidov:

flaque:

Created 6 months ago

Updated 2 days ago

EPLB by deepseek-ai

Load balancer for expert-parallel MoE models

parano:

comaniac:

Created 6 months ago

Updated 5 months ago

DualPipe by deepseek-ai

Pipeline parallelism algorithm for training large models

luiscape:

JohannesHa:

Created 6 months ago

Updated 5 months ago

DeepGEMM by deepseek-ai

CUDA library for efficient FP8 GEMM kernels with fine-grained scaling

patrickvonplaten:

parano:

shyamal-anadkat:

hammer:

Created 6 months ago

Updated 18 hours ago

DeepEP by deepseek-ai

Expert-parallel communication library for MoE, targeting high-throughput and low-latency

parano:

JustinLin610:

CodeCreator:

calvinfo:

Created 6 months ago

Updated 13 hours ago

FlashMLA by deepseek-ai

Efficient CUDA kernels for MLA decoding

hammer:

jmorganca:

shizhediao:

suquark:

Created 6 months ago

Updated 1 day ago

Moonlight by MoonshotAI

MoE model and optimizer tech report

jph00:

lucidrains:

Created 6 months ago

Updated 3 weeks ago

open-infra-index by deepseek-ai

AI infrastructure tools for efficient AGI development

lattner:

vincentweisser:

hammer:

pankajroark:

Created 6 months ago

Updated 3 months ago

ktransformers by kvcache-ai

Framework for LLM inference optimization experimentation

luiscape:

patrickvonplaten:

JustinLin610:

shimmyshimmer:

Created 1 year ago

Updated 3 weeks ago

skypilot by skypilot-org

Framework for cloud AI/batch jobs, unifying execution across diverse infrastructure

karpathy:

tobi:

amin3141:

luiscape:

Created 4 years ago

Updated 11 hours ago

ZhiLight by zhihu

LLM inference engine for Llama and variants, optimized for PCIe GPUs

hiyouga:

Created 8 months ago

Updated 1 month ago

Janus by deepseek-ai

Unified multimodal model research paper for understanding and generation

jiamings:

omarsar:

taranjeet:

xiaofan-luan:

Created 10 months ago

Updated 6 months ago

UI-TARS by bytedance

Multimodal agent for GUI interaction in virtual worlds (research paper)

jiamings:

hugs:

jmorganca:

omarsar:

Created 7 months ago

Updated 1 week ago

rtp-llm by alibaba

LLM inference engine for diverse applications

Created 1 year ago

Updated 1 month ago

tilelang by tile-ai

DSL for high-performance GPU/CPU kernel development (GEMM, attention, etc.)

luiscape:

zhuohan123:

yang-song:

Created 10 months ago

Updated 14 hours ago

DeepSeek-R1 by deepseek-ai

Reasoning models research paper

shimmyshimmer:

rasbt:

transitive-bullshit:

omarsar:

Created 7 months ago

Updated 2 months ago

MInference by microsoft

Framework for long-context LLM inference speedup via sparse attention

pgarbacki:

Created 1 year ago

Updated 2 weeks ago

flash-linear-attention by fla-org

Efficient Torch/Triton implementations for linear attention models

hiyouga:

xiezhq-hermann:

yang-song:

winglian:

Created 1 year ago

Updated 2 days ago

MiniMax-01 by MiniMax-AI

Large language & vision-language models based on linear attention

sxyu:

transitive-bullshit:

omarsar:

hiyouga:

Created 7 months ago

Updated 1 month ago

dash-infer by modelscope

LLM inference engine for optimized performance across diverse hardware

Created 1 year ago

Updated 3 weeks ago

smoothquant by mit-han-lab

Post-training quantization research paper for large language models

zhiyuan8:

gakonst:

forresti:

Edward-Sun:

Created 2 years ago

Updated 1 year ago

picotron by huggingface

Minimalist distributed training framework for educational use

karpathy:

transitive-bullshit:

srush:

pgarbacki:

Created 11 months ago

Updated 2 days ago

hqq by mobiusml

Model quantizer for fast, accurate post-training quantization, skipping calibration

danielhanchen:

winglian:

hiyouga:

osanseviero:

Created 1 year ago

Updated 1 week ago

torchtitan by pytorch

PyTorch platform for generative AI model training research

karpathy:

zhuohan123:

CodeCreator:

luiscape:

Created 1 year ago

Updated 11 hours ago

LightLLM by ModelTC

Python framework for LLM inference and serving

chiphuyen:

pgarbacki:

JustinLin610:

winglian:

Created 2 years ago

Updated 11 hours ago

cline by cline

VS Code extension for autonomous coding agent

snowzurfer:

ishaan-jaff:

chiphuyen:

huybery:

Created 1 year ago

Updated 13 hours ago

DeepSeek-V3 by deepseek-ai

MoE language model research paper with 671B total parameters

tobi:

shimmyshimmer:

jiamings:

syrusakbary:

Created 8 months ago

Updated 17 hours ago

infinity by michaelfeil

REST API for high-throughput, low-latency embedding and reranking

luiscape:

ishaan-jaff:

ankane:

jn2clark:

Created 1 year ago

Updated 12 hours ago

Star-Attention by NVIDIA

PyTorch code for efficient LLM inference on long sequences

Created 9 months ago

Updated 2 months ago

Qwen3-Coder by QwenLM

Code LLM for code completion, generation, and assistant use cases

hugs:

victortaelin:

luiscape:

cournape:

Created 1 year ago

Updated 4 weeks ago

xgrammar by mlc-ai

Library for efficient structured generation

merrymercy:

ogabrielluiz:

youkaichao:

simonw:

Created 1 year ago

Updated 4 days ago

prompt-lookup-decoding by apoorvumang

Decoding method for faster LLM generation

stas00:

jph00:

casper-hansen:

ggerganov:

Created 1 year ago

Updated 1 year ago

verl by volcengine

RL training library for LLMs

hammer:

yiranwu0:

luiscape:

binarybana:

Created 10 months ago

Updated 15 hours ago

duo-attention by mit-han-lab

Framework for efficient long-context LLM inference

Created 10 months ago

Updated 6 months ago

corenet by apple

DNN toolkit for training standard and novel models

transitive-bullshit:

omarsar:

t3dotgg:

hammer:

Created 1 year ago

Updated 4 days ago

tensorrtllm_backend by triton-inference-server

Triton backend for serving TensorRT-LLM models

NikolaBorisov:

guberti:

Ying1123:

tuhins:

Created 1 year ago

Updated 1 day ago

Pai-Megatron-Patch by alibaba

Training toolkit for LLMs & VLMs using Megatron

Jiayi-Pan:

Created 2 years ago

Updated 1 day ago

whisper by openai

Speech recognition model for multilingual transcription/translation

bcherny:

karpathy:

ogabrielluiz:

swyxio:

Created 2 years ago

Updated 1 week ago

TensorRT-LLM by NVIDIA

LLM inference optimization SDK for NVIDIA GPUs

hammer:

shizhediao:

luiscape:

parano:

Created 2 years ago

Updated 11 hours ago

EAGLE by SafeAILab

Speculative decoding research paper for faster LLM inference

shizhediao:

philschmid:

pgarbacki:

JustinLin610:

Created 1 year ago

Updated 4 days ago

Nanoflow by efeslab

LLM serving framework for high throughput

WoosukKwon:

Ying1123:

zhuohan123:

Created 1 year ago

Updated 2 weeks ago

truss by basetenlabs

Model deployment tool for productionizing AI/ML models

apsdehal:

ogabrielluiz:

hammer:

pankajroark:

Created 3 years ago

Updated 19 hours ago

mscclpp by microsoft

GPU-driven communication stack for scalable AI applications

jrk:

WoosukKwon:

Created 2 years ago

Updated 15 hours ago

ao by pytorch

PyTorch library for quantization and sparsity in training/inference

parano:

willccbb:

alexchen4ai:

luiscape:

Created 1 year ago

Updated 16 hours ago

simple-evals by openai

Lightweight library for evaluating language models

patrickvonplaten:

simonw:

philschmid:

Ying1123:

Created 1 year ago

Updated 4 weeks ago

sglang by sgl-project

Fast serving framework for LLMs and vision language models

samlambert:

ebursztein:

philschmid:

pankajroark:

Created 1 year ago

Updated 11 hours ago

QuaRot by spcl

Code for a NeurIPS 2024 research paper on LLM quantization

jph00:

Created 1 year ago

Updated 9 months ago

FlagGems by FlagOpen

Operator library for LLM training/inference, implemented in Triton

Created 1 year ago

Updated 1 day ago

ring-attention-pytorch by lucidrains

Pytorch impl of Ring Attention for near-infinite context

sxyu:

vincentweisser:

Created 1 year ago

Updated 3 months ago

Mooncake by kvcache-ai

Research paper on a disaggregated architecture for LLM serving

luiscape:

merrymercy:

WoosukKwon:

hiyouga:

Created 1 year ago

Updated 12 hours ago

DeepSeek-V2 by deepseek-ai

MoE language model for research/API use

gakonst:

jiamings:

Jiayi-Pan:

Created 1 year ago

Updated 11 months ago

llama3 by meta-llama

*Deprecated* minimal example for loading and running Llama 3 models

tobi:

mckaywrigley:

osanseviero:

simonw:

Created 1 year ago

Updated 7 months ago

mlx by ml-explore

Array framework for machine learning on Apple silicon

geohot:

karpathy:

jmorganca:

chiphuyen:

Created 1 year ago

Updated 17 hours ago

flashinfer by flashinfer-ai

Kernel library for LLM serving

hammer:

JustinLin610:

luiscape:

youkaichao:

Created 2 years ago

Updated 14 hours ago

flash-attention by Dao-AILab

Fast, memory-efficient attention implementation

karpathy:

Jiayi-Pan:

zhiyuan8:

alexchen4ai:

Created 3 years ago

Updated 1 day ago

Medusa by FasterDecoding

Framework for accelerating LLM generation using multiple decoding heads

osanseviero:

parano:

luiscape:

chiphuyen:

Created 1 year ago

Updated 1 year ago

lmdeploy by InternLM

Toolkit for LLM compression, deployment, and serving

shimmyshimmer:

wsxiaoys:

luiscape:

jn2clark:

Created 2 years ago

Updated 1 day ago

Feedback? Help us improve.