Lianmin Zheng

Research paper on a disaggregated architecture for LLM serving

Starred by

Woosuk Kwon(Author of vLLM) and

Ying Sheng(Author of SGLang).

Mooncake by kvcache-ai

1.1%

Distributed RL system for LLM reasoning

updated 11 hours ago

AReaL by inclusionAI

2.9%

created 5 months ago

Unified fine-tuning tool for 100+ LLMs & VLMs (ACL 2024)

Starred by

Lysandre Debut(Chief Open-Source Officer at Hugging Face),

Calvin French-Owen(Coounder of Segment), and

8 more.

LLaMA-Factory by hiyouga

0.8%

55k

RL training library for LLMs

updated 3 days ago

Starred by

Lewis Tunstall(Researcher at Hugging Face),

Robert Nishihara(Cofounder of Anyscale; Author of Ray), and

3 more.

verl by volcengine

2.4%

12k

created 9 months ago

Starred by

Tobi Lutke(Cofounder of Shopify),

Michael Han(Cofounder of Unsloth), and

6 more.

DeepSeek-V3 by deepseek-ai

MoE language model research paper with 671B total parameters

98k

created 7 months ago

gemlite by mobiusml

Triton kernels for efficient low-bit matrix multiplication

338

updated 5 days ago

Starred by

Didier Lopes(Founder of OpenBB).

HunyuanVideo by Tencent-Hunyuan

PyTorch code for video generation research

11k

created 8 months ago

updated 3 weeks ago

Starred by

Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems).

MiniCPM by OpenBMB

Ultra-efficient LLMs for end devices, achieving 5x+ speedup

High-performance 4-bit diffusion model inference engine

updated 3 weeks ago

Starred by

Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems) and

Jaret Burkett(Founder of Ostris).

nunchaku by nunchaku-tech

2.1%

created 8 months ago

Model compression toolbox for LLMs and diffusion models

deepcompressor by nunchaku-tech

2.7%

565

Library for efficient structured generation

Starred by

Simon Willison(Author of Django),

Woosuk Kwon(Author of vLLM), and

1 more.

xgrammar by mlc-ai

2.0%

ML SYS learning notes and code

updated 3 days ago

Starred by

Lilian Weng(Cofounder of Thinking Machines Lab).

Awesome-ML-SYS-Tutorial by zhaochenyang20

2.6%

created 8 months ago

RLHF framework for scalable training of large language models

Starred by

Ying Sheng(Author of SGLang),

Nathan Lambert(AI Researcher at AI2), and

2 more.

OpenRLHF by OpenRLHF

0.9%

Attention kernel for plug-and-play inference acceleration

Starred by

Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems) and

Philipp Schmid(DevRel at Google DeepMind).

SageAttention by thu-ml

2.4%

created 10 months ago

updated 1 week ago

Starred by

Ying Sheng(Author of SGLang).

sgl-learning-materials by sgl-project

2.1%

514

Learning materials for SGLang, an efficient LLM serving engine

created 10 months ago

updated 2 weeks ago

Starred by

Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems).

fish-speech by fishaudio

Open-source TTS for multilingual speech synthesis

23k

updated 1 week ago

Starred by

Andrej Karpathy(Founder of Eureka Labs; Formerly at Tesla, OpenAI; Author of CS 231n),

Jiayi Pan(Author of SWE-Gym; AI Researcher at UC Berkeley), and

4 more.

Liger-Kernel by linkedin

Triton kernels for efficient LLM training

Starred by

Woosuk Kwon(Author of vLLM),

Soumith Chintala(Author of PyTorch), and

4 more.

ao by pytorch

PyTorch library for quantization and sparsity in training/inference

updated 17 hours ago

Starred by

Alex Gajewski(Cofounder of San Francisco Compute Company) and

Georgios Konstantopoulos(CTO, General Partner at Paradigm).

ttt-lm-jax by test-time-training

JAX implementation of test-time training RNN research paper

418

Starred by

Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems),

Elie Bursztein(Cybersecurity Lead at Google DeepMind), and

5 more.

RouteLLM by lm-sys

Framework for LLM routing and cost reduction (research paper)

Disaggregated serving system for LLMs

DistServe by LLMServe

0.9%

651

Starred by

Andrej Karpathy(Founder of Eureka Labs; Formerly at Tesla, OpenAI; Author of CS 231n),

Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems), and

12 more.

gpt-fast by pytorch-labs

PyTorch text generation for efficient transformer inference

Starred by

Michael Han(Cofounder of Unsloth),

Ying Sheng(Author of SGLang), and

1 more.

lmdeploy by InternLM

Toolkit for LLM compression, deployment, and serving

Starred by

Jiayi Pan(Author of SWE-Gym; AI Researcher at UC Berkeley).

LLaVA-NeXT by LLaVA-VL

Multimodal model for image, video, and 3D understanding

Automatic LLM benchmark for instruction-tuned models, correlating with human preference

Starred by

Zhuohan Li(Author of vLLM),

Lewis Tunstall(Researcher at Hugging Face), and

2 more.

arena-hard-auto by lmarena

0.9%

888

Starred by

Tobi Lutke(Cofounder of Shopify),

Mckay Wrigley(Founder of Takeoff AI), and

8 more.

llama3 by meta-llama

*Deprecated* minimal example for loading and running Llama 3 models

29k

C/C++ library for local LLM inference

Starred by

Andrej Karpathy(Founder of Eureka Labs; Formerly at Tesla, OpenAI; Author of CS 231n),

Nat Friedman(Former CEO of GitHub), and

31 more.

llama.cpp by ggml-org

0.4%

84k

Starred by

Michael Truell(Cofounder of Cursor),

Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems), and

13 more.

SWE-agent by SWE-agent

Agent for automated software engineering (NeurIPS 2024)

17k

updated 2 days ago

Starred by

Tobi Lutke(Cofounder of Shopify),

Jeffrey Morgan(Cofounder of Ollama), and

22 more.

ollama by ollama

CLI tool for running LLMs locally

148k

Parallel decoder for efficient LLM inference

Starred by

Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems) and

Zhuohan Li(Author of vLLM).

Consistency_LLM by hao-ai-lab

397

updated 8 months ago

Starred by

George Hotz(Author of tinygrad; Founder of the tiny corp, comma.ai),

Fabian Hedin(Cofounder of Lovable), and

11 more.

grok-1 by xai-org

JAX example code for loading and running Grok-1 open-weights model

50k

Kernel library for LLM serving

Starred by

Jeff Hammerbacher(Cofounder of Cloudera),

Zhiqiang Xie(Author of SGLang), and

3 more.

flashinfer by flashinfer-ai

1.4%

Research paper for distributed parallel inference of high-resolution diffusion models

updated 12 hours ago

Starred by

Philipp Schmid(DevRel at Google DeepMind) and

Zhiqiang Xie(Author of SGLang).

distrifuser by mit-han-lab

698

Multimodal autoregressive model for long-context video/text

updated 8 months ago

Starred by

Matei Zaharia(Cofounder of Databricks),

Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems), and

2 more.

LWM by LargeWorldModel

0.0%

Large language model series by Qwen team, Alibaba Cloud

updated 9 months ago

Starred by

Georgi Gerganov(Author of llama.cpp, whisper.cpp),

Travis Fischer(Founder of Agentic), and

3 more.

Qwen3 by QwenLM

1.3%

24k

updated 4 days ago

Starred by

Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems),

Soumith Chintala(Author of PyTorch), and

4 more.

search_with_lepton by leptonai

Conversational search engine demo

Fast serving framework for LLMs and vision language models

Starred by

Elie Bursztein(Cybersecurity Lead at Google DeepMind),

Philipp Schmid(DevRel at Google DeepMind), and

17 more.

sglang by sgl-project

1.1%

16k

Speculative decoding research paper for faster LLM inference

Starred by

Philipp Schmid(DevRel at Google DeepMind),

Omar Sanseviero(DevRel at Google DeepMind), and

1 more.

EAGLE by SafeAILab

1.2%

LLM contamination detector for quantifying rephrased samples

updated 4 days ago

Starred by

Ying Sheng(Author of SGLang) and

Wei-Lin Chiang(Cofounder of LMArena).

llm-decontaminator by lm-sys

306

Starred by

Jerry Liu(Cofounder of LlamaIndex) and

Ying Sheng(Author of SGLang).

ChatRTX by NVIDIA

Demo app for local RAG chatbot on Windows

Starred by

Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems),

Omar Sanseviero(DevRel at Google DeepMind), and

1 more.

S-LoRA by S-LoRA

System for scalable LoRA adapter serving

Starred by

Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems),

Simon Willison(Author of Django), and

3 more.

Yi by 01-ai

Open-source bilingual LLMs trained from scratch

updated 8 months ago

Starred by

Simon Willison(Author of Django),

Eugene Yan(AI Scientist at AWS), and

4 more.

CTranslate2 by OpenNMT

Fast inference engine for Transformer models

created 5 years ago

Starred by

Ying Sheng(Author of SGLang),

Aman Sanger(Cofounder of Cursor), and

3 more.

DeepSpeed-MII by deepspeedai

Python library for high-throughput, low-latency, and cost-effective model inference

Starred by

Victor Taelin(Author of Bend, Kind, HVM).

ChatGLM3 by zai-org

Bilingual chat LLM for complex scenarios (tool use, code execution, agents)

14k

AgentTuning by THUDM

Agent tuning for generalized LLM agent abilities

Starred by

Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems),

Omar Sanseviero(DevRel at Google DeepMind), and

4 more.

TensorRT-LLM by NVIDIA

LLM inference optimization SDK for NVIDIA GPUs

11k

updated 14 hours ago

Starred by

Tobi Lutke(Cofounder of Shopify),

Stas Bekman(Author of Machine Learning Engineering Open Book; Research Engineer at Snowflake), and

20 more.

guidance by guidance-ai

Guidance is a programming paradigm for steering LLMs

21k

updated 22 hours ago

Starred by

Omar Sanseviero(DevRel at Google DeepMind),

Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems), and

2 more.

Medusa by FasterDecoding

Framework for accelerating LLM generation using multiple decoding heads

Starred by

Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems),

Travis Fischer(Founder of Agentic), and

5 more.

codellama by meta-llama

Inference code for CodeLlama models

16k

Attack framework for aligned LLMs, based on a research paper

Starred by

Elie Bursztein(Cybersecurity Lead at Google DeepMind),

Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems), and

2 more.

llm-attacks by llm-attacks

0.4%

Position embeddings research paper

Starred by

Jeremy Howard(Cofounder of fast.ai) and

Ying Sheng(Author of SGLang).

rerope by bojone

375

Python framework for LLM inference and serving

Starred by

Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems),

Philipp Schmid(DevRel at Google DeepMind), and

1 more.

LightLLM by ModelTC

0.7%

Starred by

Ying Sheng(Author of SGLang) and

Simon Willison(Author of Django).

long_llama by CStanKonrad

LLM for long context handling, fine-tuned with Focused Transformer

Starred by

Tobi Lutke(Cofounder of Shopify),

Tim J. Baek(Founder of Open WebUI), and

19 more.

text-generation-inference by huggingface

Rust/Python/gRPC server for fast LLM text generation

10k

updated 1 week ago

Starred by

Michele Castata(President of Replit),

Wei-Lin Chiang(Cofounder of LMArena), and

1 more.

gorilla-cli by gorilla-llm

CLI tool using LLMs to generate commands

Long-context LLM chatbot training and evaluation framework

Starred by

Ying Sheng(Author of SGLang) and

Zhuohan Li(Author of vLLM).

LongChat by DachengLi1

524

LLM serving engine for high-throughput, memory-efficient inference

Starred by

Andrej Karpathy(Founder of Eureka Labs; Formerly at Tesla, OpenAI; Author of CS 231n),

Tobi Lutke(Cofounder of Shopify), and

26 more.

vllm by vllm-project

1.0%

54k

Starred by

Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems),

Woosuk Kwon(Author of vLLM), and

10 more.

WizardLM by nlpxucan

LLMs built using Evol-Instruct for complex instruction following

Starred by

Tobi Lutke(Cofounder of Shopify),

Omar Sanseviero(DevRel at Google DeepMind), and

12 more.

mlc-llm by mlc-ai

Universal LLM deployment engine with ML compilation

21k

updated 5 days ago

Starred by

Travis Fischer(Founder of Agentic),

Patrick von Platen(Core Contributor to Hugging Face Transformers and Diffusers), and

8 more.

LLaVA by haotian-liu

Multimodal assistant with GPT-4 level capabilities

23k

Starred by

Travis Fischer(Founder of Agentic),

Calvin French-Owen(Coounder of Segment), and

3 more.

MiniGPT-4 by Vision-CAIR

Vision-language model for multi-task learning

26k

Starred by

Tobi Lutke(Cofounder of Shopify),

Didier Lopes(Founder of OpenBB), and

12 more.

web-llm by mlc-ai

In-browser LLM inference engine using WebGPU for hardware acceleration

16k

updated 2 months ago

Starred by

Jiayi Pan(Author of SWE-Gym; AI Researcher at UC Berkeley),

Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems), and

5 more.

EasyLM by young-geng

LLM training/finetuning framework in JAX/Flax

Starred by

Georgios Konstantopoulos(CTO, General Partner at Paradigm).

zkml by ddkang

Framework for proofs of ML model execution in ZK-SNARKs

359

Starred by

Zhiqiang Xie(Author of SGLang),

Fabian Hedin(Cofounder of Lovable), and

21 more.

llama by meta-llama

Inference code for Llama 2 models (deprecated)

59k

Starred by

Anastasios Angelopoulos(Cofounder of LMArena),

Omar Sanseviero(DevRel at Google DeepMind), and

20 more.

FastChat by lm-sys

Open platform for training, serving, and evaluating LLM-based chatbots

39k

updated 2 months ago

Starred by

Georgios Konstantopoulos(CTO, General Partner at Paradigm) and

Shawn Wang(Editor of Latent Space).

web-stable-diffusion by mlc-ai

Browser-based Stable Diffusion demo with no server support

Starred by

Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems),

Jonathan Ragan-Kelley(Professor at MIT), and

13 more.

FlexLLMGen by FMInference

High-throughput generation engine for LLMs with limited GPU memory

Post-training quantization research paper for large language models

updated 9 months ago

Starred by

Georgios Konstantopoulos(CTO, General Partner at Paradigm).

smoothquant by mit-han-lab

0.8%

Starred by

Ying Sheng(Author of SGLang) and

Patrick von Platen(Core Contributor to Hugging Face Transformers and Diffusers).

dpm-solver by LuChengTHU

Fast ODE solver for diffusion probabilistic model sampling

Starred by

Mckay Wrigley(Founder of Takeoff AI),

Simon Willison(Author of Django), and

2 more.

GLM-130B by zai-org

Bilingual model for research and evaluation

updated 2 years ago

Starred by

Zhuohan Li(Author of vLLM) and

Woosuk Kwon(Author of vLLM).

compiler-and-arch by KnowingNothing

Compiler/architecture resources for emerging domains

474

Starred by

Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems),

Jiayi Pan(Author of SWE-Gym; AI Researcher at UC Berkeley), and

10 more.

alpa by alpa-projects

Auto-parallelization framework for large-scale neural network training and serving

created 4 years ago

Framework for cloud AI/batch jobs, unifying execution across diverse infrastructure

Starred by

Andrej Karpathy(Founder of Eureka Labs; Formerly at Tesla, OpenAI; Author of CS 231n),

Tobi Lutke(Cofounder of Shopify), and

14 more.

skypilot by skypilot-org

0.4%

created 4 years ago

updated 14 hours ago

Starred by

Shishir Patil(Author of BFCL, Gorilla),

Jeff Hammerbacher(Cofounder of Cloudera), and

2 more.

aqueduct by RunLLM

521

MLOps framework for cloud deployment of LLM/ML workloads

updated 2 years ago

Starred by

Ying Sheng(Author of SGLang) and

Jiayi Pan(Author of SWE-Gym; AI Researcher at UC Berkeley).

paper-reading by mli

Deep learning paper readings

31k

Starred by

Aravind Srinivas(Cofounder of Perplexity),

Georgios Konstantopoulos(CTO, General Partner at Paradigm), and

3 more.

vision_transformer by google-research

Vision Transformer and MLP-Mixer models in JAX/Flax

12k

created 4 years ago

Starred by

Charlie Marsh(Founder of Astral; Author of Ruff, uv),

Jiayi Pan(Author of SWE-Gym; AI Researcher at UC Berkeley), and

10 more.

flax by google

NN library for JAX, designed for flexibility in neural network research

created 5 years ago

updated 16 hours ago

Starred by

Zhiqiang Xie(Author of SGLang).

antares by microsoft

Compiler solution for PyTorch operator optimization on diverse accelerators

478

created 5 years ago

Python package for deep learning on graphs

Starred by

Omar Sanseviero(DevRel at Google DeepMind),

Zhiqiang Xie(Author of SGLang), and

7 more.

dgl by dmlc

0.0%

14k

created 7 years ago

Starred by

Aravind Srinivas(Cofounder of Perplexity),

Travis Fischer(Founder of Agentic), and

8 more.

tvm by apache

Compiler stack for deep learning systems

12k

created 8 years ago

updated 14 hours ago

Starred by

Aravind Srinivas(Cofounder of Perplexity) and

Chenlin Meng(Cofounder of Pika).

MARL-Papers by LantaoYu