Woosuk Kwon

@WoosukKwon

Author of vLLM

View on GitHub

Starred Projects (54)

Starred by

Taranjeet Singh(Cofounder of Mem0),

Zhiqiang Xie(Author of SGLang), and

1 more.

openevolve by codelion

2.4%

Coding agent for scientific/algorithmic discovery, based on AlphaEvolve paper

created 2 months ago

updated 2 days ago

Starred by

Eric Zhang(Founding Engineer at Modal).

nano-vllm by GeeeekExplorer

2.3%

Lightweight vLLM implementation from scratch

created 1 month ago

updated 1 month ago

Starred by

Chris Van Pelt(Cofounder of Weights & Biases) and

Jeff Hammerbacher(Cofounder of Cloudera).

llm-d by llm-d

2.5%

Kubernetes-native framework for distributed LLM inference

created 3 months ago

updated 4 days ago

Starred by

Carol Willing(Core Contributor to CPython, Jupyter),

Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems), and

3 more.

dynamo by ai-dynamo

1.1%

Inference framework for distributed generative AI model serving

created 5 months ago

updated 13 hours ago

MiMo by XiaomiMiMo

0.6%

LLM for reasoning, pre-trained and post-trained for math/code tasks

created 3 months ago

updated 1 month ago

Starred by

Simon Willison(Author of Django),

Thomas Wolf(Cofounder of Hugging Face), and

4 more.

chatgpt_system_prompt by LouisShark

0.2%

GPT system prompt collection for prompt engineering and security education

created 1 year ago

updated 3 days ago

fairseq2 by facebookresearch

0.3%

Sequence modeling toolkit for content generation research

created 2 years ago

updated 23 hours ago

Starred by

Lianmin Zheng(Author of SGLang) and

Ying Sheng(Author of SGLang).

Mooncake by kvcache-ai

1.1%

Research paper on a disaggregated architecture for LLM serving

created 1 year ago

updated 11 hours ago

Starred by

Lianmin Zheng(Author of SGLang),

Simon Willison(Author of Django), and

1 more.

xgrammar by mlc-ai

2.0%

Library for efficient structured generation

created 1 year ago

updated 3 days ago

Starred by

Andrej Karpathy(Founder of Eureka Labs; Formerly at Tesla, OpenAI; Author of CS 231n),

Jiayi Pan(Author of SWE-Gym; AI Researcher at UC Berkeley), and

4 more.

Liger-Kernel by linkedin

0.6%

Triton kernels for efficient LLM training

created 1 year ago

updated 1 day ago

Starred by

Ying Sheng(Author of SGLang) and

Zhuohan Li(Author of vLLM).

Nanoflow by efeslab

0.9%

856

LLM serving framework for high throughput

created 11 months ago

updated 3 weeks ago

Starred by

Aravind Srinivas(Cofounder of Perplexity),

Jiayi Pan(Author of SWE-Gym; AI Researcher at UC Berkeley), and

9 more.

xla by pytorch

0.2%

PyTorch on XLA devices

created 6 years ago

updated 19 hours ago

Starred by

Lianmin Zheng(Author of SGLang),

Soumith Chintala(Author of PyTorch), and

4 more.

ao by pytorch

0.5%

PyTorch library for quantization and sparsity in training/inference

created 1 year ago

updated 17 hours ago

TensorRT-Model-Optimizer by NVIDIA

1.5%

Library for optimizing deep learning models for GPU inference

created 1 year ago

updated 2 weeks ago

Starred by

Zhuohan Li(Author of vLLM).

llm-compressor by vllm-project

2.1%

Transformers-compatible library for LLM compression, optimized for vLLM deployment

created 1 year ago

updated 19 hours ago

Starred by

Jeff Hammerbacher(Cofounder of Cloudera) and

Luca Antiga(CTO of Lightning AI).

intel-extension-for-pytorch by intel

0.4%

PyTorch extension for performance boost on Intel platforms

created 5 years ago

updated 5 days ago

Starred by

Andrej Karpathy(Founder of Eureka Labs; Formerly at Tesla, OpenAI; Author of CS 231n),

Georgios Konstantopoulos(CTO, General Partner at Paradigm), and

6 more.

ThunderKittens by HazyResearch

0.6%

CUDA kernel framework for fast deep learning primitives

created 1 year ago

updated 3 days ago

Starred by

George Hotz(Author of tinygrad; Founder of the tiny corp, comma.ai),

Zhuohan Li(Author of vLLM), and

1 more.

mirage by mirage-project

1.3%

Tool for fast GPU kernel generation via superoptimization

created 1 year ago

updated 1 day ago

Starred by

Jeff Hammerbacher(Cofounder of Cloudera),

Philipp Schmid(DevRel at Google DeepMind), and

2 more.

AutoAWQ by casper-hansen

0.4%

AutoAWQ is a tool for 4-bit quantized LLM inference

created 1 year ago

updated 2 months ago

Starred by

George Hotz(Author of tinygrad; Founder of the tiny corp, comma.ai),

Fabian Hedin(Cofounder of Lovable), and

11 more.

grok-1 by xai-org

0.1%

50k

JAX example code for loading and running Grok-1 open-weights model

created 1 year ago

updated 11 months ago

Starred by

Aravind Srinivas(Cofounder of Perplexity),

Zhuohan Li(Author of vLLM), and

8 more.

lm-evaluation-harness by EleutherAI

0.7%

10k

Framework for few-shot language model evaluation

created 5 years ago

updated 17 hours ago

LLMSys-PaperList by AmberLJC

1.4%

Curated list of LLM systems papers

created 2 years ago

updated 1 day ago

Starred by

Tobi Lutke(Cofounder of Shopify),

Jeff Hammerbacher(Cofounder of Cloudera), and

6 more.

aici by microsoft

0.1%

AICI constrains LLM output using (Wasm) programs

created 1 year ago

updated 6 months ago

Starred by

Tobi Lutke(Cofounder of Shopify),

Omar Sanseviero(DevRel at Google DeepMind), and

12 more.

mlc-llm by mlc-ai

0.2%

21k

Universal LLM deployment engine with ML compilation

created 2 years ago

updated 5 days ago

Starred by

Jonathan Ragan-Kelley(Professor at MIT).

mscclpp by microsoft

0.8%

392

GPU-driven communication stack for scalable AI applications

created 2 years ago

updated 12 hours ago

Starred by

Elie Bursztein(Cybersecurity Lead at Google DeepMind),

Philipp Schmid(DevRel at Google DeepMind), and

17 more.

sglang by sgl-project

1.1%

16k

Fast serving framework for LLMs and vision language models

created 1 year ago

updated 10 hours ago

Starred by

Jeff Hammerbacher(Cofounder of Cloudera),

Lianmin Zheng(Author of SGLang), and

3 more.

flashinfer by flashinfer-ai

1.4%

Kernel library for LLM serving

created 2 years ago

updated 12 hours ago

Starred by

Jeff Hammerbacher(Cofounder of Cloudera),

Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems), and

2 more.

punica by punica-ai

LoRA serving system (research paper) for multi-tenant LLM inference

created 1 year ago

updated 1 year ago

Starred by

Jeff Hammerbacher(Cofounder of Cloudera),

Travis Fischer(Founder of Agentic), and

1 more.

LLMCompiler by SqueezeAILab

0.2%

LLM compiler for parallel function calling

created 1 year ago

updated 1 year ago

Starred by

Andrej Karpathy(Founder of Eureka Labs; Formerly at Tesla, OpenAI; Author of CS 231n),

Lianmin Zheng(Author of SGLang), and

12 more.

gpt-fast by pytorch-labs

0.1%

PyTorch text generation for efficient transformer inference

created 1 year ago

updated 3 months ago

Starred by

Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems),

Omar Sanseviero(DevRel at Google DeepMind), and

4 more.

TensorRT-LLM by NVIDIA

0.6%

11k

LLM inference optimization SDK for NVIDIA GPUs

created 1 year ago

updated 14 hours ago

Starred by

Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems),

Thomas Wolf(Cofounder of Hugging Face), and

10 more.

WizardLM by nlpxucan

0.1%

LLMs built using Evol-Instruct for complex instruction following

created 2 years ago

updated 1 month ago

Starred by

Tobi Lutke(Cofounder of Shopify),

Clément Renault(Cofounder of Meilisearch), and

21 more.

outlines by dottxt-ai

0.3%

12k

SDK for structured LLM text generation

created 2 years ago

updated 11 hours ago

Starred by

Andreas Jansson(Cofounder of Replicate),

Omar Sanseviero(DevRel at Google DeepMind), and

2 more.

Awesome-LLM by Hannibal046

0.5%

24k

Curated list of Large Language Model resources

created 2 years ago

updated 2 days ago

Starred by

Lewis Tunstall(Researcher at Hugging Face),

Georgios Konstantopoulos(CTO, General Partner at Paradigm), and

7 more.

gorilla by ShishirPatil

0.1%

12k

LLM tool-use framework for API invocation and function calling

created 2 years ago

updated 5 days ago

Starred by

Travis Fischer(Founder of Agentic),

Ying Sheng(Author of SGLang), and

1 more.

LLMSurvey by RUCAIBox

0.2%

12k

Survey paper for large language models

created 2 years ago

updated 4 months ago

Starred by

Lianmin Zheng(Author of SGLang),

Simon Willison(Author of Django), and

4 more.

CTranslate2 by OpenNMT

0.3%

Fast inference engine for Transformer models

created 5 years ago

updated 3 months ago

SqueezeLLM by SqueezeAILab

0.1%

698

Quantization framework for efficient LLM serving (ICML 2024 paper)

created 2 years ago

updated 11 months ago

Starred by

Andrej Karpathy(Founder of Eureka Labs; Formerly at Tesla, OpenAI; Author of CS 231n),

Tobi Lutke(Cofounder of Shopify), and

26 more.

vllm by vllm-project

1.0%

54k

LLM serving engine for high-throughput, memory-efficient inference

created 2 years ago

updated 10 hours ago

Starred by

Jeff Hammerbacher(Cofounder of Cloudera).

Awesome-LLMOps by tensorchord

0.5%

Curated list of LLMOps tools for developers

created 3 years ago

updated 2 days ago

Starred by

Anastasios Angelopoulos(Cofounder of LMArena),

Omar Sanseviero(DevRel at Google DeepMind), and

20 more.

FastChat by lm-sys

0.1%

39k

Open platform for training, serving, and evaluating LLM-based chatbots

created 2 years ago

updated 2 months ago

Starred by

Zhiqiang Xie(Author of SGLang),

Fabian Hedin(Cofounder of Lovable), and

21 more.

llama by meta-llama

0.1%

59k

Inference code for Llama 2 models (deprecated)

created 2 years ago

updated 6 months ago

Starred by

Wei-Lin Chiang(Cofounder of LMArena),

Ying Sheng(Author of SGLang), and

8 more.

Megatron-LM by NVIDIA

0.4%

13k

Framework for training transformer models at scale

created 6 years ago

updated 22 hours ago

Starred by

Andrej Karpathy(Founder of Eureka Labs; Formerly at Tesla, OpenAI; Author of CS 231n),

Jiayi Pan(Author of SWE-Gym; AI Researcher at UC Berkeley), and

15 more.

flash-attention by Dao-AILab

0.7%

19k

Fast, memory-efficient attention implementation

created 3 years ago

updated 14 hours ago

Starred by

Jeff Hammerbacher(Cofounder of Cloudera),

Zhuohan Li(Author of vLLM), and

2 more.

TransformerEngine by NVIDIA

0.8%

Library for Transformer model acceleration on NVIDIA GPUs

created 2 years ago

updated 20 hours ago

Starred by

Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems),

Jeff Hammerbacher(Cofounder of Cloudera), and

5 more.

x-transformers by lucidrains

0.2%

Transformer library with extensive experimental features

created 4 years ago

updated 3 days ago

Starred by

Zhuohan Li(Author of vLLM) and

Lianmin Zheng(Author of SGLang).

compiler-and-arch by KnowingNothing

0.2%

474

Compiler/architecture resources for emerging domains

created 2 years ago

updated 6 months ago

Starred by

Andrej Karpathy(Founder of Eureka Labs; Formerly at Tesla, OpenAI; Author of CS 231n),

Tobi Lutke(Cofounder of Shopify), and

14 more.

skypilot by skypilot-org

0.4%

Framework for cloud AI/batch jobs, unifying execution across diverse infrastructure

created 4 years ago

updated 14 hours ago

Starred by

Nat Friedman(Former CEO of GitHub),

Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems), and

5 more.

FasterTransformer by NVIDIA

0.2%

Optimized transformer library for inference

created 4 years ago

updated 1 year ago

Starred by

Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems),

Jiayi Pan(Author of SWE-Gym; AI Researcher at UC Berkeley), and

10 more.

alpa by alpa-projects

0.1%

Auto-parallelization framework for large-scale neural network training and serving

created 4 years ago

updated 1 year ago

Starred by

Lilian Weng(Cofounder of Thinking Machines Lab),

Andrej Karpathy(Founder of Eureka Labs; Formerly at Tesla, OpenAI; Author of CS 231n), and

41 more.

transformers by huggingface

0.2%

148k

ML library for pretrained model inference and training

created 6 years ago

updated 10 hours ago

Starred by

Hiroshi Shibata(Core Contributor to Ruby),

Jiayi Pan(Author of SWE-Gym; AI Researcher at UC Berkeley), and

26 more.

ray by ray-project

0.3%

38k

AI compute engine for scaling Python and AI applications

created 8 years ago

updated 12 hours ago

Starred by

Aravind Srinivas(Cofounder of Perplexity),

Travis Fischer(Founder of Agentic), and

8 more.

tvm by apache

0.1%

12k

Compiler stack for deep learning systems

created 8 years ago

updated 14 hours ago

Starred by

Ben Firshman(Cofounder of Replicate),

Zhuohan Li(Author of vLLM), and

1 more.

DeepLearningExamples by NVIDIA

0.1%

14k

Deep learning examples for training and deployment

created 7 years ago

updated 11 months ago

Feedback? Help us improve.