Cody Yu

Coauthor of vLLM; MTS at OpenAI

Starred Projects (26)

EPLB by deepseek-ai

Load balancer for expert-parallel MoE models

parano:

zhyncs:

Created 8 months ago

Updated 7 months ago

open-infra-index by deepseek-ai

AI infrastructure tools for efficient AGI development

lattner:

vincentweisser:

hammer:

pankajroark:

Created 8 months ago

Updated 5 months ago

kvpress by NVIDIA

LLM KV cache compression made easy

luiscape:

Created 1 year ago

Updated 1 week ago

xgrammar by mlc-ai

Library for efficient structured generation

merrymercy:

ogabrielluiz:

youkaichao:

simonw:

Created 1 year ago

Updated 10 hours ago

llm-compressor by vllm-project

Transformers-compatible library for LLM compression, optimized for vLLM deployment

patrickvonplaten:

peakji:

zhuohan123:

WoosukKwon:

Created 1 year ago

Updated 18 hours ago

LMCache by LMCache

LLM serving engine extension for reduced TTFT and increased throughput

taranjeet:

chiphuyen:

omarsar:

luiscape:

Created 1 year ago

Updated 15 hours ago

xDiT by xdit-project

Inference engine for parallel Diffusion Transformer (DiT) deployment

sxyu:

Created 1 year ago

Updated 5 days ago

Consistency_LLM by hao-ai-lab

Parallel decoder for efficient LLM inference

chiphuyen:

zhuohan123:

merrymercy:

Created 1 year ago

Updated 11 months ago

attorch by BobMcDear

PyTorch nn module subset, implemented in Python using Triton

karpathy:

Edward-Sun:

sualehasif:

srush:

Created 2 years ago

Updated 2 months ago

grok-1 by xai-org

JAX example code for loading and running Grok-1 open-weights model

geohot:

yiranwu0:

omarsar:

handotdev:

Created 1 year ago

Updated 1 year ago

sglang by sgl-project

Fast serving framework for LLMs and vision language models

samlambert:

ebursztein:

philschmid:

pankajroark:

Created 1 year ago

Updated 9 hours ago

Prompt-Engineering-Guide by dair-ai

Prompt engineering resource for language model (LLM) applications

lilianweng:

tunguz:

rodrigosnader:

fabhed:

Created 2 years ago

Updated 3 days ago

llmperf by ray-project

LLM validation/benchmark library for LLM APIs

ishaan-jaff:

robertnishihara:

omarsar:

pirroh:

Created 2 years ago

Updated 11 months ago

DeepSpeed-MII by deepspeedai

Python library for high-throughput, low-latency, and cost-effective model inference

hiyouga:

Ying1123:

merrymercy:

Sanger2000:

Created 3 years ago

Updated 4 months ago

TensorRT-LLM by NVIDIA

LLM inference optimization SDK for NVIDIA GPUs

hammer:

zhyncs:

shizhediao:

luiscape:

Created 2 years ago

Updated 9 hours ago

Awesome-LLM by Hannibal046

Curated list of Large Language Model resources

rodrigosnader:

shizhediao:

zhiyuan8:

vincentweisser:

Created 2 years ago

Updated 3 months ago

generative_agents by joonspk-research

Research paper code for interactive human behavior simulation using generative agents

tobi:

sxyu:

chiphuyen:

xiezhq-hermann:

Created 2 years ago

Updated 1 year ago

LightLLM by ModelTC

Python framework for LLM inference and serving

zhyncs:

chiphuyen:

pgarbacki:

JustinLin610:

Created 2 years ago

Updated 10 hours ago

text-generation-inference by huggingface

Rust/Python/gRPC server for fast LLM text generation

tobi:

clmnt:

tjbck:

chiphuyen:

Created 3 years ago

Updated 1 month ago

GPTCache by zilliztech

Semantic cache for LLM queries, integrated with LangChain and LlamaIndex

chiphuyen:

hammer:

Ying1123:

ogabrielluiz:

Created 2 years ago

Updated 3 months ago

StableLM by Stability-AI

Language models by Stability AI

geohot:

vincentweisser:

casper-hansen:

calvinfo:

Created 2 years ago

Updated 1 year ago

xla by pytorch

PyTorch on XLA devices

aravindsrinivas:

WoosukKwon:

Jiayi-Pan:

ankane:

Created 7 years ago

Updated 1 day ago

alpa by alpa-projects

Auto-parallelization framework for large-scale neural network training and serving

chiphuyen:

Jiayi-Pan:

transitive-bullshit:

soldni:

Created 4 years ago

Updated 1 year ago

antares by microsoft

Compiler solution for PyTorch operator optimization on diverse accelerators

parasj:

merrymercy:

xiezhq-hermann:

Created 5 years ago

Updated 6 months ago

awesome-tensor-compilers by merrymercy

Curated list of tensor compiler projects and papers

chiphuyen:

infwinston:

Ying1123:

luiscape:

Created 5 years ago

Updated 1 year ago

tvm by apache

Compiler stack for deep learning systems

aravindsrinivas:

transitive-bullshit:

guberti:

wesm:

Created 9 years ago

Updated 12 hours ago

Feedback? Help us improve.