Ying Sheng

Starred by

Lianmin Zheng(Author of SGLang),

Nathan Lambert(AI Researcher at AI2), and

2 more.

OpenRLHF by OpenRLHF

RLHF framework for scalable training of large language models

Starred by

Lewis Tunstall(Researcher at Hugging Face),

Robert Nishihara(Cofounder of Anyscale; Author of Ray), and

3 more.

verl by volcengine

2.4%

RL training library for LLMs

created 9 months ago

updated 10 hours ago

how-to-optim-algorithm-in-cuda by BBuf

CUDA optimization guide for common algorithms

created 7 years ago

Learning materials for SGLang, an efficient LLM serving engine

Starred by

Lianmin Zheng(Author of SGLang).

sgl-learning-materials by sgl-project

2.1%

514

created 10 months ago

updated 2 weeks ago

glake by antgroup

GPU optimization library for memory management and IO

471

Library for efficient structured generation

Starred by

Lianmin Zheng(Author of SGLang),

Simon Willison(Author of Django), and

1 more.

xgrammar by mlc-ai

2.0%

Starred by

Tim J. Baek(Founder of Open WebUI),

Philipp Schmid(DevRel at Google DeepMind), and

8 more.

NeMo by NVIDIA

Scalable generative AI framework for LLMs, multimodal, and speech AI research

15k

created 6 years ago

updated 14 hours ago

Starred by

Woosuk Kwon(Author of vLLM) and

Zhuohan Li(Author of vLLM).

Nanoflow by efeslab

LLM serving framework for high throughput

856

created 11 months ago

Benchmark for evaluating language models on super-long contexts (100k+ tokens)

Starred by

Omar Sanseviero(DevRel at Google DeepMind).

InfiniteBench by OpenBMB

343

updated 10 months ago

Starred by

Patrick von Platen(Core Contributor to Hugging Face Transformers and Diffusers),

Simon Willison(Author of Django), and

8 more.

simple-evals by openai

Lightweight library for evaluating language models

LLM inference system for production environments

ScaleLLM by vectorch-ai

0.7%

460

CLI tool for merging pretrained language models, combining strengths without retraining

updated 1 week ago

Starred by

Travis Fischer(Founder of Agentic),

Thomas Wolf(Cofounder of Hugging Face), and

10 more.

mergekit by arcee-ai

0.5%

Framework for LLM routing and cost reduction (research paper)

Starred by

Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems),

Elie Bursztein(Cybersecurity Lead at Google DeepMind), and

5 more.

RouteLLM by lm-sys

0.5%

LLM compression toolkit for accelerated CPU/GPU inference

updated 11 months ago

GPTQModel by ModelCloud

1.4%

705

updated 2 days ago

Starred by

Lianmin Zheng(Author of SGLang) and

Woosuk Kwon(Author of vLLM).

Mooncake by kvcache-ai

Research paper on a disaggregated architecture for LLM serving

Benchmark for evaluating LLMs on real-world GitHub issues

updated 11 hours ago

Starred by

Travis Fischer(Founder of Agentic),

Jiayi Pan(Author of SWE-Gym; AI Researcher at UC Berkeley), and

4 more.

SWE-bench by SWE-bench

0.8%

Starred by

Simon Willison(Author of Django),

Eugene Yan(AI Scientist at AWS), and

3 more.

inspect_ai by UKGovernmentBEIS

Framework for large language model evaluations

Inference framework for efficient long-context LLM inference

updated 19 hours ago

Quest by mit-han-lab

1.3%

308

Starred by

Tobi Lutke(Cofounder of Shopify),

Matei Zaharia(Cofounder of Databricks), and

27 more.

dspy by stanfordnlp

Framework for programming language models, not prompting

27k

PyTorch code for weight-decomposed low-rank adaptation (DoRA)

Starred by

Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems).

DoRA by NVlabs

818

updated 10 months ago

Starred by

Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems) and

Didier Lopes(Founder of OpenBB).

RULER by NVIDIA

Evaluation suite for long-context language models research paper

Tool for fast GPU kernel generation via superoptimization

updated 1 week ago

Starred by

George Hotz(Author of tinygrad; Founder of the tiny corp, comma.ai),

Zhuohan Li(Author of vLLM), and

1 more.

mirage by mirage-project

1.3%

Starred by

Jerry Liu(Cofounder of LlamaIndex) and

Jeff Hammerbacher(Cofounder of Cloudera).

beir by beir-cellar

IR benchmark for evaluating NLP retrieval models

Starred by

Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems),

Teknium(Cofounder of Nous Research), and

2 more.

storm by stanford-oval

LLM system for automated knowledge curation and article generation

27k

Starred by

Peter Norvig(Author of Artificial Intelligence: A Modern Approach; Research Director at Google),

Didier Lopes(Founder of OpenBB), and

14 more.

llm.c by karpathy

LLM training in pure C/CUDA, no PyTorch needed

27k

CUDA learning notes for beginners using PyTorch

LeetCUDA by xlite-dev

2.1%

Starred by

Tobi Lutke(Cofounder of Shopify),

Jason Miller(Author of Preact), and

11 more.

candle by huggingface

Minimalist ML framework for Rust, emphasizing performance and ease of use

18k

Starred by

Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems) and

Omar Sanseviero(DevRel at Google DeepMind).

Open-Sora-Plan by PKU-YuanGroup

Open-source project aiming to reproduce Sora-like T2V model

updated 2 weeks ago

MiniGPT4-video by Vision-CAIR

Video-language model for short and long video understanding

627

updated 7 months ago

Starred by

Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems),

Omar Sanseviero(DevRel at Google DeepMind), and

2 more.

adapters by adapter-hub

Unified library for parameter-efficient transfer learning in NLP

Starred by

Michael Han(Cofounder of Unsloth),

Lianmin Zheng(Author of SGLang), and

1 more.

lmdeploy by InternLM

Toolkit for LLM compression, deployment, and serving

Starred by

Tobi Lutke(Cofounder of Shopify),

Stas Bekman(Author of Machine Learning Engineering Open Book; Research Engineer at Snowflake), and

20 more.

guidance by guidance-ai

Guidance is a programming paradigm for steering LLMs

21k

updated 22 hours ago

Starred by

Tim J. Baek(Founder of Open WebUI),

Michael Han(Cofounder of Unsloth), and

14 more.

trl by huggingface

Library for transformer RL

15k

updated 18 hours ago

Starred by

Elie Bursztein(Cybersecurity Lead at Google DeepMind),

Philipp Schmid(DevRel at Google DeepMind), and

17 more.

sglang by sgl-project

Fast serving framework for LLMs and vision language models

16k

updated 10 hours ago

Starred by

Jerry Liu(Cofounder of LlamaIndex).

rags by run-llama

Streamlit app for building RAG pipelines via natural language

Starred by

Lewis Tunstall(Researcher at Hugging Face),

Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems), and

1 more.

llm-reasoners by maitrix-org

Library for advanced LLM reasoning with search algorithms

Starred by

Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems),

Andreas Jansson(Cofounder of Replicate), and

1 more.

ToolBench by OpenBMB

Open platform for LLM tool learning (ICLR'24 spotlight)

Starred by

Wes McKinney(Author of Pandas),

Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems), and

8 more.

autogen by microsoft

Agentic framework for multi-agent AI applications

48k

updated 19 hours ago

Starred by

Jiayi Pan(Author of SWE-Gym; AI Researcher at UC Berkeley) and

Travis Fischer(Founder of Agentic).

webarena by web-arena-x

Web environment for autonomous agent development

updated 5 months ago

WebGLM by THUDM

Web-enhanced question answering system using a 10B GLM

Starred by

Lianmin Zheng(Author of SGLang),

Aman Sanger(Cofounder of Cursor), and

3 more.

DeepSpeed-MII by deepspeedai

Python library for high-throughput, low-latency, and cost-effective model inference

Starred by

Matei Zaharia(Cofounder of Databricks),

Alexander Wettig(Author of SWE-bench, SWE-agent), and

6 more.

megablocks by databricks

Lightweight library for mixture-of-experts (MoE) training

Speculative decoding research paper for faster LLM inference

Starred by

Philipp Schmid(DevRel at Google DeepMind),

Omar Sanseviero(DevRel at Google DeepMind), and

1 more.

EAGLE by SafeAILab

1.2%

updated 4 days ago

Starred by

Andrej Karpathy(Founder of Eureka Labs; Formerly at Tesla, OpenAI; Author of CS 231n),

Lianmin Zheng(Author of SGLang), and

12 more.

gpt-fast by pytorch-labs

PyTorch text generation for efficient transformer inference

updated 3 months ago

VLM_survey by jingyi0000

VLM survey paper with links to models/methods for vision tasks

Starred by

Tobi Lutke(Cofounder of Shopify),

Didier Lopes(Founder of OpenBB), and

5 more.

dify by langgenius

Open-source LLM app development platform

109k

updated 9 hours ago

Starred by

Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems) and

Zhuohan Li(Author of vLLM).

LookaheadDecoding by hao-ai-lab

Parallel decoding algorithm for faster LLM inference

Starred by

Travis Fischer(Founder of Agentic).

ARES by stanford-futuredata

0.8%

634

RAG evaluation framework

Starred by

Stas Bekman(Author of Machine Learning Engineering Open Book; Research Engineer at Snowflake).

llm-analysis by cli99

CLI tool for LLM latency/memory analysis during training/inference

441

updated 3 months ago

Awesome-LLM-Inference by xlite-dev

Curated list of LLM/VLM inference research papers with code

ArXiv scanner using GPT-4 for personalized paper recommendations

gpt_paper_assistant by tatsu-lab

0.8%

530

Codebase for merging language models via parameter averaging

MergeLM by yule-BUAA

842

Starred by

Jerry Liu(Cofounder of LlamaIndex) and

Lianmin Zheng(Author of SGLang).

ChatRTX by NVIDIA

Demo app for local RAG chatbot on Windows

Starred by

Carol Willing(Core Contributor to CPython, Jupyter),

Travis Fischer(Founder of Agentic), and

1 more.

DeepSeek-Coder by deepseek-ai

Code LLM for code completion and generation

22k

LLM contamination detector for quantifying rephrased samples

Starred by

Lianmin Zheng(Author of SGLang) and

Wei-Lin Chiang(Cofounder of LMArena).

llm-decontaminator by lm-sys

306

Starred by

Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems),

Omar Sanseviero(DevRel at Google DeepMind), and

1 more.

S-LoRA by S-LoRA

System for scalable LoRA adapter serving

Kernel library for LLM serving

Starred by

Jeff Hammerbacher(Cofounder of Cloudera),

Lianmin Zheng(Author of SGLang), and

3 more.

flashinfer by flashinfer-ai

1.4%

updated 12 hours ago

Starred by

Andrej Karpathy(Founder of Eureka Labs; Formerly at Tesla, OpenAI; Author of CS 231n),

Tobi Lutke(Cofounder of Shopify), and

14 more.

skypilot by skypilot-org

Framework for cloud AI/batch jobs, unifying execution across diverse infrastructure

updated 14 hours ago

gpu_poor by RahulSChand

CLI tool for LLM memory and throughput estimation

updated 8 months ago

Starred by

Bojan Tunguz(AI Scientist; Formerly at NVIDIA),

Mckay Wrigley(Founder of Takeoff AI), and

7 more.

ggml by ggml-org

Tensor library for machine learning

13k

tensorrtllm_backend by triton-inference-server

Triton backend for serving TensorRT-LLM models

871

updated 16 hours ago

Starred by

Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems),

Omar Sanseviero(DevRel at Google DeepMind), and

4 more.

TensorRT-LLM by NVIDIA

LLM inference optimization SDK for NVIDIA GPUs

11k

updated 14 hours ago

Starred by

Jeff Hammerbacher(Cofounder of Cloudera) and

Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems).

leptonai by leptonai

Python framework for simplifying AI service building

LoRA serving system (research paper) for multi-tenant LLM inference

updated 19 hours ago

Starred by

Jeff Hammerbacher(Cofounder of Cloudera),

Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems), and

2 more.

punica by punica-ai

Starred by

Tobi Lutke(Cofounder of Shopify),

Evan Conrad(Cofounder of SF Compute), and

7 more.

modular by modular

AI toolchain unifying fragmented AI deployment workflows

25k

LLM developer's reference for key numbers

Starred by

David Cournapeau(Author of scikit-learn),

Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems), and

5 more.

llm-numbers by ray-project

Starred by

Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems),

Travis Fischer(Founder of Agentic), and

1 more.

LLaMA2-Accessory by Alpha-VLLM

Open-source toolkit for LLM development, pretraining, finetuning, and deployment

Position embeddings research paper

Starred by

Jeremy Howard(Cofounder of fast.ai) and

Lianmin Zheng(Author of SGLang).

rerope by bojone

375

Python framework for LLM inference and serving

Starred by

Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems),

Philipp Schmid(DevRel at Google DeepMind), and

1 more.

LightLLM by ModelTC

0.7%

updated 10 hours ago

Starred by

Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems),

Jeff Hammerbacher(Cofounder of Cloudera), and

3 more.

GPTCache by zilliztech

Semantic cache for LLM queries, integrated with LangChain and LlamaIndex

Starred by

Zhiqiang Xie(Author of SGLang),

Fabian Hedin(Cofounder of Lovable), and

21 more.

llama by meta-llama

Inference code for Llama 2 models (deprecated)

59k

Hands-on puzzles for large language model training

Starred by

Jiayi Pan(Author of SWE-Gym; AI Researcher at UC Berkeley),

Patrick von Platen(Core Contributor to Hugging Face Transformers and Diffusers), and

6 more.

LLM-Training-Puzzles by srush

1.0%

Jax implementation of RingAttention for large context models (research paper)

ringattention by haoliuhl

719

Starred by

Omar Sanseviero(DevRel at Google DeepMind),

Jeremy Howard(Cofounder of fast.ai), and

2 more.

metal-flash-attention by philipturner

Metal port of FlashAttention for Apple silicon

511

updated 10 months ago

Starred by

Tobi Lutke(Cofounder of Shopify),

Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems), and

8 more.

companion-app by a16z-infra

AI companion stack for personalized chatbots

Starred by

Tim J. Baek(Founder of Open WebUI),

Daniel Han(Cofounder of Unsloth), and

14 more.

bitsandbytes by bitsandbytes-foundation

PyTorch library for k-bit quantization, enabling accessible LLMs

Starred by

Lianmin Zheng(Author of SGLang) and

Simon Willison(Author of Django).

long_llama by CStanKonrad

LLM for long context handling, fine-tuned with Focused Transformer

Starred by

Zhiqiang Xie(Author of SGLang),

Zhuohan Li(Author of vLLM), and

8 more.

scalene by plasma-umass

Python profiler with AI-powered optimization proposals

13k

Starred by

Travis Fischer(Founder of Agentic),

Zhuohan Li(Author of vLLM), and

1 more.

LLMSurvey by RUCAIBox

Survey paper for large language models

Awesome-LLM-Compression by HuangOwen

LLM compression papers and tools for efficient training/inference

Starred by

Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems).

fastllm by ztxz16

High-performance C++ LLM inference library

KV cache eviction research paper for efficient LLM inference

updated 2 weeks ago

H2O by FMInference

462

Long-context LLM chatbot training and evaluation framework

Starred by

Lianmin Zheng(Author of SGLang) and

Zhuohan Li(Author of vLLM).

LongChat by DachengLi1

524

LLM research release with 8k sequence length

Starred by

Jared Palmer(Ex-VP of AI at Vercel; Founder of Turborepo; Author of Formik, TSDX).

xgen by salesforce

720

Starred by

Tobi Lutke(Cofounder of Shopify),

Georgios Konstantopoulos(CTO, General Partner at Paradigm), and

13 more.

peft by huggingface

Parameter-efficient fine-tuning (PEFT) library

19k

Starred by

Lilian Weng(Cofounder of Thinking Machines Lab),

Julien Chaumond(Cofounder of Hugging Face), and

25 more.

AutoGPT by Significant-Gravitas

AI agent platform for building, deploying, and running autonomous workflows

177k

updated 12 hours ago

Starred by

Lianmin Zheng(Author of SGLang),

Simon Willison(Author of Django), and

4 more.

CTranslate2 by OpenNMT

Fast inference engine for Transformer models

updated 3 months ago

Starred by

Tobi Lutke(Cofounder of Shopify),

Tim J. Baek(Founder of Open WebUI), and

19 more.

text-generation-inference by huggingface

Rust/Python/gRPC server for fast LLM text generation

10k

updated 1 week ago

Starred by

Tobi Lutke(Cofounder of Shopify),

Omar Sanseviero(DevRel at Google DeepMind), and

12 more.

mlc-llm by mlc-ai

Universal LLM deployment engine with ML compilation

21k

updated 5 days ago

awesome-mixture-of-experts by XueFuzhao

Curated list of resources for mixture-of-experts (MoE) research

Multilingual LLM research paper with 104B parameters

updated 7 months ago

InternLM-techreport by InternLM

905

updated 2 years ago

Starred by

Wei-Lin Chiang(Cofounder of LMArena).

awesome-chatgpt-dataset by voidful

Dataset repo for LLM training

731

Starred by

George Hotz(Author of tinygrad; Founder of the tiny corp, comma.ai),

Daniel Gross(Cofounder of Safe Superintelligence), and

12 more.

RWKV-LM by BlinkDL

RNN for LLM, transformer-level performance, parallelizable training

14k

updated 6 days ago

Starred by

Tobi Lutke(Cofounder of Shopify),

Jeff Huber(Cofounder of Chroma), and

14 more.

qdrant by qdrant

Vector database for similarity search in AI applications

25k

updated 20 hours ago

Starred by

Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems),

Travis Fischer(Founder of Agentic), and

8 more.

alpaca-lora by tloen

LoRA fine-tuning for LLaMA

19k

Starred by

Nat Friedman(Former CEO of GitHub),

Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems), and

8 more.

trlx by CarperAI

Distributed RLHF for LLMs

Starred by

Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems),

Omar Sanseviero(DevRel at Google DeepMind), and

1 more.

MOSS by OpenMOSS

Open-source tool-augmented conversational language model

CodeGeeX by zai-org

Code generation model for multilingual programming

Chat model trained via LoRA, using ChatGPT-generated dialogs

updated 11 months ago

Starred by

Bojan Tunguz(AI Scientist; Formerly at NVIDIA),

Teknium(Cofounder of Nous Research), and

1 more.

baize-chatbot by project-baize

CLI tool for debugging with natural language via LLM

Starred by

Zhiqiang Xie(Author of SGLang).

ChatGDB by pgosar

922

updated 7 months ago

Starred by

Anastasios Angelopoulos(Cofounder of LMArena),

Omar Sanseviero(DevRel at Google DeepMind), and

20 more.

FastChat by lm-sys

Open platform for training, serving, and evaluating LLM-based chatbots

39k

Triton backend for optimized transformer inference

fastertransformer_backend by triton-inference-server

411

Starred by

Tobi Lutke(Cofounder of Shopify),

Elie Bursztein(Cybersecurity Lead at Google DeepMind), and

5 more.

LMFlow by OptimalScale

Toolkit for finetuning and inference of large foundation models

Starred by

Andrej Karpathy(Founder of Eureka Labs; Formerly at Tesla, OpenAI; Author of CS 231n),

Carol Willing(Core Contributor to CPython, Jupyter), and

37 more.

langchain by langchain-ai

Framework for building LLM-powered applications

113k

updated 21 hours ago

Starred by

Andrej Karpathy(Founder of Eureka Labs; Formerly at Tesla, OpenAI; Author of CS 231n),

Anton Troynikov(Cofounder of Chroma), and

19 more.

llama_index by run-llama

Data framework for building LLM-powered agents

43k

updated 15 hours ago

Starred by

Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems),

Jonathan Ragan-Kelley(Professor at MIT), and

13 more.

FlexLLMGen by FMInference

High-throughput generation engine for LLMs with limited GPU memory

updated 9 months ago

Starred by

Yang Song(Professor at Caltech; Research Scientist at OpenAI),

Jeremy Howard(Cofounder of fast.ai), and

3 more.

PiPPy by pytorch

PyTorch tool for pipeline parallelism

775

updated 11 months ago

best_AI_papers_2022 by louisfb01

AI paper list (2022) with video explanations and code

Starred by

Lianmin Zheng(Author of SGLang) and

Patrick von Platen(Core Contributor to Hugging Face Transformers and Diffusers).

dpm-solver by LuChengTHU

Fast ODE solver for diffusion probabilistic model sampling

Starred by

Nat Friedman(Former CEO of GitHub),

Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems), and

5 more.

FasterTransformer by NVIDIA

Optimized transformer library for inference

Starred by

Nat Friedman(Former CEO of GitHub),

Jeff Hammerbacher(Cofounder of Cloudera), and

4 more.

CodeGen by salesforce

Open-source model family for program synthesis

Starred by

Mckay Wrigley(Founder of Takeoff AI),

Simon Willison(Author of Django), and

2 more.

GLM-130B by zai-org

Bilingual model for research and evaluation

updated 2 years ago

Starred by

Wei-Lin Chiang(Cofounder of LMArena),

Woosuk Kwon(Author of vLLM), and

8 more.

Megatron-LM by NVIDIA

Framework for training transformer models at scale

13k

created 6 years ago

updated 22 hours ago

Starred by

Jeff Hammerbacher(Cofounder of Cloudera).

llm-seminar by craffel

311

Course reading list for large language models

updated 2 years ago

Starred by

Jiayi Pan(Author of SWE-Gym; AI Researcher at UC Berkeley) and

Lianmin Zheng(Author of SGLang).

paper-reading by mli

Deep learning paper readings

31k

Starred by

Jeff Hammerbacher(Cofounder of Cloudera),

Eugene Yan(AI Scientist at AWS), and

2 more.

CodeT5 by salesforce

Code LLMs for code understanding and generation research

Dive-into-DL-PyTorch by ShusenTang

PyTorch rewrite of "Dive into Deep Learning" book

19k

created 6 years ago

updated 3 years ago

Starred by

Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems),

Jiayi Pan(Author of SWE-Gym; AI Researcher at UC Berkeley), and

10 more.

alpa by alpa-projects

Auto-parallelization framework for large-scale neural network training and serving