Georgi Gerganov

Local LLM serving with hardware acceleration

Starred by

Travis Fischer(Founder of Agentic) and

Tim J. Baek(Founder of Open WebUI).

lemonade by lemonade-sdk

19.0%

332

created 2 months ago

updated 22 hours ago

PlayDiffusion by playht

Diffusion model for speech editing

510

created 2 months ago

Android app for local SLM/LLM inference

Starred by

Omar Sanseviero(DevRel at Google DeepMind).

SmolChat-Android by shubham0204

2.6%

429

created 8 months ago

Large language model series by Qwen team, Alibaba Cloud

Starred by

Travis Fischer(Founder of Agentic),

Jason Miller(Author of Preact), and

3 more.

Qwen3 by QwenLM

1.3%

24k

updated 4 days ago

Starred by

Lysandre Debut(Chief Open-Source Officer at Hugging Face) and

Thomas Wolf(Cofounder of Hugging Face).

smolvlm-realtime-webcam by ngxson

Webcam demo using SmolVLM for real-time object detection

created 2 months ago

updated 2 months ago

ChatterUI by Vali-98

1.5%

Mobile app frontend for LLMs

updated 3 days ago

Starred by

Lilian Weng(Cofounder of Thinking Machines Lab),

Andrej Karpathy(Founder of Eureka Labs; Formerly at Tesla, OpenAI; Author of CS 231n), and

41 more.

transformers by huggingface

ML library for pretrained model inference and training

148k

created 6 years ago

updated 10 hours ago

orpheus-tts-local by isaiahbjork

Local client for text-to-speech using LM Studio API

445

created 4 months ago

macOS app for local, private voice-to-text transcription

Starred by

Jonathan Ragan-Kelley(Professor at MIT).

VoiceInk by Beingpax

4.8%

macOS app for local LLM chat using files, folders, and websites

updated 10 hours ago

Starred by

Dominik Moritz(Professor at CMU; ML Researcher at Apple).

Sidekick by johnbean393

0.8%

Proxy server for local model swapping with llama.cpp

updated 4 days ago

llama-swap by mostlygeek

3.8%

updated 3 days ago

Starred by

Nathan Lambert(AI Researcher at AI2).

OLMoE.swift by allenai

Swift app for local, offline AI experience

296

VS Code extension for local LLM-assisted code completion

llama.vscode by ggml-org

1.6%

863

created 6 months ago

WavTokenizer by jishengpeng

Research paper for discrete acoustic codec models

created 11 months ago

Starred by

Omar Sanseviero(DevRel at Google DeepMind) and

Lysandre Debut(Chief Open-Source Officer at Hugging Face).

meshgen by huggingface

AI agents for Blender control via natural language

635

CLI tool for terminal context analysis using LLMs

shell_sage by AnswerDotAI

355

created 8 months ago

Starred by

Tim J. Baek(Founder of Open WebUI).

OuteTTS by edwko

TTS interface for unified text-to-speech, treating audio as language

search by kelindar

Go library for embedded vector search and semantic embeddings

478

Starred by

Matteo Collina(Cofounder of Platformatic),

Shawn Wang(Editor of Latent Space), and

1 more.

llama.vim by ggml-org

Vim plugin for LLM-assisted code/text completion

Mobile app for on-device language model inference

pocketpal-ai by a-ghorbani

3.5%

created 11 months ago

Local AI assistant for daily tasks, powered by Llama 3

llama-assistant by nrl-ai

512

GPU cluster manager for AI model deployment

Starred by

Jeff Hammerbacher(Cofounder of Cloudera),

Stas Bekman(Author of Machine Learning Engineering Open Book; Research Engineer at Snowflake), and

1 more.

gpustack by gpustack

1.6%

updated 2 days ago

Starred by

Tobi Lutke(Cofounder of Shopify).

LARS by abgulati

Desktop app for local LLM RAG with detailed citations

602

updated 9 months ago

llama-zip by AlexBuz

LLM-powered lossless compression tool

285

paddler by intentee

Load balancer for llama.cpp servers

799

Starred by

Thomas Wolf(Cofounder of Hugging Face),

Omar Sanseviero(DevRel at Google DeepMind), and

3 more.

huggingface.js by huggingface

JS utils for Hugging Face Hub API interaction

updated 16 hours ago

pi-card by nkasmanoff

Voice assistant for Raspberry Pi

796

Starred by

Peter Norvig(Author of Artificial Intelligence: A Modern Approach; Research Director at Google),

Didier Lopes(Founder of OpenBB), and

14 more.

llm.c by karpathy

LLM training in pure C/CUDA, no PyTorch needed

27k

Starred by

Alex Cheema(Cofounder of EXO Labs) and

Joe Walnes(Head of Experimental Projects at Stripe).

Dot by alexpinel

Local RAG app for document interaction using local LLMs

Starred by

Tobi Lutke(Cofounder of Shopify) and

Jeffrey Morgan(Cofounder of Ollama).

LLocalSearch by nilsherzig

Local search aggregator using LLM agents

Starred by

Simon Willison(Author of Django).

wllama by ngxson

WebAssembly binding for on-browser LLM inference

784

updated 1 week ago

tlm by yusufcanb

Local CLI copilot using Ollama for command-line assistance

AI plugins for Audacity audio editor, running locally

openvino-plugins-ai-audacity by intel

0.8%

AI assistant app mixing AI features with personal productivity

nucleo-ai by AndrewVeee

328

Starred by

Andrej Karpathy(Founder of Eureka Labs; Formerly at Tesla, OpenAI; Author of CS 231n),

Tim J. Baek(Founder of Open WebUI), and

4 more.

gemma.cpp by google

C++ inference engine for Google's Gemma models

updated 23 hours ago

Starred by

Carol Willing(Core Contributor to CPython, Jupyter),

Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems), and

6 more.

minbpe by karpathy

Minimal BPE encoder/decoder for LLM tokenization

10k

Starred by

Tobi Lutke(Cofounder of Shopify),

Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems), and

3 more.

reor by reorproject

Local AI personal knowledge management app

updated 2 months ago

llama-cpp-wasm by tangledgroup

273

Wasm build for llama.cpp

Starred by

Jeffrey Morgan(Cofounder of Ollama) and

Chris Van Pelt(Cofounder of Weights & Biases).

tenere by pythops

Rust TUI for LLM interaction

562

distributed-llama by b4rtaz

CLI tool for distributed LLM inference across networked devices

chatllm.cpp by foldl

C++ inference for real-time chatting with RAG

666

Framework for LLM interaction, function calls, and structured output

updated 2 days ago

Starred by

Travis Fischer(Founder of Agentic).

llama-cpp-agent by Maximilian-Winter

579

LLMUnity by undreamai

Unity package for LLM integration, enabling AI characters

Starred by

Simon Willison(Author of Django).

gguf-tools by antirez

CLI tool for GGUF file manipulation and analysis

276

Starred by

Lysandre Debut(Chief Open-Source Officer at Hugging Face),

Omar Sanseviero(DevRel at Google DeepMind), and

3 more.

PowerInfer by SJTU-IPADS

LLM inference engine for local deployment on consumer GPUs

updated 5 days ago

LLMFarm by guinmoon

iOS/MacOS app for local LLM inference

Starred by

George Hotz(Author of tinygrad; Founder of the tiny corp, comma.ai),

Andrej Karpathy(Founder of Eureka Labs; Formerly at Tesla, OpenAI; Author of CS 231n), and

20 more.

mlx by ml-explore

Array framework for machine learning on Apple silicon

22k

Swift SDK for local LLM interaction on Apple platforms

updated 16 hours ago

LLM.swift by eastriverlee

1.2%

687

Starred by

Tim J. Baek(Founder of Open WebUI),

Simon Willison(Author of Django), and

2 more.

seamless_communication by facebookresearch

Multilingual speech and text translation models for natural communication

12k

updated 8 months ago

Starred by

Andrej Karpathy(Founder of Eureka Labs; Formerly at Tesla, OpenAI; Author of CS 231n),

Lianmin Zheng(Author of SGLang), and

12 more.

gpt-fast by pytorch-labs

PyTorch text generation for efficient transformer inference

Starred by

Andrej Karpathy(Founder of Eureka Labs; Formerly at Tesla, OpenAI; Author of CS 231n),

Anil Dash(Former CEO of Glitch), and

14 more.

llamafile by Mozilla-Ocho

Single-file LLM distribution and runtime via `llama.cpp` and Cosmopolitan Libc

23k

FastAPI service for semantic text search using precomputed embeddings

Starred by

Simon Willison(Author of Django).

swiss_army_llama by Dicklesworthstone

Decoding method for faster LLM generation

Starred by

Stas Bekman(Author of Machine Learning Engineering Open Book; Research Engineer at Snowflake) and

Jeremy Howard(Cofounder of fast.ai).

prompt-lookup-decoding by apoorvumang

1.1%

556

vit.cpp by staghado

C/C++ inference engine for Vision Transformer (ViT) models

289

Web app for describing scenes using a local LLM

llavavision by lxe

490

Starred by

Travis Fischer(Founder of Agentic),

Patrick von Platen(Core Contributor to Hugging Face Transformers and Diffusers), and

8 more.

LLaVA by haotian-liu

Multimodal assistant with GPT-4 level capabilities

23k

Starred by

Jon Bratseth(Cofounder of Vespa) and

Omar Sanseviero(DevRel at Google DeepMind).

qwen.cpp by QwenLM

C++ project for Qwen-LM inference (now integrated into llama.cpp)

602

Starred by

Omar Sanseviero(DevRel at Google DeepMind),

Didier Lopes(Founder of OpenBB), and

2 more.

magentic by jackmpcollins

SDK for integrating LLMs as Python functions using decorators

Starred by

Dan Guido(Cofounder of Trail of Bits) and

Simon Willison(Author of Django).

localpilot by danielgross

Local Copilot alternative for offline use

octogen by dbpunk-labs

Open-source framework for code interpreter agents

257

Starred by

Peter Norvig(Author of Artificial Intelligence: A Modern Approach; Research Director at Google),

Michael Han(Cofounder of Unsloth), and

14 more.

open-interpreter by openinterpreter

Natural language interface for computers

60k

updated 4 days ago

Starred by

Anton Osika(Cofounder of Lovable),

Fabian Hedin(Cofounder of Lovable), and

1 more.

gptme by gptme

CLI tool for terminal agent workflows

C/C++ inference for Meta's Segment Anything Model (SAM)

updated 6 days ago

Starred by

Jonathan Ragan-Kelley(Professor at MIT).

sam.cpp by YavorGIvanov

Starred by

Andrej Karpathy(Founder of Eureka Labs; Formerly at Tesla, OpenAI; Author of CS 231n),

George Hotz(Author of tinygrad; Founder of the tiny corp, comma.ai), and

9 more.

TinyLlama by jzhang38

Tiny pretraining project for a 1.1B Llama model

java-llama.cpp by kherud

Java SDK for local LLaMA model inference

374

React Native binding for llama.cpp

llama.rn by mybigday

1.9%

596

Self-hosted chatbot app for local, private LLM inference

updated 17 hours ago

Starred by

Peer Richelsen(Cofounder of Cal.com),

Travis Fischer(Founder of Agentic), and

2 more.

llama-gpt by getumbrel

0.0%

11k

Jlama by tjake

LLM inference engine for Java applications

Starred by

Tobi Lutke(Cofounder of Shopify),

Andreas Jansson(Cofounder of Replicate), and

4 more.

stable-diffusion.cpp by leejet

C/C++ inference for Stable Diffusion models

updated 17 hours ago

Starred by

Travis Fischer(Founder of Agentic) and

Simon Willison(Author of Django).

LLM-Reading-List by evanmiller

LLM paper list focused on efficient inference and compression

736

C++ port for MiniGPT4 inference

Starred by

Jeffrey Morgan(Cofounder of Ollama).

minigpt4.cpp by Maknee

568

Starred by

Tobi Lutke(Cofounder of Shopify),

Joe Walnes(Head of Experimental Projects at Stripe), and

8 more.

tabby by TabbyML

Self-hosted AI coding assistant for on-prem code completion

32k

Starred by

Tim J. Baek(Founder of Open WebUI).

bark.cpp by PABannier

C/C++ library for fast, local text-to-speech generation using Suno AI's Bark model

834

updated 8 months ago

Starred by

Travis Fischer(Founder of Agentic).

inference by xorbitsai

Model serving library for language, speech, and multimodal models

C++ project for ChatGLM & GLM model inference

chatglm.cpp by li-plus

Starred by

Andrey Vasnetsov(Cofounder of Qdrant).

clip.cpp by monatis

Plain C/C++ CLIP inference, dependency-free

515

Local conversational engine demo with audio

Starred by

Teknium(Cofounder of Nous Research).

talk by yacineMTB

591

Finetuned speech model for speaker diarization

Starred by

Tobi Lutke(Cofounder of Shopify) and

Jonathan Ragan-Kelley(Professor at MIT).

tinydiarize by akashmjn

503

Starred by

Tobi Lutke(Cofounder of Shopify),

Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems), and

6 more.

ctransformers by marella

Python bindings for fast Transformer model inference

Starred by

Jared Palmer(Ex-VP of AI at Vercel; Founder of Turborepo; Author of Formik, TSDX),

Eugene Yan(AI Scientist at AWS), and

1 more.

starcoder.cpp by bigcode-project

C++ example for StarCoder inference

456

LLamaSharp by SciSharp

C#/.NET library for local LLM inference (LLaMA/LLaVA, etc.)

C++ library for local BERT inference

updated 1 week ago

bert.cpp by skeskinen

495

Starred by

Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems),

Woosuk Kwon(Author of vLLM), and

10 more.

WizardLM by nlpxucan

LLMs built using Evol-Instruct for complex instruction following

Desktop app for local LLM inference

alpaca-electron by ItsPi3141

Starred by

Tobi Lutke(Cofounder of Shopify),

Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems), and

9 more.

bark by suno-ai

Generative audio model for realistic speech and sound effects

38k

Unity3d bindings for local speech-to-text inference

whisper.unity by Macoron

1.4%

591

Node.js library for local LLM inference

Starred by

Georgios Konstantopoulos(CTO, General Partner at Paradigm) and

Travis Fischer(Founder of Agentic).

llama-node by Atome-FE

874

llama.go by gotzmann

Go library for local LLM inference, like llama.cpp

updated 10 months ago

LlamaChat by alexrozanski

macOS app for local LLM chats

Self-hosted code completion engine (deprecated)

Starred by

Jeff Hammerbacher(Cofounder of Cloudera),

David Cournapeau(Author of scikit-learn), and

5 more.

turbopilot by ravenscroftj

0.0%

Starred by

Simon Willison(Author of Django) and

Victor Taelin(Author of Bend, Kind, HVM).

rwkv.cpp by RWKV

CPU inference lib for RWKV language model

Starred by

Dan Guido(Cofounder of Trail of Bits) and

Mckay Wrigley(Founder of Takeoff AI).

cheetah by leetcode-mafia

Mac app for crushing tech interviews with AI

Rust CLI tool for LLaMA model inference

Starred by

Georgios Konstantopoulos(CTO, General Partner at Paradigm).

rllama by Noeda

549

Starred by

Tim J. Baek(Founder of Open WebUI),

Pietro Schirano(Founder of MagicPath), and

9 more.

transformers.js by huggingface

Run Transformers models directly in your browser

14k

updated 23 hours ago

Starred by

Anton Troynikov(Cofounder of Chroma),

Lysandre Debut(Chief Open-Source Officer at Hugging Face), and

6 more.

llama-cpp-python by abetlen

Python bindings for llama.cpp, enabling local LLM inference

Starred by

Tobi Lutke(Cofounder of Shopify),

Tim J. Baek(Founder of Open WebUI), and

6 more.

LocalAI by mudler

Open-source OpenAI alternative for local AI inference

34k

LLM playground for local laptop use

Starred by

Tobi Lutke(Cofounder of Shopify),

Anton Osika(Cofounder of Lovable), and

10 more.

openplayground by nat

vigogne by bofenghuang

French LLMs for instruction-following and chat

506

Chinese LLaMA & Alpaca: LLMs for Chinese NLP research

updated 8 months ago

Chinese-LLaMA-Alpaca by ymcui

0.0%

19k

React Native binding for high-performance local speech recognition

whisper.rn by mybigday

1.2%

587

pywhispercpp by absadiki

Python bindings for whisper.cpp, enabling local speech transcription

278

CPU inference for transformer models via a C++ backend

cformers by NolanoOrg

309

Rust ecosystem for LLM Rust inference (unmaintained)

Starred by

Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems),

Shawn Wang(Editor of Latent Space), and

7 more.

llm by rustformers

C++ project for BLOOM model inference

Starred by

Lewis Tunstall(Researcher at Hugging Face),

Salvatore Sanfilippo(Author of Redis), and

4 more.

bloomz.cpp by NouamaneTazi

810

Starred by

Dan Guido(Cofounder of Trail of Bits),

Jeremy Howard(Cofounder of fast.ai), and

2 more.

faster-whisper by SYSTRAN

Faster Whisper reimplementation using CTranslate2

17k

Pybind11 bindings for whisper.cpp

updated 2 months ago

whispercpp by aarnphm

334

Starred by

Zhiqiang Xie(Author of SGLang),

Fabian Hedin(Cofounder of Lovable), and

21 more.

llama by meta-llama

Inference code for Llama 2 models (deprecated)

59k

whisper.net by sandrohanea

.NET library for speech-to-text using Whisper models

773

updated 1 week ago

Whisper by Const-me

GPGPU inference for OpenAI's Whisper ASR model

10k

eepp by SpartanJ

Cross-platform framework for game/application development focused on rich GUIs

485

created 7 years ago

updated 5 days ago

Starred by

Nat Friedman(Former CEO of GitHub) and

Jeff Hammerbacher(Cofounder of Cloudera).

cbp-translate by elanmart

Cyberpunk-style video translation with modern DL stack

AR app for natural language AR experience creation via ChatGPT

ChatARKit by trzy

438

Starred by

Andrej Karpathy(Founder of Eureka Labs; Formerly at Tesla, OpenAI; Author of CS 231n),

Jiayi Pan(Author of SWE-Gym; AI Researcher at UC Berkeley), and

15 more.

flash-attention by Dao-AILab

Fast, memory-efficient attention implementation

19k

created 3 years ago

updated 15 hours ago

teleprompter by danielgross

On-device teleprompter for charismatic speech using quotes

329

Real-time speech-to-text transcription app

transcriber_app by davabase

420

Starred by

Amir Salihefendic(Founder of Doist),

Ross Taylor(Cofounder of General Reasoning; Creator of Papers with Code), and

1 more.

galai by paperswithcode

Scientific language model API

Starred by

Travis Fischer(Founder of Agentic).

buzz by chidiwilliams

Desktop app for offline audio transcription and translation

15k

updated 2 days ago

Starred by

Boris Cherny(Creator of Claude Code; MTS at Anthropic),

Andrej Karpathy(Founder of Eureka Labs; Formerly at Tesla, OpenAI; Author of CS 231n), and

18 more.

whisper by openai

Speech recognition model for multilingual transcription/translation

86k