Search results
90 results for "model serving"
Showing 1 - 25 of 9025 of 90 repos
Rows
| Repository | Description | Stars | Stars 7d Δ | Stars 7d % | PRs 7d Δ | Created | Response rate | Issues 30d | Last active | |
|---|---|---|---|---|---|---|---|---|---|---|
1 | A high-performance serving system for machine learning models in production. Supports model versioning, gRPC & HTTP endpoints, & batch sched... | 6k Top 10% | 3 | 0.1% | 0 | 9y ago | 1 week | 1w ago | ||
2 | ServingPaddlePaddle | A high-performance, flexible, and easy-to-use industrial-grade online inference service for deep learning models. Supports multiple protocol... | 914 Top 50% | 1 | 0.1% | 0 | 6y ago | Inactive | 2mo ago | |
3 | Framework to build model inference APIs and serving systems for any AI/ML model. Supports optimization features like dynamic batching. | 8k Top 10% | 23 | 0.3% | 4 | 6y ago | 1 day | 3d ago | ||
4 | Toolkit to serve ML models in Docker containers using SageMaker. Implements a model serving stack deployable to SageMaker, built on Multi Mo... | 407 | 1 | 0.3% | 0 | 6y ago | 1 week | 1y ago | ||
5 | Routes Claude Code requests to different models. Uses one model for request routing and others for tool invocation, coding, and reasoning.
| 9k Top 10% | 1,813 | 23.3% | 10 | 5mo ago | Inactive | 1d ago | ||
6 | mosecmosecorg | High-performance, cloud-friendly model serving framework. It supports dynamic batching, pipelined stages, and Prometheus monitoring metrics. | 849 Top 50% | 0 | 0% | 2 | 4y ago | 1 day | 1d ago | |
7 | Open-source platform for the machine learning lifecycle, including experiment tracking, model packaging, registry, serving, evaluation, & ob... | 21k Top 5% | 103 | 0.5% | 103 | 7y ago | 1 day | 16h ago | ||
8 | MLServerSeldonIO | Open source inference server to serve ML models via REST and gRPC, compliant with KFServing's V2 Dataplane. Supports multi-model serving.
| 831 Top 50% | 4 | 0.5% | 6 | 5y ago | 1 day | 5d ago | |
9 | Kubernetes CRD for serving predictive and generative ML models. Supports autoscaling, canary rollouts, & standardized data plane protocols.
| 4k Top 25% | 22 | 0.5% | 14 | 6y ago | Inactive | 2d ago | ||
10 | Framework to package, test, and deploy AI/ML models. Supports any Python framework, with a fast dev loop and batteries-included environment.... | 1k Top 50% | 5 | 0.5% | 15 | 3y ago | 1 day | 17h ago | ||
11 | Open-source engine for fine-tuning and serving LLMs like LLaMA, MPT, and Falcon. Features optimized inference and Hugging Face integrations.... | 808 Top 50% | 0 | 0% | 0 | 2y ago | Inactive | 2w ago | ||
12 | Open platform for training, serving, and evaluating LLM-based chatbots. Powers Chatbot Arena, a large-scale LLM conversation dataset. | 39k Top 1% | 30 | 0.1% | 0 | 2y ago | 1 week | 2mo ago | ||
13 | Proxy server for llama.cpp that enables automatic model swapping. Supports OpenAI API endpoints and custom endpoints for monitoring and cont... | 1k Top 50% | 41 | 3.8% | 8 | 10mo ago | 1 day | 2d ago | ||
14 | Framework to streamline training of ML models on Jax/Flax. Supports Transformers, Mamba, RWKV, vision models, DPO, and efficient inference. | 294 | 2 | 0.7% | 3 | 2y ago | 1 day | 3d ago | ||
15 | Node.js package for question answering using TFJS and 🤗Transformers. Supports DistilBERT, BERT, and RoBERTa models in SavedModel and TFJS f... | 466 | 0 | 0% | 0 | 5y ago | 1+ week | 2y ago | ||
16 | Generic serving service for machine learning models. Supports distributed TensorFlow, RESTful APIs, GPU inference, and multiple model types. | 757 Top 50% | 1 | 0.1% | 0 | 7y ago | 1 week | 4mo ago | ||
17 | Lightweight tool for prompt design and iteration with foundation models. Supports caching, unified API, and multiple model providers. | 444 | 0 | 0% | 0 | 3y ago | 1+ week | 1y ago | ||
18 | Defines desired behaviors for models via a specification. Includes evaluation prompts to test model performance in challenging situations.
| 524 | 36 | 7.1% | 0 | 5mo ago | Inactive | 3mo ago | ||
19 | Framework to build & deploy AI services via gRPC, HTTP, & WebSockets. Supports scaling, streaming, dynamic batching, & LLM serving. | 22k Top 5% | 18 | 0.1% | 0 | 5y ago | Inactive | 4mo ago | ||
20 | BentoDiffusionbentoml | Deploy Stable Diffusion models (SDXL Turbo, SD 3, ControlNet, etc) for image/video generation and manipulation, with BentoML and BentoCloud.... | 373 | 0 | 0% | 0 | 2y ago | 1 week | 3mo ago | |
21 | byzer-llmallwefantasy | Ray-based platform for LLM lifecycle management: pretraining, fine-tuning, deployment, and serving. Supports Python/SQL APIs and model quant... | 311 | 1 | 0.3% | 0 | 2y ago | 1 week | 2w ago | |
22 | model_serveropenvinotoolkit | High-performance system for serving models. It applies OpenVINO for inference, and supports gRPC/REST APIs, multiple frameworks & accelerato... | 745 Top 50% | 1 | 0.1% | 15 | 6y ago | Inactive | 23h ago | |
23 | model.nvimgsuuon | A Neovim plugin for AI completions and chat. It supports OpenAI, local models (llama.cpp, Ollama), programmatic prompts, and streaming.
| 382 | 0 | 0% | 0 | 2y ago | 1 day | 1w ago | |
24 | A collection of machine learning models for vision (CNNs), text (RNNs, NLP), and games (Reinforcement Learning). Ready-to-use starting point... | 931 Top 50% | 0 | 0% | 0 | 8y ago | 1 week | 8mo ago | ||
25 | model-cataloglmstudio-ai | Standardized JSON descriptors for Large Language Models (LLMs). Captures model size, architecture, file format, and quantization details.
| 799 Top 50% | 2 | 0.3% | 0 | 2y ago | 1 day | 1y ago |
Showing 1 - 25 of 9025 of 90 repos
Rows
Feedback? Help us improve.