airunner  by Capsize-Games

Local AI inference engine for text, images, speech, and agent workflows

created 2 years ago
1,216 stars

Top 32.9% on sourcepulse

GitHubView on GitHub
Project Summary

AI Runner is an open-source, local-first application designed to run various AI models, including LLMs, Stable Diffusion, TTS, and STT, without cloud dependencies. It targets developers and end-users seeking an all-in-one, offline inference engine for prototyping, private data processing, or custom UI development.

How It Works

AI Runner leverages HuggingFace and Llama-index libraries to provide a unified interface for diverse AI tasks. Its architecture supports local LLM inference, Stable Diffusion image generation, text-to-speech, and speech-to-text. The project emphasizes a "local-first" approach, ensuring data privacy and offline functionality, while offering a plugin and extension API for customization and integration into other Python projects.

Quick Start & Requirements

  • Install/Run: Docker is recommended. After installing NVIDIA Container Toolkit, run ./src/airunner/bin/docker.sh airunner.
  • Prerequisites: NVIDIA GPU (RTX 3060 minimum, RTX 4090 recommended), Docker, NVIDIA Container Toolkit.
  • Storage: 22 GB minimum, 50 GB recommended for models.
  • Links: Installation Wiki, Discord

Highlighted Details

  • Unified interface for LLMs, Stable Diffusion, TTS, and STT.
  • Real-time voice conversations with LLMs.
  • Supports Stable Diffusion 1.5, SDXL, Turbo, ControlNet, LoRA, and embeddings.
  • Plugin and extension API for custom features and UI.
  • Can be installed as a Python library (pip install airunner).
  • Offers memory optimization techniques like TF32 Mode and VAE/Attention Slicing.

Maintenance & Community

The project welcomes contributions and provides a Discord server for questions and ideas. Detailed contribution guidelines are available in the repository.

Licensing & Compatibility

The repository does not explicitly state a license in the README. This requires further investigation for commercial use or closed-source linking.

Limitations & Caveats

The README does not specify a license, which is a critical factor for adoption, especially for commercial or closed-source projects. While Docker simplifies setup, it requires the NVIDIA Container Toolkit for GPU acceleration.

Health Check
Last commit

2 weeks ago

Responsiveness

1 day

Pull Requests (30d)
3
Issues (30d)
0
Star History
796 stars in the last 90 days

Explore Similar Projects

Starred by Jeff Hammerbacher Jeff Hammerbacher(Cofounder of Cloudera), Stas Bekman Stas Bekman(Author of Machine Learning Engineering Open Book; Research Engineer at Snowflake), and
2 more.

gpustack by gpustack

1.6%
3k
GPU cluster manager for AI model deployment
created 1 year ago
updated 2 days ago
Starred by Andrej Karpathy Andrej Karpathy(Founder of Eureka Labs; Formerly at Tesla, OpenAI; Author of CS 231n), Nat Friedman Nat Friedman(Former CEO of GitHub), and
32 more.

llama.cpp by ggml-org

0.4%
84k
C/C++ library for local LLM inference
created 2 years ago
updated 12 hours ago
Feedback? Help us improve.