Open-LLM-VTuber provides a hands-free, voice-interactive AI companion with a Live2D avatar, designed for offline, cross-platform use. It caters to users seeking a personalized AI companion for entertainment or assistance, offering a unique blend of real-time conversation, visual perception, and customizable personas.

How It Works

The project integrates multiple open-source components for LLM inference, speech recognition (ASR), and text-to-speech (TTS), enabling a fully offline experience. It supports various backend models and allows users to customize their AI's appearance and persona. Key features include voice interruption (AI won't hear its own voice), visual perception via camera/screen recording, and Live2D expression mapping for dynamic avatar responses.

Quick Start & Requirements

• Installation is detailed in the Quick Start documentation.
• Supports Windows, macOS, and Linux.
• GPU acceleration is available for NVIDIA and some macOS components; CPU fallback is supported.
• Requires configuration for remote access due to HTTPS requirements for microphone functionality.

Highlighted Details

• Extensive Model Support: Integrates with Ollama, OpenAI-compatible APIs, Gemini, Claude, Mistral, DeepSeek, GGUF, LM Studio, vLLM for LLMs; sherpa-onnx, FunASR, Faster-Whisper, Whisper.cpp for ASR; and sherpa-onnx, pyttsx3, MeloTTS, Coqui-TTS, GPTSoVITS, Bark, CosyVoice for TTS.
• Advanced Interaction: Features visual perception, voice interruption, touch feedback, Live2D expression mapping, and a transparent desktop pet mode.
• High Customizability: Allows easy module configuration, custom Live2D model import, prompt-based persona shaping, and voice cloning.
• Flexible Agent Implementation: Supports integration of various agent architectures via an Agent interface.

Maintenance & Community

• Active development with a public Roadmap.
• Community support available via Discord and QQ groups.
• Version v1.0.0 introduced breaking changes requiring re-deployment.

Licensing & Compatibility

• The project is licensed under MIT.
• Live2D sample models are subject to separate licensing terms from Live2D Inc. and may have restrictions for commercial use.

Limitations & Caveats

The project is in its early stages with active development and breaking changes (e.g., v1.0.0). Long-term memory functionality was temporarily removed but is planned for reintroduction. Commercial use of provided Live2D sample models requires adherence to Live2D Inc.'s licensing.