This project provides a highly customizable, end-to-end AI VTuber solution for streamers, particularly those on Bilibili. It integrates multiple popular AI models for voice conversion, synthesis, image generation, and digital human animation, offering a unified client for management and operation.

How It Works

The AI VTuber leverages a modular architecture, integrating various specialized open-source projects. It uses Zhipu AI API as its language foundation, incorporating direct and associative memory for long-term recall. Knowledge bases are built using graph databases for character and song data, and vector databases for general knowledge, enabling sophisticated persona management and interactive capabilities like singing and drawing.

Quick Start & Requirements

• Installation: Run condaenv.bat to set up the environment (or use provided integration packages).
• Prerequisites: A GPU capable of running Stable Diffusion is recommended. Requires downloading pre-trained models.
• Resources: Full deployment can exceed 20GB.
• Documentation: 文档教程 | 视频效果演示 | 整合包/模型下载

Highlighted Details

• Integrates so-vits-svc4.1 (voice conversion), GPT-SoVITS (voice synthesis), UVR5 (vocal separation), fast-whisper (speech recognition), Stable Diffusion (image generation), and EasyAIVtuber (digital human driver).
• Features a Streamlit-based client for environment management, VTuber customization, auto-starting extensions, and database editing.
• Supports building character prompts, knowledge bases via knowledge graphs and vector databases, and managing multiple VTuber personas.
• Offers a one-stop service for so-vits-svc4.1 training and inference.
• Provides a backend API server for accessing most services via GET/POST requests.

Maintenance & Community

The project is maintained by a university student, with plans for a v2 version focusing on reducing size, developing a desktop application with Electron, and exploring more digital human solutions like Live2D. Updates may be slower due to the author's academic commitments.

Licensing & Compatibility

The README does not explicitly state a license. Integration with various open-source projects implies adherence to their respective licenses. Compatibility for commercial use or closed-source linking is not specified.

Limitations & Caveats

The project is under active development, with some features marked as incomplete. The author is a university student, and future updates might be slow. The initial deployment size is significant, though efforts are planned to reduce it.