AI-YinMei is an open-source project that enables users to create an AI-powered virtual anchor (Vtuber) capable of live streaming interactions. It targets content creators and hobbyists looking to build interactive AI personalities for platforms like Bilibili, offering features like real-time chat, singing, dancing, and image generation.

How It Works

The project integrates multiple AI models and services to achieve its functionality. It leverages large language models (LLMs) via FastGPT, One-API, and Xinference for conversational abilities, with options for different LLM backends like ChatGLM. Speech synthesis is handled by Bert-VITS2 and GPT-SoVITS, while Microsoft Edge-TTS provides a built-in option. Visuals are driven by Stable Diffusion for image generation and VTube Studio for Vtuber avatar control, with integrations for OBS for live streaming.

Quick Start & Requirements

• Installation: Download the integrated package (e.g., yinmei-all) and run start.bat.
• Prerequisites: Python 3.11.6, NVIDIA GPU (for N-card version), Virtual Audio Cable, Voicemeeter, ffmpeg, MongoDB.
• Resources: Requires significant disk space for models and dependencies. Setup involves configuring multiple services and potentially downloading large model files.
• Documentation: Development Documentation, Tutorials

Highlighted Details

• Supports Bilibili live stream integration for real-time chat and welcome messages.
• Features dynamic AI personality switching between "gentle maid" and "tsundere" modes.
• Enables AI singing with automatic accompaniment and dancing with pre-defined video clips.
• Includes NSFW image detection and content filtering capabilities.

Maintenance & Community

• Actively developed by Winlone, with a Q group (27831318) for community support and discussion.
• The project acknowledges contributions from various open-source projects in AI TTS, LLM, and Stable Diffusion.

Licensing & Compatibility

• The project itself appears to be open-source, but it integrates several third-party projects with their own licenses (e.g., FastGPT, Stable Diffusion WebUI, Bert-VITS2, GPT-SoVITS). Users must ensure compatibility with their intended use, especially for commercial applications.

Limitations & Caveats

• The project relies heavily on specific versions of Python and requires significant configuration of multiple external services.
• Many dependencies are distributed via Baidu Netdisk, which can be prone to removal and requires joining a group for access.
• The "N Card version" implies a dependency on NVIDIA hardware for optimal performance.