This project provides an AI-powered virtual YouTuber (Vtuber) assistant for live streaming on platforms like YouTube and Twitch. It aims to automate audience interaction and content generation, enabling streamers to engage their audience with a virtual persona.

How It Works

The system integrates several AI services: OpenAI's Whisper for speech-to-text transcription, a choice of TTS engines (VoiceVox for Japanese, Seliro TTS for multiple languages), and DeepL or Google Translate for optional text translation. User chat messages are processed, potentially translated, and fed to an LLM (via OpenAI API) for response generation. The LLM's output is then converted to speech using the selected TTS engine, and can be routed to VtubeStudio via virtual audio cables for real-time lip-syncing and animation.

Quick Start & Requirements

• Install dependencies: pip install -r requirements.txt
• Requires an OpenAI API key.
• VoiceVox Engine (local or Colab) or Seliro TTS.
• VtubeStudio for animation.
• VB-Cable for audio routing.
• Twitch API token for Twitch integration.

Highlighted Details

• Supports multiple TTS languages via Seliro TTS (RU, EN, DE, ES, FR, etc.).
• Integrates Whisper OpenAI for robust speech transcription.
• Offers both DeepL (free version) and Google Translate for text translation.
• Configurable character identity and blacklisted users.

Maintenance & Community

Inspired by shioridotdev. Credits given to VoiceVox, DeepL, Whisper OpenAI, and VtubeStudio creators. No explicit community links or roadmap provided in the README.

Licensing & Compatibility

The README does not specify a license. Compatibility for commercial use or closed-source linking is not detailed.

Limitations & Caveats

The project mentions potential installation issues with the Mecab library, suggesting a workaround by disabling a specific katakana conversion function. It also notes that VoiceVox requires Japanese input, necessitating translation for non-Japanese TTS output.