ChatWaifu_Mobile  by Voine

Mobile app for AI character chat

created 2 years ago
1,328 stars

Top 30.9% on sourcepulse

GitHubView on GitHub
Project Summary

This project provides a mobile application for Android that allows users to interact with AI-powered virtual companions, specifically anime-styled characters. It aims to offer a personalized and engaging experience for users interested in AI chatbots and virtual waifus, featuring local VITS speech synthesis and Live2D character rendering.

How It Works

The application leverages several on-device technologies for a rich, interactive experience. It uses VITS (Variational Inference with adversarial learning for Text-to-Speech) via ncnn for local speech synthesis, allowing for offline voice generation. Live2D Cubism SDK (native C++ version) is used for rendering animated 2D character models. Speech input is handled by Sherpa-ncnn, and lip-syncing is integrated using Meta's lipSync library, though currently implemented as a looping animation due to synchronization challenges. The core architecture follows Retrofit, MVVM, LiveData/Flow, and Room patterns, with inter-process communication handled via AIDL.

Quick Start & Requirements

  • Installation: APK installation from source.
  • Prerequisites: Android device.
  • Dependencies: OpenAI API key for ChatGPT, optional Baidu Translate API key.
  • Model Replacement: VITS models require conversion to ncnn format and placement in chatwaifu/vits/, Live2D models in chatwaifu/live2d/.
  • Resources: Local model processing may require significant device resources.
  • Documentation: VITS conversion, Live2D, Sherpa-ncnn, Meta LipSync.

Highlighted Details

  • On-device VITS speech synthesis for offline voice generation.
  • Native Live2D rendering for animated character visuals.
  • Local speech recognition via Sherpa-ncnn.
  • Support for custom VITS and Live2D models.

Maintenance & Community

Licensing & Compatibility

  • License: Not explicitly stated in the README.
  • Commercial Use: Models are explicitly prohibited from commercial use. The software itself is not licensed for commercial use.

Limitations & Caveats

  • The lip-sync functionality is currently a looping animation due to synchronization issues.
  • The README warns that non-Japanese VITS models may have poor playback quality if translation is not used.
  • Model usage is restricted to non-commercial purposes.
Health Check
Last commit

2 years ago

Responsiveness

1 day

Pull Requests (30d)
0
Issues (30d)
0
Star History
47 stars in the last 90 days

Explore Similar Projects

Feedback? Help us improve.