This repository provides a framework for building interactive digital humans, targeting developers and enthusiasts who want to create AI-powered companions. It simplifies the process by integrating modular components for speech recognition, large language models, text-to-speech, and agent capabilities, with support for Live2D character models and web-based deployment.

How It Works

The project leverages a modular architecture, allowing for flexible integration of various AI services. It supports different interaction modes, including chat, digital human-focused, and an upcoming immersive mode. The framework is designed to be easily extended with custom agents and character models, facilitating the creation of personalized digital human experiences.

Quick Start & Requirements

• Install/Run: Docker is the primary deployment method.
• Prerequisites: Docker, potentially API keys for ASR, LLM, and TTS services (e.g., Baidu, Google, OpenAI, Edge).
• Resources: Low configuration requirements (as low as 2 cores, 2GB RAM).
• Links:
  • Demo: https://github.com/user-attachments/assets/0650c443-d19b-4d5e-a409-5eed9a9ea9f7
  • Deployment Tutorials: B站视频教程-部署, B站视频教程-All-in-Dify部署 (Note: Actual links are placeholders as per instructions)

Highlighted Details

• Supports Docker for rapid deployment and low resource footprint (2-core/2GB).
• Integrates with orchestration frameworks like Dify and FastGPT.
• Modular design supports ASR, LLM, TTS, and Agent extensions.
• Features Live2D character model support and control.
• Frontend supports PC and mobile web access with Next.js, NextUI, and Tailwind CSS.

Maintenance & Community

• Active development with recent updates (v2.0.0 in August 2024).
• Community engagement via WeChat groups and Bilibili tutorials.
• Resources include articles on RAG, Dify, and digital human concepts.

Licensing & Compatibility

• The repository does not explicitly state a license in the provided README. Compatibility for commercial use or closed-source linking is not specified.

Limitations & Caveats

The "Immersive Mode" for direct voice interaction is currently a reserved feature and not yet supported. The project relies on external API services for core AI functionalities, which may incur costs or have usage limitations.