This project provides an open-source, real-time 3D digital human powered by Unity, targeting developers and researchers interested in creating interactive AI-driven characters. It integrates speech recognition, LLM-based conversational AI, and text-to-speech with lip-syncing for a lifelike experience.

How It Works

The system processes user microphone input through speech recognition, feeds the text to a chosen LLM API for response generation, and then uses text-to-speech (TTS) for audio output. Lip synchronization is achieved using the uLipSync package, ensuring mouth movements match the synthesized speech. The architecture supports various LLM APIs and is built on Unity's Universal Render Pipeline (URP) for cross-platform compatibility.

Quick Start & Requirements

• Install: Requires Unity 2022.3.31 or later.
• Prerequisites: LLM API key and configuration (URL, model name, optional role settings). Integration with the Fay digital human framework is supported.
• Resources: Downloadable runtime package available via Baidu Netdisk.
• Documentation: Reference video available at https://www.bilibili.com/video/BV1YJ4m1T7Rj?t=2.7.

Highlighted Details

• Utilizes URP for rendering and supports all platform releases.
• Integrates with various LLM APIs (e.g., Doubao, Zhipu, Kimi).
• Features uLipSync for accurate lip-syncing.
• Designed for extensibility and ease of use.

Maintenance & Community

The project is maintained by LKZMuZiLi. Community engagement is encouraged via WeChat for group invitations.

Licensing & Compatibility

The specific license is not explicitly stated in the README, but the project is presented as an open-source version derived from a primary project. Compatibility for commercial use or closed-source linking would require clarification on licensing terms.

Limitations & Caveats

Users must configure LLM API details themselves. Older Unity versions may require manual handling of URP package errors. The project relies on external services for LLM and TTS, and the availability and performance of these services are external dependencies.