This toolkit provides Unity developers with a streamlined way to integrate AI chat functionalities, including large language models (LLMs) and speech services, into cross-platform applications. It targets developers building interactive AI experiences, offering rapid implementation of features like text-based chat, speech-to-text, text-to-speech, and voice-activated wake words.

How It Works

The toolkit is structured into distinct modules: LLM for API calls to various large language models (ChatGPT, ChatGLM, RWKV, Xunfei Spark, Baidu Qianfan, Zhipu AI, Ollama, DeepSeek), and TTS/STT for speech synthesis and recognition via web APIs from Microsoft Azure, Baidu AI, OpenAI Whisper, and Xunfei. It also includes a WOV module for keyword voice wake-up using Unity's built-in speech library on Windows and integrates Oculus Lipsync for character lip-syncing.

Quick Start & Requirements

• Install: Requires Unity 2020.3.44 or later.
• Dependencies: API keys for services like OpenAI, Azure, Baidu AI, or deployment endpoints for self-hosted models (e.g., ChatGLM, RWKV Runner, Ollama).
• Platforms: Supports Windows, WebGL, and Android. WebGL requires specific code adjustments due to microphone class limitations.
• Docs: Example scenes demonstrate usage; configuration involves setting model and speech scripts in the Unity inspector.

Highlighted Details

• Integrates a wide array of LLMs including OpenAI, ChatGLM, RWKV, and cloud providers like Baidu and Xunfei.
• Supports multiple speech services (TTS/STT) from Azure, Baidu, OpenAI, and Xunfei, with both web API and WebSocket options.
• Includes voice wake-word functionality for Windows and Oculus Lipsync integration for character animation.
• Offers compatibility with self-hosted LLMs via Ollama and RWKV Runner.

Maintenance & Community

The project appears to be a personal integration effort, with links provided to older, more feature-rich versions on Gitee. No explicit community channels or active development signals are present in the README.

Licensing & Compatibility

The README does not specify a license. Compatibility for commercial use or closed-source linking is not detailed.

Limitations & Caveats

The project is presented as a consolidation of previous work, with core AI models and character assets removed. WebGL support for microphone input requires manual code adjustments beyond the provided Unity configuration. The lack of a specified license may pose adoption risks.