This project provides a backend service for the xiaozhi-esp32 hardware project, enabling users to quickly set up an ESP32 device control server. It targets users who have ESP32 hardware and wish to independently host their own backend, offering features like smart dialogue, IoT control, and OTA updates.

How It Works

The service implements the xiaozhi-esp32 communication protocol using WebSockets for data exchange. It supports various LLM and TTS modules for natural language interaction, intent recognition via LLMs or function calls, and multi-language ASR. The architecture emphasizes flexibility, allowing users to switch between different LLM/TTS providers and offering memory management options.

Quick Start & Requirements

• Installation: Docker deployment is recommended for ease of use. Source code deployment is also available.
• Prerequisites: ESP32 hardware is required for the client-side. The server supports various LLM, TTS, VAD, and ASR services, some of which may require API keys or specific configurations.
• Resources: A demo platform is available at https://2662r3426b.vicp.fun.

Highlighted Details

• Supports flexible LLM integration (OpenAI API, Ollama, Dify, Fastgpt, Coze) and TTS modules (EdgeTTS, Volcengine, Tencent, etc.).
• Features intent recognition via LLM function calls and multi-language support (Mandarin, Cantonese, English, Japanese, Korean).
• Includes an IoT function for managing registered devices and context-aware control.
• Offers a web-based "Smart Console" for managing agents, users, and system configurations.

Maintenance & Community

The project is part of a larger "xiaozhi" ecosystem, with related projects including Android/iOS clients and a Java backend. Key contributors and design inspirations are acknowledged.

Licensing & Compatibility

The project is open-source, but the README does not explicitly state a license. Users are warned against using it in production environments due to incomplete functionality and lack of network security assessment.

Limitations & Caveats

The project is explicitly stated as not production-ready and has not undergone network security assessment. Users are advised to implement necessary protections if deploying in public networks. The project does not guarantee the service quality or financial security of third-party API providers.