This project provides an open-source framework for building a personal AI chatbot companion using ESP32 microcontrollers. It targets hobbyists, students, and developers interested in integrating large language models (LLMs) with hardware, enabling custom AI experiences with voice interaction and display capabilities.

How It Works

The system leverages ESP32 for embedded processing, integrating offline voice wake-up (ESP-SR), multi-language speech recognition, and voiceprint identification (SenseVoice). It supports streaming voice dialogue via WebSocket or UDP, connecting to various LLMs like Qwen, DeepSeek, and Doubao. Text-to-speech is handled by 3D Speaker TTS engines (Volcano Engine or CosyVoice), with configurable prompts and voice tones for personalized AI characters. Short-term memory is implemented through per-dialogue summarization.

Quick Start & Requirements

• Firmware Flashing: Users can flash pre-compiled firmware via a no-development-environment method, defaulting to the official xiaozhi.me server for free Qwen model access.
• Development Environment: Requires ESP-IDF (SDK version 5.3+) with VS Code or Cursor, and a Linux environment is recommended for faster compilation and fewer driver issues.
• Hardware: ESP32-S3 based boards are primarily supported, with a growing list of compatible development boards and accessories.

Highlighted Details

• Supports Wi-Fi and Cat.1 4G connectivity.
• Features boot button wake-up/interrupt with click and long-press triggers.
• Includes OLED/LCD display support for signal strength or dialogue content, with image/emoji display.
• Offers short-term memory and self-summarization for conversational context.

Maintenance & Community

The project is actively developed and maintained by "虾哥" (XiaGe). Community engagement is encouraged via QQ groups (376893254) and GitHub Issues.

Licensing & Compatibility

Released under the MIT license, permitting free use and commercial application.

Limitations & Caveats

The project is primarily focused on ESP32-S3 hardware, and while a server component is mentioned, its setup and private deployment details are linked to a separate repository.