Summary

This project provides a detailed guide and codebase for building a desktop AI robotic cat, the "AICat," powered by the Seeed XIAO ESP32S3 Sense. It integrates visual display, audio input/output, and multi-servo animation, controlled via a web interface and driven by the Qwen3.5 Omni large language model for interactive dialogue. Aimed at hardware enthusiasts, it offers a replicable platform for DIY AI robotics projects.

How It Works

The AICat utilizes a Seeed XIAO ESP32S3 Sense for core processing, managing a ST7789 SPI display for expressions, an onboard camera for video streaming, and a microphone for voice capture. Audio output is handled by a MAX98357A amplifier. Servo motors, including standard PWM types for facial features and STS3032 bus servos for leg articulation, are controlled via PCA9685 drivers. A Python backend, running on a PC, performs Automatic Speech Recognition (ASR) and AI dialogue using the Qwen3.5 Omni model, exposing a web interface for real-time control and monitoring of all integrated systems.

Quick Start & Requirements

• Hardware: Seeed XIAO ESP32S3 Sense (camera/mic version), ST7789 SPI display (170x320 recommended), MAX98357A amplifier, PCA9685 PWM servo driver, STS3032 bus servos, standard PWM servos, external servo power supply (common GND required).
• Software: Python 3.10-3.12, Arduino IDE with ESP32 board support (install esp32 by Espressif Systems), specific Arduino libraries (ESP32Servo, Adafruit GFX, Adafruit ST7735 and ST7789, Adafruit PWM Servo Driver, ArduinoWebsockets, JPEGDEC, SCServo). Python backend dependencies installed via pip install -r requirements.txt.
• Setup: Configure Arduino IDE (PSRAM enabled, appropriate Partition Scheme). Set DASHSCOPE_API_KEY in the backend's .env file. Upload ESP32 firmware (integrated.ino), configuring Wi-Fi credentials and backend server IP. Run Python backend (python app.py) and access the web interface via the PC's LAN IP.
• Assets: Expression/animation assets are not included and must be generated locally using provided Python scripts (e.g., generate_all_headers.py) and flashed to the ESP32's LittleFS using dedicated Arduino sketches (flash_files.ino, mouth_flash_files.ino).
• Links: ESP32 Firmware (upload_facial_expression/integrated/integrated.ino), Python Backend (upload_facial_expression/integrated/server/app.py), Backend Requirements (upload_facial_expression/integrated/server/requirements.txt), Asset Generation Tools (in upload_facial_expression/flash_files/ and upload_facial_expression/mouth_flash_files/), Partition Scheme Docs (upload_facial_expression/分区表说明.md).

Highlighted Details

• Multi-Modal Interaction: Seamlessly integrates visual feedback (screen expressions, camera), audio input (microphone for ASR), and audio output (speaker).
• Advanced Actuation: Supports both standard PWM servos for basic movements and serial bus STS3032 servos for complex leg articulation (walking, jumping).
• Web-Based Debugging: A Python backend provides a web interface for real-time monitoring of video streams, servo positions, and system status.
• LLM Integration: Utilizes Qwen3.5 Omni for conversational AI capabilities, requiring a DashScope API key.

Maintenance & Community

No specific details regarding project maintainers, community channels (e.g., Discord, Slack), or roadmap are provided in the README. The focus is on the technical implementation and build process.

Licensing & Compatibility

The provided README does not specify a software license. This absence prevents an assessment of its terms for commercial use, modification, or distribution.

Limitations & Caveats

• Hardware Specificity: Requires precise hardware components, particularly the Seeed XIAO ESP32S3 Sense variant.
• Power Requirements: Servo operation necessitates a stable, separate external power supply with a common ground connection to the ESP32 to prevent system instability.
• Asset Generation Overhead: Users must locally generate all visual expression and animation assets using provided scripts, which can be time-consuming and resource-intensive.
• Flash Management Complexity: Correctly configuring ESP32 partition schemes and flashing assets requires careful attention to detail to avoid data loss or insufficient storage.
• API Key Dependency: Core AI functionality relies on a DashScope API key, incurring potential costs and external service dependency.