This project transforms a Big Mouth Billy Bass animatronic into an AI-powered voice assistant using a Raspberry Pi and OpenAI's Realtime API. It targets hobbyists and makers seeking to blend nostalgic entertainment with modern AI capabilities, offering interactive conversations and synchronized physical animations. The primary benefit is breathing new life into a classic toy with advanced features like customizable personalities and smart home control.

How It Works

The system leverages a Raspberry Pi to stream conversations in real-time via the OpenAI Realtime API. It processes spoken input, generates responses using AI, and translates these into physical actions: the animatronic's mouth, head, and tail move in sync with the dialogue. Key architectural choices include a modular design supporting different Billy Bass hardware versions, a personality system configurable via persona.ini, and integration with external services like Home Assistant and custom song playback. This approach allows for a dynamic, interactive experience beyond typical voice assistants.

Quick Start & Requirements

• Primary Install/Run: Clone the repository, set up a Python virtual environment, install dependencies (pip3 install -r ./requirements.txt), and run the main script (python3 main.py) or enable systemd services for background operation.
• Prerequisites: Raspberry Pi (Pi 5 recommended), Big Mouth Billy Bass (Classic or Modern), USB microphone, speaker, microSD card, Raspberry Pi OS Lite (64-bit), Python 3.11+, git, ffmpeg, libportaudio2, liblgpio-dev, swig. An OpenAI API key is mandatory. Optional: Pi Camera or USB webcam.
• Setup: Detailed instructions cover OS flashing, SSH setup, system updates, sound configuration, and software installation. Estimated setup time is several hours.
• Links:
  • Pre-assembled versions: https://thingsfromthom.etsy.com/
  • OpenAI API Keys: https://platform.openai.com/api-keys
  • Picovoice (for wake word): https://console.picovoice.ai/

Highlighted Details

• Real-time Animated Conversation: Utilizes OpenAI's Realtime API for low-latency voice interaction and synchronizes Billy's physical movements (mouth, tail, head) with speech.
• Configurable Personality & Backstory: Allows deep customization of Billy's character, traits (humor, sarcasm, etc.), and origin story through the persona.ini file and web UI.
• Multi-modal Capabilities: Integrates a news digest tool, camera vision for object description (Pi Camera/USB webcam support), and a custom song manager with synchronized animations.
• Smart Home Integration: Seamlessly controls Home Assistant devices via its Conversation API, enabling voice commands for home automation.
• Local Wake Word: Supports fully local wake-word detection using Porcupine or openWakeWord, enhancing privacy and responsiveness.
• Web Configuration Interface: Provides a lightweight UI for managing settings, API keys, personas, logs, and system services.

Maintenance & Community

The project is maintained by Thokoop and appears to be a personal project focused on experimentation and fun. Community interaction and support primarily occur through GitHub Issues for bug reports, feature requests, and discussions. No specific community channels like Discord or Slack are mentioned.

Licensing & Compatibility

The README states "See LICENSE for full details" but does not specify the license type within the provided text. It is described as free for personal and educational use. Compatibility is primarily for Raspberry Pi hardware. Commercial use implications require consulting the LICENSE file.

Limitations & Caveats

This project is explicitly marked as being in BETA, meaning users should expect potential instability, crashes, or unexpected behavior. It requires specific hardware (Raspberry Pi and a Billy Bass animatronic) and significant configuration effort. The use of OpenAI's API incurs costs based on usage.