This project provides a local, real-time AI voice chat experience using the Zephyr 7B model, targeting users who want a customizable, offline conversational AI. It enables interactive voice communication with an AI that has a user-defined personality and voice.

How It Works

The system integrates Zephyr 7B via llama_cpp for natural language processing, faster_whisper (via RealtimeSTT) for fast speech-to-text, and Coqui XTTS (via RealtimeTTS) for high-quality, real-time text-to-speech. This stack allows for low-latency, end-to-end voice interaction without relying on external cloud services.

Quick Start & Requirements

• Install: pip install RealtimeSTT==0.1.7 RealtimeTTS==0.2.7 and pip install llama-cpp-python --force-reinstall --upgrade --no-cache-dir --verbose.
• Prerequisites: Requires a GPU with ~8GB VRAM. NVIDIA users need CUDA 11.8 and cuDNN 8.7.0. AMD users need ROCm 5.7.1. FFmpeg is also required.
• Model: Download zephyr-7b-beta.Q5_K_M.gguf and configure creation_params.json.
• Docs: https://github.com/KoljaB/LocalAIVoiceChat

Highlighted Details

• Real-time, fully local AI voice chat.
• Customizable AI personality and voice cloning via .wav files.
• Supports NVIDIA CUDA and AMD ROCm.
• Experimental alpha state with potential dependency issues.

Maintenance & Community

• Project author: Kolja Beigel (kolja.beigel@web.de).
• Open to contributions via pull requests.

Licensing & Compatibility

• License: Coqui Public Model License 1.0.0.
• Restrictions: Strictly non-commercial use for the model and its outputs.

Limitations & Caveats

The software is in an experimental alpha state and may lack production stability. The XTTS model can exhibit glitches, and Zephyr 7B's quality is not comparable to larger models like GPT-4. Dependency conflicts may require manual installation of specific library versions.