RealChar provides a unified platform for creating, customizing, and interacting with AI characters in real-time across web, mobile, and terminal interfaces. It targets AI enthusiasts and developers looking to build personalized AI companions, leveraging a modular architecture and a variety of leading AI services for natural language processing, speech, and data management.

How It Works

RealChar employs a modular architecture, allowing users to swap components for LLMs (OpenAI, Anthropic, Anyscale), Speech-to-Text (Whisper, Google), and Text-to-Speech (ElevenLabs, Edge TTS). It uses FastAPI for the backend, React for the web frontend, and Swift for the mobile app, with WebSockets for real-time communication. Data ingestion and LLM orchestration are handled by LlamaIndex and LangChain, with Chroma Vector DB for memory.

Quick Start & Requirements

• Docker: docker compose up (requires Docker installation).
• Python: git clone the repository, install dependencies (pip install -r requirements.txt), set up .env with API keys, and run python cli.py run-uvicorn for the backend and python cli.py next-web-dev for the frontend.
• Prerequisites: Python 3.x, Docker, portaudio, ffmpeg (>=4.4). Optional: CUDA for GPU acceleration.
• API Keys: OpenAI, Anthropic, ElevenLabs, ReByte, or Anyscale API keys are required for full functionality.
• Docs: RealChar.ai

Highlighted Details

• Supports multiple LLMs including OpenAI GPT-3.5/4, Anthropic Claude 2, and Anyscale Llama2.
• Offers local and cloud-based Speech-to-Text (WhisperX, Google) and Text-to-Speech (ElevenLabs, Edge TTS) with voice cloning capabilities.
• Provides multi-platform support (Web, Mobile via Swift, Terminal).
• Modular design allows for easy component swapping and customization.

Maintenance & Community

• Active development with a roadmap including RAG, Agents, and Session Management.
• Community support available via Discord.

Licensing & Compatibility

• The repository does not explicitly state a license in the README. Users should verify licensing terms before commercial use.

Limitations & Caveats

• Requires significant API key setup for full functionality.
• Twilio integration for phone calls is limited to US numbers and ElevenLabs voices.
• Some advanced features are noted as being powered by the ReByte platform, implying potential dependencies or costs.