This project demonstrates a real-time speech assistant using Node.js, Twilio Voice Media Streams, and OpenAI's Realtime API. It enables users to have two-way voice conversations with an AI assistant over a phone call, targeting developers building voice-enabled AI applications.

How It Works

The application establishes simultaneous WebSocket connections with Twilio Media Streams and the OpenAI Realtime API. It captures audio from the phone call via Twilio, streams it to OpenAI for processing, and then streams OpenAI's synthesized speech back to the caller. This approach facilitates low-latency, interactive voice communication by directly piping audio data between the two services.

Quick Start & Requirements

• Install dependencies: npm install
• Run the application: node index.js
• Prerequisites: Node.js 18+, Twilio account and number, OpenAI account and API Key, ngrok for local tunneling.
• Setup involves configuring Twilio to point to your ngrok URL and updating the .env file with your OpenAI API key.

Highlighted Details

• Real-time, two-way voice conversation with an AI.
• Supports AI preemption (interrupt handling) via input_audio_buffer.speech_started and conversation.item.truncate.
• Option to have the AI speak first by uncommenting sendInitialConversationItem().

Maintenance & Community

No specific information on contributors, sponsorships, or community channels is provided in the README.

Licensing & Compatibility

The README does not specify a license. Compatibility for commercial use or closed-source linking is not mentioned.

Limitations & Caveats

Outbound calling functionality is explicitly stated as beyond the scope of this application, though a separate demo is linked. The use of ngrok for local development requires an internet-accessible endpoint for Twilio integration.