This repository provides a reference implementation for an AI calling assistant, combining OpenAI's Realtime API with Twilio's telephony services. It targets developers looking to build interactive voice applications that leverage real-time AI for conversational agents, offering a quick start guide for local setup and deployment.

How It Works

The system comprises a NextJS web application for user interaction and call configuration, and an Express-based websocket server. The websocket server acts as the central hub, receiving incoming calls via Twilio, establishing a bidirectional stream, and forwarding messages between the phone call and the OpenAI Realtime API. Twilio is configured via TwiML to stream audio to the backend, which then relays it to OpenAI and streams the AI's responses back to the caller.

Quick Start & Requirements

• Install/Run:
  • Webapp: npm run dev (in webapp/ directory)
  • Websocket server: npm run dev (in websocket-server/ directory)
  • Tunneling: ngrok http 8081
• Prerequisites: Node.js, npm, Twilio account, OpenAI API key, ngrok.
• Setup: Requires setting environment variables in webapp/.env and websocket-server/.env.
• Links: Full Setup

Highlighted Details

• Demonstrates real-time, bidirectional audio streaming between a phone call and an AI model.
• Utilizes ngrok for exposing local development servers to Twilio webhooks.
• Includes a mock function calling mechanism for simulating AI-driven actions.
• Provides a NextJS frontend for managing call configurations and viewing transcripts.

Maintenance & Community

This is an official OpenAI demo repository. Further community or maintenance details are not specified in the README.

Licensing & Compatibility

The repository's license is not explicitly stated in the provided README. Compatibility for commercial use or closed-source linking is not detailed.

Limitations & Caveats

The project is explicitly stated as not polished, with security practices that "leave some to be desired." It is intended for reference only, and users are advised to audit their applications thoroughly before deployment.