This repository provides a Node.js and JavaScript reference client for OpenAI's Realtime API, targeting developers building conversational applications. It simplifies prototyping by offering a high-level abstraction for managing real-time audio, text, and function calls, enabling rapid development of interactive AI experiences.

How It Works

The client abstracts the underlying WebSocket connection into three primitives: RealtimeClient for high-level control, RealtimeAPI for direct WebSocket interaction, and RealtimeConversation for managing client-side conversation state and event validation. The RealtimeClient simplifies common tasks like sending messages, streaming audio, and integrating tools via callbacks, while also providing curated events for application logic.

Quick Start & Requirements

• Install via npm: npm i openai/openai-realtime-api-beta --save
• Requires Node.js environment.
• OpenAI API key is necessary.
• Browser usage is possible but discouraged due to API key exposure; use a relay server or dangerouslyAllowAPIKeyInBrowser: true.
• Official Realtime Console: https://platform.openai.com/docs/realtime-api/getting-started

Highlighted Details

• Supports server-side (Node.js) and browser (React, Vue) integration with JavaScript and TypeScript.
• Enables streaming audio input via appendInputAudio() and tool integration with automatic callback execution.
• Provides fine-grained control through direct WebSocket events and methods like cancelResponse.
• Offers a Realtime Console for API inspection and voice visualization.

Maintenance & Community

• Developed by OpenAI Developers.
• Key contributors listed include Jordan Sitkin, Mark Hudnall, Peter Bakkum, Atty Eleti, Jason Clark, and Keith Horwood.

Licensing & Compatibility

• License details are not explicitly stated in the README, but it's an official OpenAI reference client. Compatibility for commercial use should be verified against OpenAI's API terms.

Limitations & Caveats

This library is in beta and not a final implementation. Direct browser usage of API keys is not recommended due to security risks.