This React component library provides voice controls for browser-based UIs, built on OpenAI Realtime. It targets developers who need to integrate voice interaction into their applications while maintaining strict control over available actions (tools). The primary benefit is a reusable React controller and optional UI widget that simplifies voice command integration without sacrificing app-defined functionality.

How It Works

The core approach leverages OpenAI Realtime transports, offering a createVoiceControlController for managing session, tool execution, and transcript assembly. It emphasizes an "app-owned tools" model, where developers define specific actions via Zod schemas. This design is advantageous for UIs requiring precise, predictable voice command execution rather than general-purpose automation. Optional GhostCursorOverlay provides visual feedback.

Quick Start & Requirements

• Install: Local checkout: npm install ../path/to/realtime-voice-component and import realtime-voice-component/styles.css.
• Prerequisites: OpenAI API Key, a backend /session endpoint proxying to OpenAI's Realtime API, Zod, React, and browser media/WebRTC support.
• Demo: Run locally via cp demo/.env.example demo/.env.local, set OPENAI_API_KEY, npm install, npm run demo.
• Docs: Links provided for overview, getting started, integration, architecture, controller, widget, authentication, and API reference.

Highlighted Details

• defineVoiceTool(): Maps Zod-backed app actions to Realtime functions.
• createVoiceControlController(): Manages voice session lifecycle.
• useVoiceControl(): React hook for controller binding.
• VoiceControlWidget: A packaged launcher UI.
• Recommended Flow: Proxy SDP/session config via a backend, register narrow tools, and sync app state back to the session post-action.
• Turn Detection: Defaults to server_vad with configurable interrupt_response.

Maintenance & Community

This repository is an "open-source reference implementation" intended for education, demos, and local adoption. It is not published to npm and does not promise long-term product support or production readiness. No specific community channels or contributor details are provided in the README.

Licensing & Compatibility

Licensed under Apache-2.0, which is generally permissive for commercial use and integration into closed-source applications.

Limitations & Caveats

The component is explicitly not a "production-ready UI kit" and lacks long-term product support guarantees. It is "intentionally opinionated" and not a general orchestration framework. Integration into complex React apps requires careful management of controller lifecycles, especially concerning React's Strict Mode remounts, which can lead to silent connection failures.