This repository provides a sample implementation of the VoiceRAG pattern, enabling interactive voice-driven generative AI experiences. It targets developers building voice-enabled applications, leveraging Azure AI Search for retrieval and Azure OpenAI's GPT-4o Realtime API for advanced audio processing and response generation.

How It Works

The application utilizes a real-time client (RTClient) in the frontend to capture voice input via the browser's microphone. This audio is transmitted to a Python backend featuring an RTMiddleTier object. This tier interfaces with the Azure OpenAI Realtime API for speech processing and integrates a tool for querying Azure AI Search. The RAG pattern is central, retrieving relevant documents from a knowledge base to inform GPT-4o's responses, which are then synthesized into audio output. Citations of used search results are also displayed.

Quick Start & Requirements

• Primary install/run: azd up (Azure Developer CLI) for deployment to Azure Container Apps. Local development requires scripts/start.ps1 (Windows) or ./scripts/start.sh (Linux/Mac).
• Prerequisites: Azure Developer CLI, Node.js, Python >=3.11, Git, Powershell (Windows). Azure AI Search and Azure OpenAI services are required.
• Setup: GitHub Codespaces or VS Code Dev Containers offer the quickest setup. Local setup involves cloning the repo and running azd auth login. Deployment via azd up provisions Azure resources and incurs costs.
• Links: Blog post, Demo video, Azure OpenAI Realtime Documentation

Highlighted Details

• Implements the VoiceRAG pattern for interactive voice generative AI.
• Leverages GPT-4o Realtime API for low-latency audio input and output.
• Integrates Azure AI Search for Retrieval Augmented Generation (RAG).
• Supports deployment via Azure Container Apps with infrastructure-as-code.

Maintenance & Community

This is an Azure Samples repository, indicating official Microsoft backing. Community interaction channels are not explicitly listed in the README.

Licensing & Compatibility

The repository's license is not specified in the README. As an Azure sample, it is likely intended for demonstration and development purposes, with potential usage governed by Azure service terms. Commercial use of the underlying Azure services is subject to their respective pricing and terms.

Limitations & Caveats

Deployment via azd up incurs costs for provisioned Azure resources, primarily Azure AI Search. Users are advised to run azd down or manually delete resources to avoid ongoing charges. The sample data disclaimer notes that generated content is for demonstration only and may not be accurate or reliable.