This repository provides experimental labs and patterns for integrating Azure API Management (APIM) with various Azure AI services, primarily Azure OpenAI. It targets developers and architects seeking to build robust, secure, and cost-effective AI gateways, offering practical examples for accelerating AI use case experimentation and production deployment.

How It Works

The project leverages Azure API Management policies and Bicep/Terraform for infrastructure deployment to manage, secure, and optimize access to AI models. It explores patterns like load balancing, token rate limiting, semantic caching, prompt shielding, and fine-grained access control, demonstrating how APIM can act as a central control plane for AI services. The labs are structured around Jupyter notebooks with Python scripts, providing step-by-step guidance.

Quick Start & Requirements

• Install: Clone the repository. For local setup, ensure Python 3.12+, VS Code with Jupyter extension, and Azure CLI are installed and configured. Alternatively, use GitHub Codespaces.
• Prerequisites: An Azure Subscription with appropriate roles (Contributor, RBAC Administrator, or Owner).
• Getting Started: Navigate to a lab's notebook and follow the provided steps. Recommended starting point: token rate limiting lab.
• Links: AI Gateway Workshop

Highlighted Details

• Extensive coverage of Azure OpenAI features: real-time audio/text, function calling, GPT-4o inferencing, model routing, and vector searching.
• Integrations with AI Foundry, Model Context Protocol (MCP), and OpenAI Agents SDK.
• Labs focus on operational aspects: FinOps for cost management, load balancing, tracing, streaming, and content/prompt filtering.
• Demonstrates deployment via Bicep and Terraform.

Maintenance & Community

The project is part of Azure Samples, indicating official Microsoft backing. Feedback and contributions are encouraged via GitHub issues and pull requests.

Licensing & Compatibility

The software is provided for demonstration purposes only and is not intended for production reliance. The specific license is not explicitly stated in the README, but the disclaimer suggests it's for experimental use.

Limitations & Caveats

The disclaimer explicitly states the software is for demonstration purposes only and should not be relied upon for any purpose, with no warranties provided regarding accuracy or reliability.