Arch Gateway is an AI-native proxy server designed to streamline the development of agentic applications. It addresses common challenges like prompt routing, LLM integration, input clarification, and observability, allowing developers to focus on core agent logic rather than low-level infrastructure. The target audience includes developers building AI agents and LLM-powered applications who need a flexible, framework-agnostic solution.

How It Works

Arch Gateway is built on Envoy Proxy, leveraging its robust networking and scalability features. It acts as an intermediary, intercepting user prompts and intelligently routing them to the appropriate LLM or specialized tool (e.g., API endpoints). Key functionalities include prompt clarification using LLMs, tool/API call generation, centralized LLM access with smart retries, and built-in guardrails for safety. This approach centralizes complex agentic workflows outside the main application code, promoting modularity and faster development.

Quick Start & Requirements

• Install: pip install archgw==0.2.8
• Prerequisites: Docker (v24), Docker Compose (v2.29), Python (v3.12).
• Setup: Requires creating a configuration file (arch_config.yaml) and running archgw up <config_file>.
• Documentation: https://docs.archgw.com

Highlighted Details

• Built on Envoy Proxy for proven scalability and extensibility.
• Supports LLM routing, tool use, prompt guardrails, and unified LLM access.
• Provides W3C-compatible request tracing and LLM metrics for observability.
• Offers sample applications demonstrating currency exchange and network operator agents.

Maintenance & Community

• Active development with contributions from former Envoy Proxy contributors.
• Community support and discussion available via Discord.
• Roadmap and contribution guidelines are provided.

Licensing & Compatibility

• The README does not explicitly state the license. Compatibility for commercial use or closed-source linking is not specified.

Limitations & Caveats

The project is in active development, and the hosted function-calling LLM will soon require developer keys, with an option to run locally. The license is not clearly stated in the README, which may impact commercial adoption.