Higress is an AI-native API gateway designed for cloud-native environments, offering robust support for AI applications, LLM providers, and microservices. It targets developers and enterprises seeking a scalable, extensible, and secure gateway solution, providing unified management for AI and traditional API traffic with enhanced observability and control.

How It Works

Higress is built upon Istio and Envoy, leveraging their battle-tested capabilities for traffic management and proxying. Its core innovation lies in its Wasm plugin extensibility, allowing custom logic written in Go, Rust, or JavaScript to be sandboxed and executed within the gateway. This approach enables features like AI-specific traffic shaping, model load balancing, token rate limiting, and caching, while also facilitating the hosting of MCP (Model Context Protocol) Servers for AI agents to interact with various tools and services.

Quick Start & Requirements

• Install/Run: docker run -d --rm --name higress-ai -v ${PWD}:/data -p 8001:8001 -p 8080:8080 -p 8443:8443 higress-registry.cn-hangzhou.cr.aliyuncs.com/higress/all-in-one:latest
• Prerequisites: Docker.
• Resources: Minimal for local testing.
• Docs: Official Site, MCP Server QuickStart, Wasm Plugin Hub

Highlighted Details

• Production-grade, validated within Alibaba Cloud, handling hundreds of thousands of requests per second.
• Eliminates Nginx reload traffic jitter; configuration changes take effect in milliseconds.
• Supports true end-to-end streaming processing for requests and responses, reducing memory overhead.
• Extensible via Wasm plugins with independent versioning and lossless hot updates.

Maintenance & Community

• Developed by Alibaba.
• Community support via Slack.
• Related repositories: Higress Console, Higress Standalone.

Licensing & Compatibility

• License: Apache License 2.0.
• Compatible with commercial use and closed-source linking.

Limitations & Caveats

Gateway API support is noted as "coming soon," indicating potential migration effort from Ingress API. While Wasm plugins offer extensibility, managing multiple plugin versions and ensuring compatibility across different languages might require careful orchestration.