Enterprise AI bastion host for secure AI API and MCP access, with unified proxying, RBAC, audit logs, rate limiting, and cost tracking across multiple LLM providers. It addresses the growing governance challenges of scattered API keys, lack of visibility, and compliance gaps in organizations adopting AI agents, providing a single control plane for secure, auditable, and governable AI interactions.

How It Works

ThinkWatch acts as a central gateway, analogous to an SSH bastion host, for all AI API calls and Machine Control Protocol (MCP) tool invocations. It employs a dual-port architecture: a public-facing Gateway (port 3000) handles AI/MCP traffic, while an internal-only Console (port 3001) provides management UI and API. The system unifies access to diverse providers (OpenAI, Anthropic, Gemini, Azure, Bedrock) through multi-format proxying and automatic format conversion, enforcing security policies like RBAC, virtual API keys, rate limits, and cost tracking.

Quick Start & Requirements

• Primary install/run: Development uses make commands and pnpm. Production deployment via Docker Compose or Kubernetes Helm.
• Prerequisites: PostgreSQL, Redis, ClickHouse databases; an OIDC-compliant provider (e.g., Zitadel, Okta) for SSO.
• Development Dependencies: Rust toolchain, Node.js/pnpm.
• Links: Full documentation: thinkwat.ch/docs. Local setup wizard: http://localhost:5173/setup.

Highlighted Details

• AI API Gateway: Supports OpenAI Chat Completions, Anthropic Messages, and other formats with multi-provider routing and automatic conversion. Features virtual API keys with lifecycle management.
• Composable Rate Limits & Budgets: Granular, multi-window sliding rate limits and natural-period token budgets configurable per user, API key, provider, or MCP server. Supports per-model token weighting.
• MCP Gateway: Centralized tool proxy with namespace isolation and tool-level RBAC for secure invocation of upstream tools.
• Security & Compliance: Dual-port architecture, 5-tier RBAC, SSO/OIDC integration, AES-256-GCM encryption for secrets, and distroless containers for minimal attack surface.
• Observability: Integrated Prometheus metrics, enhanced health checks, and ClickHouse-powered, SQL-queryable audit logs with multi-channel forwarding options.

Licensing & Compatibility

ThinkWatch is source-available under the Business Source License 1.1 (BSL 1.1). Non-production use is free. Production use is free up to 10,000,000 Billable Tokens and 10,000 MCP Tool Calls per UTC calendar month. Above these thresholds, a commercial license is required, priced by usage tiers. The license will transition to GPL-2.0-or-later.

Limitations & Caveats

On streaming (SSE) responses, PII redaction is applied before forwarding to the upstream provider, but placeholders are not restored client-side. This means clients may see PII placeholders verbatim if the model echoes user PII, though the upstream provider never receives the original PII. Non-streaming responses restore PII. Streaming token accounting also depends on upstream providers surfacing usage data.