Agenta is an open-source LLMOps platform designed for engineering and product teams to accelerate the development of reliable LLM applications. It provides integrated prompt management, evaluation, and observability features, enabling collaborative prompt engineering, systematic testing, and production monitoring.

How It Works

Agenta offers a unified platform for the LLM development lifecycle. Prompt management includes an interactive playground, support for over 50 LLM models (or custom providers), version control for prompts, and complex configuration schema collaboration. Evaluation features allow for flexible test sets, pre-built and custom evaluators (including LLM-as-judge), and both UI and programmatic access for running tests and collecting human feedback. Observability is achieved through cost and performance tracking, detailed tracing, and OpenTelemetry native compatibility for debugging complex workflows.

Quick Start & Requirements

To self-host Agenta, clone the repository and run the following Docker command:

docker compose -f hosting/docker-compose/oss/docker-compose.gh.yml --env-file hosting/docker-compose/oss/.env.oss.gh --profile with-web up -d

Access Agenta at http://localhost. Refer to the self-hosting documentation for detailed deployment instructions.

Highlighted Details

• Supports over 50 LLM models and custom providers.
• Features an interactive playground for side-by-side prompt comparison.
• Integrates pre-built and custom evaluators, including LLM-as-judge.
• Provides OpenTelemetry native tracing for observability.
• Offers a cloud-hosted version with a free tier.

Maintenance & Community

The project has 49 contributors and maintains an active Slack Community for support and discussion. Bug reporting and feature suggestions are managed via GitHub issues and discussions.

Licensing & Compatibility

The project is licensed under the Apache License 2.0, which permits commercial use and linking with closed-source applications.

Limitations & Caveats

Agenta collects anonymized usage statistics by default, which can be disabled by setting TELEMETRY_ENABLED to false in the .env file.