Phoenix is an open-source platform for AI observability, focusing on LLM application experimentation, evaluation, and troubleshooting. It targets AI engineers and researchers, offering tools to trace LLM runtime, benchmark performance with LLM-based evals, manage datasets, track experiments, and optimize prompts.

How It Works

Phoenix leverages OpenTelemetry for vendor and language-agnostic tracing of LLM applications. It provides specialized Python and JavaScript clients for interacting with its backend, enabling structured data collection and analysis. The platform supports integrations with popular LLM frameworks like LangChain and LlamaIndex, and various LLM providers, facilitating a unified view of application performance and behavior.

Quick Start & Requirements

• Install via pip: pip install arize-phoenix
• Docker images are available on Docker Hub.
• Integrations require specific openinference-instrumentation-* packages.
• Official documentation and community Slack are available.

Highlighted Details

• Vendor and language agnostic, built on OpenTelemetry.
• Supports major LLM frameworks (LangChain, LlamaIndex, Haystack, DSPy) and providers (OpenAI, Bedrock, MistralAI).
• Includes tools for tracing, LLM-based evaluation, dataset versioning, and prompt management.
• Offers a Playground for prompt optimization and model comparison.

Maintenance & Community

• Active community with Slack channel and GitHub for support and bug reporting.
• Roadmap available for future development.

Licensing & Compatibility

• Licensed under the Elastic License 2.0 (ELv2).
• Portions of the code are patent protected by U.S. Patents.

Limitations & Caveats

• The Elastic License 2.0 may have restrictions on offering managed services.
• Some features might be in active development, as indicated by the project's focus on experimentation and evaluation.