<2-3 sentences summarising what the project addresses and solves, the target audience, and the benefit.> Waza is a Go CLI framework designed for AI agent developers to create, test, measure, and improve the quality and effectiveness of their agent skills. It streamlines the process of scaffolding evaluation suites, running benchmarks, and comparing performance across various AI models, offering a structured approach to enhancing AI agent capabilities.

How It Works

The tool facilitates a robust development workflow through commands for project initialization, skill scaffolding, and defining evaluation suites with tasks and fixtures. It leverages YAML for evaluation specifications and can generate test cases by recording interactions with models like Copilot. Waza enables direct comparison of results across different AI models and integrates with CI/CD pipelines for automated testing and reporting.

Quick Start & Requirements

• Primary install / run command: Binary install via curl -fsSL https://raw.githubusercontent.com/microsoft/waza/main/install.sh | bash (recommended). Source installation requires cloning the repo and using go build. Also available as an Azure Developer CLI (azd) extension.
• Non-default prerequisites and dependencies: Go 1.26+ for source builds. Azure CLI (azd) for extension usage. Git LFS is required for source builds due to LFS artifacts.
• Links: Custom Agents Guide, Dashboard Guide.

Highlighted Details

• Comprehensive CLI for AI agent skill evaluation, benchmarking, and quality improvement.
• Automated generation of evaluation suites, tasks, and fixtures from skill definitions.
• Supports multi-model comparison and detailed performance analysis with LLM-as-Judge capabilities.
• Integrates with Azure Blob Storage for collaborative result tracking and CI/CD pipelines for automated reporting.

Maintenance & Community

No specific details regarding maintainers, community channels (e.g., Discord, Slack), or roadmap were found in the provided README content.

Licensing & Compatibility

The provided README content does not specify a license. This absence is a critical factor for assessing commercial use or integration compatibility.

Limitations & Caveats

Source installation requires Go 1.26+, a relatively recent Go version. The README mentions LFS artifacts for embedded binaries during source builds, which may add complexity. The lack of explicit licensing information is a significant adoption blocker. Some features, like template packs for skill creation, are noted as "coming soon."