AGI SDK is a Python toolkit for building, evaluating, and benchmarking AI browser agents against real-world web applications. It targets AI researchers and developers seeking to test agent capabilities in realistic, complex environments, offering a standardized benchmark and a platform for comparing agent performance.

How It Works

The SDK provides high-fidelity, deterministic web application clones (e.g., Amazon, DoorDash) built with modern web stacks. It uses a harness that orchestrates agent interactions with these environments, providing structured observations (DOM, accessibility tree, screenshots) and accepting actions as function calls. This approach allows for reproducible evaluation and direct comparison of agents on standardized tasks.

Quick Start & Requirements

• Install: pip install agisdk
• Dependencies: playwright install --force
• Environment Variables: OPENAI_API_KEY (or other supported LLM providers)
• Apple Silicon: brew install --cask playwright
• Quick-start example: https://github.com/agi-inc/agisdk#60-second-quick-start

Highlighted Details

• Supports OpenAI, Anthropic, and OpenRouter models.
• Includes full-stack web replicas of popular applications like Amazon, DoorDash, Airbnb, Gmail, and LinkedIn.
• Features a leaderboard (REAL Bench) for agent performance comparison.
• Offers a robust agent API with observations, actions, memory, and error handling.
• Allows customization of the harness and integration of custom agents.

Maintenance & Community

• Community channels are listed as "coming soon" on Discord.
• AGI Inc. can be followed on LinkedIn.

Licensing & Compatibility

• The repository does not explicitly state a license in the provided README.

Limitations & Caveats

• Discord community channels are not yet available.
• The README does not specify licensing details, which may impact commercial use or closed-source integration.