BrowserGym is an open-source framework designed to accelerate research in web task automation for AI agents. It provides a standardized environment, integrating multiple popular web agent benchmarks, and allows for easy creation of new ones, targeting AI researchers and developers in the field of web automation.

How It Works

BrowserGym leverages the Gymnasium API to create standardized environments for web interaction. It supports various benchmarks, including MiniWoB, WebArena, VisualWebArena, WorkArena, AssistantBench, and WebLINX, by wrapping them as Gymnasium tasks. This approach allows agents to interact with web pages through a consistent interface, facilitating agent development and evaluation across diverse web tasks.

Quick Start & Requirements

• Install via pip: pip install browsergym (recommended, includes all components).
• Install Playwright browsers: playwright install chromium.
• Additional setup required for specific benchmarks (MiniWoB++, WebArena, etc., see benchmark-specific READMEs).
• Demo agent requires OpenAI API key and potentially Conda environment setup.
• Official Docs: https://github.com/ServiceNow/BrowserGym

Highlighted Details

• Integrates 6+ major web agent benchmarks (MiniWoB, WebArena, VisualWebArena, WorkArena, AssistantBench, WebLINX).
• Provides an "openended" task for interactive, chat-based web automation.
• Includes a demo agent using OpenAI models for quick experimentation.
• Ecosystem includes AgentLab for agent evaluation and other related benchmarks.

Maintenance & Community

• Developed by ServiceNow.
• Active development indicated by CI/CD badges for code formatting and tests.
• Links to related projects like AgentLab and specific benchmarks are provided.

Licensing & Compatibility

• License: Apache 2.0 (as indicated by PyPI badge).
• Compatible with commercial use and closed-source linking.

Limitations & Caveats

• The project is explicitly stated as "not meant to be a consumer product" and should be used with caution.
• Specific benchmark setups require additional steps beyond the core installation.