Magnitude is an open-source, AI-native testing framework for web applications, designed for developers and QA engineers. It leverages visual AI agents to create and execute end-to-end tests, adapting to UI changes and simplifying test case creation with natural language.

How It Works

Magnitude employs a two-agent system: a "planner" (a strong, multi-modal LLM like Gemini 2.5 Pro) for test case reasoning and planning, and an "executor" (a fast vision LLM like Moondream) for reliable UI interaction and execution. This separation allows for efficient test design and robust, adaptive test runs, with the planner intervening when issues arise during execution.

Quick Start & Requirements

• Install the test runner: npm install --save-dev magnitude-test
• Initialize Magnitude: npx magnitude init
• LLM Configuration: Requires API keys for a planner LLM (e.g., Gemini, Anthropic, OpenAI) and the Moondream executor LLM. Environment variables like GOOGLE_API_KEY, ANTHROPIC_API_KEY, OPENAI_API_KEY, and MOONDREAM_API_KEY are used.
• Resource Footprint: Setup involves installing dependencies and configuring API keys. Running tests locally or in CI/CD pipelines is supported.
• Documentation: Magnitude Test Docs

Highlighted Details

• Test cases are built using natural language for steps, data, and assertions.
• The framework includes a native test runner that can report bugs with descriptions.
• Supports parallel test execution with the -w <workers> flag.
• Integrates with CI/CD pipelines, with specific instructions for GitHub Actions.

Maintenance & Community

• Community support is available via Discord.
• Contact for demos: founders@magnitude.run

Licensing & Compatibility

• The README does not explicitly state the license.

Limitations & Caveats

• Currently, only Moondream is supported as the executor model.
• LLM API key configuration is a prerequisite for operation.