MobileGym provides a browser-hosted, verifiable, and highly parallel simulation platform for mobile GUI agent research. It addresses limitations in current environments by enabling trustworthy evaluation and scalable Reinforcement Learning training for complex mobile apps, with validated sim-to-real transfer.

How It Works

This platform features a three-layer architecture (Benchmark, Apps, OS) with a browser-hosted Android-like environment. Its core innovation is fully programmable, structured JSON state snapshots, allowing precise control and overcoming state accessibility issues. MobileGym uses deterministic, programmatic judges for sub-millisecond verdicts, eliminating VLM reliance. It scales to 256 parallel instances on a single server with low resource overhead and demonstrates significant sim-to-real transfer fidelity. Declarative navigation via finite-state machines enhances control.

Quick Start & Requirements

Requires Node.js (≥22) and Python (≥3.11); Conda recommended. Installation involves cloning, npm install, pip install -r bench_env/requirements.txt, playwright install chromium, and downloading a ~1.4 GB dataset. Boot the simulator via npm run dev (explore) or npm run build && npm run preview (single-agent). Heavy benchmarking uses an Nginx gateway script (./scripts/server/start_nginx_gateway.sh). Links to docs and a live demo are available at mobilegym.dev.

Highlighted Details

• Programmable State: Entire environment state as a single JSON blob for identical initial conditions.
• Deterministic Judges: Sub-millisecond, programmatic verdicts replace VLM reliance.
• Lightweight & Scalable: ≈400 MB RAM/instance; 256 parallel instances use <10% CPU.
• Sim-to-Real Validation: 95.1% retention of simulation gains on real hardware.
• Modular Design: Easily extendable with new apps, tasks, agents, and judges.
• Declarative Navigation: Apps defined as finite-state machines.

Maintenance & Community

The project maintains a leaderboard and encourages contributions via Pull Requests. Community interaction is facilitated through GitHub Issues and Discussions.

Licensing & Compatibility

Code is Apache License 2.0 (permissive, commercial use allowed). Data/content is CC BY-NC 4.0 (non-commercial, academic use only).

Limitations & Caveats

The CC BY-NC 4.0 license restricts commercial use of data/content. Large-scale parallel execution on Linux requires system configuration adjustments. Simulated apps are research surrogates, not interacting with real services or funds.