A task-based evaluation benchmark for robot manipulation policies, RoboLab is built on NVIDIA Isaac Lab to provide reproducible, large-scale benchmarking of generalist robot policies in simulation. It addresses the need for standardized evaluation by offering over 100 diverse manipulation tasks with automated success detection and a flexible server-client architecture. This benchmark enables researchers and engineers to develop and compare advanced robotic manipulation systems more effectively.

How It Works

The core of RoboLab is RoboLab-120, featuring over 120 novel manipulation tasks with automated success/failure detection via composable predicates and language instructions. It employs a server-client policy architecture, allowing external models to interface with the simulation. The system supports multi-environment parallel evaluation for efficiency and includes AI-enabled workflows for generating new scenes and tasks using natural language. Tasks are designed to be robot-agnostic, compatible with any Isaac Lab-supported robot embodiment.

Quick Start & Requirements

Installation requires ffmpeg and uv (Python 3.11). Isaac Sim 5.0 and Isaac Lab 2.2.0 are installed automatically via uv sync.

sudo apt install ffmpeg
git clone <repo_url>
cd robolab
uv venv --python 3.11
source .venv/bin/activate
uv sync

Verification: uv run pytest tests/. Prerequisites: Isaac Sim 5.0, Isaac Lab 2.2.0, Python 3.11, Linux (Ubuntu 22.04+), ffmpeg. Hardware: NVIDIA RTX GPU required, 48GB+ VRAM recommended. Resource Footprint: ~8 GB disk space. Estimated 30 GPU hours per 100 tasks. Links: 🌐 Website, 📄 Paper, 🏆 Leaderboard.

Highlighted Details

• RoboLab-120: Over 120 novel manipulation tasks (pick-and-place, stacking, tool use) with automated success/failure detection.
• Robot Agnostic: Tasks compatible with any Isaac Lab-supported robot embodiment.
• AI-Powered Generation: Natural language prompts for scene and task creation via Claude Code skills.
• Interactive Results Dashboard: Web dashboard for visualizing scenes, replaying episodes, and analyzing experiments.

Maintenance & Community

No specific contributors, sponsorships, or community channels (Discord/Slack) are detailed in the README. Contribution guidelines are in CONTRIBUTING.md.

Licensing & Compatibility

Released under the Apache License 2.0, permissive for commercial use. Third-party dependency licenses are in THIRD_PARTY_NOTICES.md.

Limitations & Caveats

High hardware requirements (NVIDIA RTX GPU, 48GB+ VRAM) present a significant adoption barrier. As a recent development (associated with a 2026 paper), it is likely an evolving system.