Summary

RealMirror is an open-source embodied AI Vision-Language-Action (VLA) platform addressing critical challenges in humanoid robotics, including high data acquisition costs, lack of standardized benchmarks, and the simulation-to-real (Sim2Real) gap. It provides researchers and engineers with an end-to-end system for data collection, model training, and inference, enabling accelerated VLA development and facilitating zero-shot Sim2Real transfer without requiring a physical robot.

How It Works

The platform integrates a comprehensive VLA benchmark for humanoid robots with a novel approach to bridging simulation and reality. It leverages generative models and 3D Gaussian Splatting to reconstruct realistic environments and robot models, enabling models trained purely in simulation to perform tasks seamlessly on real robots. This approach bypasses the need for extensive real-world data collection and fine-tuning, significantly reducing development overhead and accelerating research progress.

Quick Start & Requirements

• Installation: Clone the repository, initialize and update git submodules, apply patches, and install LeRobot and RealMirror into the NVIDIA Isaac Sim Python environment using pip install -e ..
• Prerequisites: Ubuntu 22.04, NVIDIA Isaac Sim 5.0.0, NVIDIA GPU (RTX 5090/5880 recommended), NVIDIA Driver 575.64.03, CUDA 12.9.
• Data: Requires downloading ~13 GB for benchmark data (assets, models) and ~70 GB for training data.
• Links: GitHub repository: https://github.com/terminators2025/RealMirror.git. Paper: arXiv preprint arXiv:2509.14687.

Highlighted Details

• Features a dedicated VLA benchmark for humanoid robots across five diverse tasks.
• Achieves zero-shot Sim2Real transfer, demonstrating models trained in simulation operate on real robots.
• Includes an integrated teleoperation framework for data acquisition, recording, replay, and conversion to training formats.
• Supports the Agibot A2 robot model and utilizes advanced rendering for realistic simulation.

Maintenance & Community

The project indicates active development with recent releases of evaluation frameworks and teleoperation tools. Specific details on core maintainers, community channels (e.g., Discord, Slack), or sponsorship are not provided in the README.

Licensing & Compatibility

The repository's license is not explicitly stated in the README. This omission requires clarification for commercial use or integration into proprietary systems. The platform is tightly coupled with NVIDIA Isaac Sim and requires specific NVIDIA hardware and software versions.

Limitations & Caveats

Setup demands high-end NVIDIA hardware (RTX 5090/5880) and a specific CUDA version (12.9). Significant data downloads (~83 GB total) are necessary. Future features like MirrorLimb support and gesture teleoperation are marked as "coming soon," indicating ongoing development and potential for breaking changes. The absence of a clear license is a notable adoption blocker.