DrEureka automates the sim-to-real transfer process for robotics by leveraging Large Language Models (LLMs) to generate optimal reward functions and domain randomization parameters. This approach targets robotics researchers and engineers seeking to accelerate the development and deployment of policies in real-world scenarios, reducing manual effort and improving transfer performance.

How It Works

DrEureka employs a three-stage LLM-driven pipeline. First, it uses an LLM to automatically generate a reward function by analyzing environment code and a reward signature. Second, it employs an LLM to determine domain randomization (DR) bounds by evaluating policy performance against simple success criteria. Finally, it uses the generated reward function and DR bounds to guide LLM-based DR generation, creating configurations that enhance sim-to-real transfer. This iterative, LLM-guided approach aims to discover effective sim-to-real configurations without manual tuning.

Quick Start & Requirements

• Installation: Requires Python 3.8 and PyTorch with CUDA 11.3. Installation involves creating a Conda environment, installing PyTorch, IsaacGym, and the DrEureka packages.
• Prerequisites: NVIDIA Isaac Gym is a mandatory simulator. Tested on Ubuntu 20.04.
• Setup: Detailed installation steps are provided for setting up the environment, reward generation (Eureka), RAPP (domain randomization bounds), and DR generation.
• Links: Website, arXiv

Highlighted Details

• Automates reward function and domain randomization generation using LLMs.
• Demonstrates competitive performance against human-designed sim-to-real configurations.
• Solves novel tasks like quadruped balancing and yoga ball walking without iterative manual design.
• Deployment infrastructure is based on "Walk These Ways" for Unitree robots.

Maintenance & Community

The project is associated with authors from the University of Pennsylvania and NVIDIA. No specific community channels (Discord/Slack) or roadmap are mentioned in the README.

Licensing & Compatibility

• License: MIT License.
• Compatibility: Permissive license allows for commercial use and integration with closed-source projects.

Limitations & Caveats

The deployment instructions are specific to Unitree robots and require manual steps to transfer code and run on the robot. The system relies heavily on the quality of LLM outputs, which can be variable.