RoboGen is a research project focused on enabling robots to autonomously propose, generate, and master new skills through generative simulation. It targets researchers and engineers in embodied AI and robotics who aim to create more adaptable and continuously learning robotic systems, reducing the need for manual task definition and data collection.

How It Works

RoboGen leverages a generative simulation engine (Genesis, with a PyBullet re-implementation available) to create novel robotic tasks and environments. It uses large language models (like GPT-4) for task generation based on descriptions and PartNet-Mobility for scene population. Skills are learned using reinforcement learning (SAC via Ray RLlib) or motion planning (OMPL) for manipulation tasks, and CEM for locomotion. This approach aims to create an "infinite data" pipeline for robot learning.

Quick Start & Requirements

• Install: Clone the repository and create a conda environment using environment.yaml.
• Prerequisites: Python 3.9+, OpenAI API key, PartNet-Mobility dataset, Objaverse embeddings (optional, can be generated). OMPL installation from source or pre-built wheels is required.
• Setup: Requires downloading datasets and potentially generating embeddings. OMPL installation can be complex.
• Links: Official Repo, OMPL Wheels.

Highlighted Details

• Fully automated task generation and skill acquisition pipeline.
• Supports both rigid manipulation and locomotion tasks.
• Utilizes LLMs for task conceptualization and PartNet-Mobility for environment grounding.
• Employs RL (SAC) and motion planning (OMPL) for skill learning.

Maintenance & Community

The project is associated with ICML 2024 and lists several authors from academic institutions. The core Genesis engine is under active development and planned for future release.

Licensing & Compatibility

The repository itself is not explicitly licensed in the README. The citation format suggests an academic publication. Compatibility for commercial use is not specified.

Limitations & Caveats

The provided PyBullet implementation currently only covers rigid manipulation and locomotion; soft-body manipulation and more complex tasks are planned for future release with the full Genesis engine. OMPL installation can be challenging, and the script includes a potentially system-altering apt-get upgrade command. RL training can take 1-2 hours per skill.