This repository serves as a curated list of resources, primarily research papers and associated code/projects, related to the application of foundation models in robotics. It targets roboticists, researchers, and engineers interested in leveraging large language models (LLMs), vision-language models (VLMs), and other large-scale pre-trained models for robotic tasks such as manipulation, navigation, planning, and embodied AI. The primary benefit is providing a structured overview and quick reference to relevant literature and implementations.

How It Works

The repository categorizes foundation models and their applications based on a survey paper, "Foundation Models in Robotics: Applications, Challenges, and the Future." It divides resources into two main sections: "Foundation Models Applied to Robotics" and "Foundation Models Relevant to Robotics." The former details specific applications like neural scaling laws, language-conditioned policy learning, goal-conditioned value learning, task planning, transformers, and in-context learning. The latter covers foundational perception tasks like open-vocabulary object detection, semantic segmentation, 3D scene and object representations, affordance information, and predictive models.

Quick Start & Requirements

This repository is a collection of links to papers and projects, not a runnable software package. No installation or specific requirements are needed to browse the content.

Highlighted Details

• Comprehensive coverage of recent advancements in applying LLMs and VLMs to robotics.
• Links to papers, projects, and code for various robotics tasks, including manipulation, navigation, and planning.
• Categorization follows a structured survey paper, aiding in targeted research.
• Includes resources relevant to both direct robotics applications and foundational AI research applicable to robotics.

Maintenance & Community

The repository is associated with a survey paper and welcomes contributions via pull requests. Specific contributors or community channels are not detailed in the README.

Licensing & Compatibility

The repository itself does not have a specific license mentioned, but it links to external projects which will have their own licenses. Users should verify the licenses of individual linked projects for compatibility.

Limitations & Caveats

This is a curated list of research resources, not a unified framework or library. The content is primarily paper links, and the availability and functionality of associated code or projects are not guaranteed by this repository. The rapid pace of research means the list may require frequent updates to remain comprehensive.