This repository provides a comprehensive survey of Multimodal Chain-of-Thought (MCoT) reasoning, a technique that enhances step-by-step reasoning in multimodal large language models (MLLMs). It targets researchers and practitioners in AI, particularly those working with MLLMs, robotics, healthcare, and autonomous driving, by offering a structured overview of methodologies, datasets, applications, and challenges in the field.

How It Works

The survey systematically categorizes MCoT research into key areas: datasets and benchmarks (for training and evaluation), methodologies (rationale construction, structural reasoning, information enhancement, objective granularity, and test-time scaling), and applications across various domains like Embodied AI, Autonomous Driving, and Healthcare. It also explores the use of Reinforcement Learning (RL) to improve MCoT capabilities, enabling models to learn complex reasoning without explicit supervision.

Quick Start & Requirements

This repository is a survey and does not involve direct code execution or installation. It serves as a knowledge base and reference for MCoT research.

Highlighted Details

• Comprehensive taxonomy of MCoT methodologies.
• Extensive lists of datasets and benchmarks for MCoT training and evaluation across modalities (text, image, video, audio, 3D, tables, charts).
• Detailed analysis of MCoT applications in diverse fields such as robotics, autonomous driving, and healthcare.
• Exploration of advanced techniques like Reinforcement Learning and Graph-of-Thought for enhancing MCoT reasoning.

Maintenance & Community

The project is actively maintained, with open discussions and a Slack channel available for community engagement. The authors welcome contributions and suggestions for missed related work.

Licensing & Compatibility

The repository itself is not licensed for software use. The survey references various research papers, each with its own licensing.

Limitations & Caveats

As a survey, this repository does not provide executable code. Its content is based on research papers published up to the survey's release date, and the field of MCoT is rapidly evolving.