This repository serves as a comprehensive, community-driven survey of model merging techniques across Large Language Models (LLMs), Multimodal LLMs (MLLMs), and other machine learning domains. It aims to systematically catalog methods, theories, applications, and future opportunities, addressing a gap in existing literature by providing a structured overview for researchers and practitioners.

How It Works

The repository categorizes model merging methods into distinct approaches, including pre-merging techniques (like linearization and sparse fine-tuning), during-merging methods (weighted-based, subspace-based, routing-based), and post-calibration methods. It also details applications across various ML subfields such as continual learning, multi-task learning, generative models, and federated learning, providing a taxonomic framework for understanding the landscape.

Quick Start & Requirements

This is a curated list of research papers and does not involve code execution. Requirements are met by accessing the cited papers, typically available via arXiv or conference proceedings.

Highlighted Details

• Extensive Taxonomy: Covers a wide array of model merging techniques, from basic weight averaging to advanced subspace and routing-based methods.
• Broad Application Scope: Details applications in LLMs, MLLMs, image/video generation, continual learning, federated learning, and more.
• Up-to-Date Research: Includes recent papers, with many from 2024 and 2025, reflecting the rapid advancements in the field.
• Community Driven: Actively welcomes contributions and clarifications from the research community.

Maintenance & Community

The repository is maintained by EnnengYang and welcomes community contributions via pull requests or direct contact. Email addresses for contributions are provided.

Licensing & Compatibility

The repository itself is a collection of links and information; licensing is determined by the individual papers cited. Compatibility for commercial use or closed-source linking depends on the licenses of the referenced research papers.

Limitations & Caveats

This repository is a survey and does not provide executable code or benchmarks. The primary limitation is its nature as a reference list, requiring users to access and evaluate the cited papers independently.