This repository provides a comprehensive, chronologically organized survey of Mixture-of-Experts (MoE) models in Large Language Models (LLMs) and related domains. It serves researchers and practitioners interested in the evolution, taxonomy, and implementation of MoE architectures, offering a curated list of papers, distinguishing between open-source and proprietary models, and categorizing them by application area (NLP, Vision, Multimodal, RecSys).

How It Works

The project's core value lies in its curated and structured compilation of research papers on MoE models. It presents a chronological overview, primarily ordered by publication or release date, to illustrate the development trajectory of MoE architectures. The survey categorizes models by domain (NLP, Computer Vision, Multimodal, Recommender Systems) using distinct color codings and differentiates between open-source and closed-source implementations, providing a clear landscape of the MoE research field.

Quick Start & Requirements

This repository is a curated list of research papers and does not involve code execution or installation. All information is presented in the README.

Highlighted Details

• Chronological overview of MoE models from 2017 to present.
• Categorization of models by domain: NLP (green), Computer Vision (yellow), Multimodal (pink), Recommender Systems (cyan).
• Distinction between open-source (above arrow) and proprietary (below arrow) models.
• Includes links to arXiv, conference proceedings, and GitHub repositories for many listed papers.

Maintenance & Community

The repository is actively maintained, with contact information provided for suggestions or corrections. Contributions are welcomed.

Licensing & Compatibility

The repository itself contains no code, only a list of research papers. The licensing of the individual papers or associated codebases would need to be checked on their respective sources.

Limitations & Caveats

The repository is a survey and does not provide implementations or code. The accuracy and completeness of the list depend on the maintainers' ongoing curation efforts.