This repository provides the code and data for a study on Multi-Agent Systems (MAS) failures, introducing the MAST taxonomy. It's targeted at researchers and practitioners in AI and MAS who need to understand and mitigate common failure modes in complex agent interactions. The project offers a structured approach to analyzing MAS failures, enabling more robust system design.

How It Works

The project introduces the Multi-Agent Systems Failure Taxonomy (MAST), a framework for categorizing and analyzing failures in MAS. It leverages a dataset of annotated MAS traces, including those annotated by LLM-as-a-Judge and human annotators, to systematically identify and classify failure patterns. This data-driven approach allows for a comprehensive understanding of the root causes of MAS malfunctions.

Quick Start & Requirements

• Install required libraries: pip install huggingface_hub pandas
• Download dataset: Use provided Python snippets to download from Hugging Face Hub (mcemri/MAD).
• Prerequisites: Python 3.x, Hugging Face Hub access.

Highlighted Details

• Presents the first comprehensive study and taxonomy (MAST) of MAS challenges.
• Offers a dataset with over 1,000 annotated MAS traces.
• Includes traces annotated by both LLM-as-a-Judge and human annotators.
• Provides a bibtex citation for the associated paper "Why Do Multi-Agent LLM Systems Fail?".

Maintenance & Community

No specific community channels or maintenance details are provided in the README.

Licensing & Compatibility

The README does not specify a license. The code and data are presented for research purposes, and citation is requested.

Limitations & Caveats

The repository focuses on failure analysis and does not provide tools for MAS development or simulation. The dataset annotation process, particularly LLM-as-a-Judge, may introduce biases or inaccuracies inherent to the models used.