This repository curates research papers on Mechanistic Interpretability (MI) for Transformer-based Language Models (LMs). It addresses the need for a structured overview of this complex field, targeting researchers, engineers, and power users. The project provides a taxonomy and a beginner's roadmap, facilitating a quicker understanding and adoption of MI techniques for analyzing LMs.

How It Works

The project organizes a comprehensive collection of academic papers related to Mechanistic Interpretability (MI) for Transformer LMs. It follows a taxonomy and a beginner's roadmap, derived from the survey paper "A Practical Review of Mechanistic Interpretability for Transformer-Based Language Models." This structured approach aims to demystify MI by categorizing techniques, evaluation methods, findings, and applications, offering a clear path for newcomers and experts alike.

Quick Start & Requirements

This repository serves as a curated list of research papers and does not include executable code or a direct installation process. Contributions of relevant papers are welcomed via pull requests.

Highlighted Details

• Techniques: Covers a wide array of MI techniques including Logit Lens, Probing, Sparse Autoencoders (SAE), Activation Patching, Path Patching, Causal Scrubbing, and Automated Circuit Discovery (ACDC).
• Key Findings: Documents discoveries such as "knowledge neurons," "skill neurons," "successor heads," and circuits for tasks like indirect object identification, modular addition, and factual association recall. It highlights evidence of component reuse across different tasks and models, suggesting universality.
• Tools: Lists essential libraries and tools for MI research, such as TransformerLens, CircuitsVis, LM Debugger, Neuroscope, and Neuronpedia, providing entry points for practical application.
• Evaluation: Details common evaluation metrics used in MI research, including Faithfulness, Completeness, Minimality, Plausibility, and Automated Explanation Scores.

Maintenance & Community

The repository was launched in June 2024 and is actively under construction, with the first iteration of paper collection completed by July 2024. Contributions of relevant papers are encouraged. For questions or suggestions, users can report an issue or contact the maintainers via email at drai2@gmu.edu and ziyuyao@gmu.edu.

Licensing & Compatibility

No specific license information is provided within the README.

Limitations & Caveats

As of its June 2024 launch, the repository is explicitly stated to be "Still under construction." Its primary function as a curated list means it does not offer direct software functionality but rather serves as a guide to existing research.