Inspectus is a Python library designed for visualizing and analyzing the internal workings of large language models (LLMs), particularly their attention mechanisms. It targets ML researchers and engineers seeking to understand model behavior, offering interactive visualizations directly within Jupyter notebooks.

How It Works

Inspectus provides several specialized visualization components. The core attention function visualizes attention scores as heatmaps, allowing users to see token-to-token relationships. It supports raw tensors and Hugging Face model outputs, with options to specify query and key tokens. The tokens function visualizes token-level metrics (like loss or entropy) and additional metadata, enabling detailed per-token analysis. The distribution plot visualizes data distributions using percentile bands, useful for identifying outliers.

Quick Start & Requirements

• Primary install: pip install inspectus
• Prerequisites: Python, Jupyter Notebook environment.
• Links:
  • Huggingface Tutorial: https://github.com/labmlai/inspectus/blob/main/notebooks/huggingface.ipynb
  • Custom Attention Map Tutorial: https://github.com/labmlai/inspectus/blob/main/notebooks/custom_attention.ipynb
  • GPT2 Example: https://github.com/labmlai/inspectus/blob/main/notebooks/gpt2.ipynb
  • Distribution Plot Tutorial: https://github.com/labmlai/inspectus/blob/main/notebooks/distribution.ipynb
  • MNIST Use Case: https://github.com/labmlai/inspectus/blob/main/notebooks/mnist.ipynb

Highlighted Details

• Interactive attention matrix, query/key heatmaps, and dimension heatmaps.
• Token visualization supports multiple metrics and custom token information.
• Distribution plots aid in identifying data outliers.
• Integrates directly with Hugging Face Transformers models.

Maintenance & Community

• Developed by labmlai.
• Citable via provided BibTeX entry.

Licensing & Compatibility

• License: Not explicitly stated in the README. Potential ambiguity for commercial use.

Limitations & Caveats

The README does not specify the license, which may impact commercial adoption. While designed for LLMs, specific compatibility with all model architectures or frameworks beyond Hugging Face is not detailed.