MAV (Model Activity Visualiser) provides a real-time, interactive visualization of Large Language Model (LLM) internal states during text generation. It is designed for researchers and developers seeking to understand and debug LLM behavior, offering insights into attention mechanisms, token predictions, and generated text.

How It Works

MAV leverages the Hugging Face transformers library to load and run various LLM architectures. It visualizes key internal states such as attention entropy and top token predictions, alongside the generated text itself. The tool is built with a plugin architecture, allowing for custom visualizations and integration into training loops, enhancing interpretability.

Quick Start & Requirements

• Install: uv run --with openmav mav --model gpt2 --prompt "hello mello" or pip install openmav.
• Prerequisites: Python, Hugging Face transformers compatible models (e.g., gpt2, meta-llama/Llama-3.2-1B). For gated models, huggingface-cli login is required.
• Documentation: documentation.md

Highlighted Details

• Real-time visualization of LLM internal states.
• Supports various Hugging Face models and sampling parameters.
• Plugin architecture for custom visualizations and training loop integration.
• Command-line interface for easy experimentation.

Maintenance & Community

The project is maintained by "attentionmech". Further community or contribution details are not explicitly provided in the README.

Licensing & Compatibility

The project appears to be licensed under the MIT License, allowing for broad use and modification, including commercial applications.

Limitations & Caveats

The README mentions a citation with a future year (2025), suggesting the project may be in early development or pre-publication. Specific performance benchmarks or detailed compatibility matrices are not provided.