exBERT is a visual analysis tool designed for researchers and practitioners to explore the learned representations and attention patterns within transformer models. It allows users to input sentences, visualize attention mechanisms, and search token embeddings across annotated corpora, offering insights into how models process language.

How It Works

exBERT processes user-provided sentences through specified Hugging Face transformer models, visualizing attention weights as curved lines. It enables interactive masking of tokens to observe changes in attention and allows users to view attention patterns from single heads or linear combinations. A key feature is the ability to search for token contextual representations across an annotated corpus, facilitating the discovery of linguistic features learned by specific attention heads.

Quick Start & Requirements

• Installation: Recommended via make env from the root directory, or manually by creating a conda environment from environment.yml.
• Prerequisites: Python 3.x, Conda, spaCy (python -m spacy download en_core_web_sm), and potentially libomp for FAISS.
• Setup: Requires installing modified sub-repositories (transformers, spacyface) and potentially compiling the frontend (npm install, npm run build).
• Resources: Corpus searching requires significant disk space for storing hidden representations and attentions, potentially many gigabytes.
• Links: exbert.net (demo and pre-paper)

Highlighted Details

• Supports a wide range of Hugging Face transformer models (BERT, GPT2, RoBERTa, etc.).
• Allows interactive masking of tokens to observe effects on attention.
• Enables searching token representations across annotated corpora.
• Visualizes attention patterns for single heads or linear combinations.
• Can discover linguistic features (POS, NER) learned by attention heads.

Maintenance & Community

Developed by researchers from IBM Research and Harvard NLP. The project utilizes modified sub-repositories of transformers and spacyface, indicating a custom fork-and-modify workflow.

Licensing & Compatibility

The README does not explicitly state a license. Compatibility for commercial use or closed-source linking is not specified.

Limitations & Caveats

exBERT does not visualize attention between pairs of input sequences, assuming all inputs are from a single sequence. It is currently limited to English language support. Corpus searching requires significant preprocessing and custom modifications to the model architecture for extracting contextual information.