AlphaEdit provides a method for targeted knowledge editing in large language models, aiming to update specific factual information while preserving existing knowledge. It is designed for researchers and practitioners working on LLM interpretability and controlled generation, offering a principled approach to minimize disruption during sequential edits.

How It Works

AlphaEdit constrains parameter perturbations to the null space of key matrices, effectively isolating the update to specific dimensions. It further removes output errors related to preserved knowledge from the objective function, allowing the model to focus solely on the knowledge update. This null-space projection ensures that the distribution of hidden representations remains invariant post-edit, enabling simultaneous knowledge update and preservation.

Quick Start & Requirements

• Install: pip install -r requirements.txt (after cloning)
• Prerequisites:
  • PyTorch 1.12.1
  • CUDA >= 12 (implied by A40 GPU requirement)
  • At least one A40 48GB GPU
  • Pre-calculated Llama3-8B "cov" matrix (download link provided, save to ./data/stats)
• Run Example:

  python3 -m experiments.evaluate \
      --alg_name=AlphaEdit \
      --model_name=meta-llama/Meta-Llama-3-8B-Instruct \
      --hparams_fname=Llama3-8B.json \
      --ds_name=mcf \
      --dataset_size_limit=2000 \
      --num_edits=100 \
      --downstream_eval_steps=5
  

• Docs: README

Highlighted Details

• ICLR 2025 Outstanding Paper award.
• Minimizes disruption to preserved knowledge via null-space projection.
• Ensures invariance of hidden representation distribution post-edit.
• Focuses on optimizing sequential editing from an objective standpoint.

Maintenance & Community

• Code is based on MEMIT and EMMET.
• No explicit community links (Discord/Slack) or roadmap are provided in the README.

Licensing & Compatibility

• The README does not explicitly state a license. The base projects (MEMIT, EMMET) have varying licenses; users should verify compatibility.

Limitations & Caveats

• Requires specific hardware (A40 48GB GPU).
• Relies on pre-calculated covariance matrices for specific models, which may not be available for all target LLMs.
• The README does not specify the license, creating potential ambiguity for commercial use.