FastEdit provides an efficient method for injecting customized knowledge into large language models (LLMs) via a single command. It targets LLM developers and researchers seeking to update or correct factual information within pre-trained models, offering a significant speed advantage over retraining.

How It Works

FastEdit implements the Rank-One Model Editing (ROME) algorithm. This approach targets specific weight matrices within the transformer architecture, enabling localized updates to factual knowledge without requiring full model retraining. This targeted editing is advantageous for its speed and reduced computational cost compared to fine-tuning or full model updates.

Quick Start & Requirements

• Installation: pip install pyfastedit or clone the repository and install requirements (pip install -r requirements.txt).
• Prerequisites: Python 3.8+, PyTorch 1.13.1+, 🤗Transformers, Datasets, Accelerate, sentencepiece, fire.
• Hardware: LLaMA 7B (FP16) requires 24GB GRAM for ~7s/it editing; LLaMA 13B (FP16) requires 32GB GRAM for ~9s/it editing.
• Data Format: JSON file with "prompt", "subject", and "target" fields.
• Editing Command: CUDA_VISIBLE_DEVICES=0 python -m fastedit.editor --data <data_path> --model <model_name> --config <config_name> --template <template_name>
• Documentation: https://github.com/hiyouga/FastEdit

Highlighted Details

• Supports editing of GPT-J, LLaMA, LLaMA-2, BLOOM, Falcon, Baichuan, and InternLM models.
• Demonstrates successful factual correction on multilingual samples.
• Claims editing times under 10 seconds for supported models.
• Codebase benefits from ROME implementation by Meng et al.

Maintenance & Community

• Project initiated by "hiyouga".
• No explicit community channels (Discord/Slack) or roadmap are linked in the README.

Licensing & Compatibility

• Licensed under Apache-2.0.
• Permissive license suitable for commercial use and integration into closed-source projects.

Limitations & Caveats

The project is marked with a TODO list indicating ongoing development, including plans to implement MEMIT, automatic subject/target identification, and methods to avoid performance degradation in instruction-following models. The current focus appears to be on factual editing rather than broader behavioral changes.