EditScore provides a series of open-source reward models (7B–72B parameters) designed to evaluate and enhance instruction-guided image editing. It addresses the critical need for high-fidelity reward signals in online Reinforcement Learning (RL) for image editing tasks. The project targets researchers and engineers working on image generation, editing, and RL fine-tuning, offering state-of-the-art performance and a reliable evaluation standard to unlock more effective AI-driven image manipulation.

How It Works

EditScore leverages meticulously curated data and a self-ensembling strategy to achieve state-of-the-art performance in reward modeling for image editing. A core contribution is EditReward-Bench, the first public benchmark specifically for evaluating reward models in this domain, featuring 13 diverse subtasks, 11 editing models, and expert human annotations. This rigorous evaluation guides the development of EditScore models, which serve as both a versatile reranker to improve editing outputs and a high-fidelity reward signal for stable and effective RL fine-tuning, outperforming general-purpose VLMs in this specialized area.

Quick Start & Requirements

• Primary Install: Recommended for users: pip install -U editscore. For developers: Clone the repository and run pip install -e ..
• Prerequisites: PyTorch (version dependent on CUDA setup). Python 3.12 is recommended for environment setup.
• Optional Dependencies: vllm is highly recommended for high-performance inference (pip install -U vllm).
• Links: News, Quick Start, Benchmark Usage, Citation, RL Fine-Tuning Guide.

Highlighted Details

• Achieves state-of-the-art performance, matching and often surpassing leading proprietary VLMs, including GPT-5 on the EditReward-Bench.
• Introduces EditReward-Bench, a comprehensive public benchmark for evaluating image editing reward models.
• Offers versatile applications as a best-in-class reranker for editing outputs and as a high-fidelity reward signal for RL fine-tuning.
• Supports various model backbones, including LLaMA-Factory, Qwen3-VL, and Qwen2.5-VL variants.

Maintenance & Community

The project shows active development with recent news indicating acceptance to ICLR 2026, release of training configurations, new Qwen3-VL variants, and a comprehensive RL training framework. Models and datasets are available on Hugging Face and ModelScope. No explicit community channels (e.g., Discord, Slack) are listed in the provided text.

Licensing & Compatibility

The provided README content does not explicitly state the software license. Therefore, compatibility for commercial use or closed-source linking cannot be determined from this information.

Limitations & Caveats

The primary caveat is the absence of explicit licensing information, which is crucial for adoption decisions. While the project offers extensive RL training code and datasets, specific hardware requirements beyond PyTorch and CUDA are not detailed, and performance may vary significantly based on the chosen model backbone and optional dependencies like vllm.