Summary

Q-ReAlign provides a framework for building lightweight, human-aligned multimodal judges using modern vision-language models (LMMs). It offers checkpoints at 0.8B, 4B, and 9B parameters, achieving performance comparable to larger models like Q-Align with significantly fewer resources. The project includes tools for dataset building, caching, scoring, and CLI workflows, making it accessible for users needing efficient image quality assessment.

How It Works

The framework employs a model-agnostic design, leveraging the ms-swift library for training and inference. It utilizes a unified template system and a configurable YAML interface to integrate various LMM backbones. A key innovation is its dataloading cache, which packs images into a single memory-mapped blob for efficient, high-throughput data access during training and evaluation. Robust in-training evaluation is built-in, logging performance curves and preserving top checkpoints.

Quick Start & Requirements

• Installation: Core functionality can be installed via pip install -e .. For runtime training/evaluation, install with pip install -e ".[runtime]", which includes ms-swift, transformers, deepspeed, and decord. A Docker image is also provided. Integration with pyiqa requires pip install pyiqa and pip install -U "transformers>=5.2".
• Prerequisites: Python ≥ 3.9. GPU is recommended for training/evaluation; the 0.8B model is CPU-runnable and requires < 4GB GPU memory.
• Links: Hugging Face Collection: https://huggingface.co/collections/q-future/q-realign. Docs: docs/METHOD.md, docs/ADAPTING.md. IQA-PyTorch integration: pyiqa.

Highlighted Details

• Achieves Q-Align-level performance with up to 50% fewer parameters.
• The 0.8B model is CPU-runnable and requires less than 4GB of GPU memory.
• All three model sizes (0.8B, 4B, 9B) match or surpass the original Q-Align's performance across seven benchmarks.
• The 9B model reaches an average SRCC of 0.896, compared to Q-Align's 0.869.
• Features a novel dataloading cache for significantly improved data throughput.
• Includes robust, distributed-safe in-training evaluation that logs performance curves and preserves top checkpoints.

Maintenance & Community

The repository is maintained by @Yushuo Zheng and @Zicheng Zhang. No community links (e.g., Discord, Slack) are explicitly provided in the README.

Licensing & Compatibility

The README does not specify a software license. This omission requires clarification for any adoption decision, particularly concerning commercial use or derivative works.

Limitations & Caveats

The primary caveat is the unspecified license, which poses a significant adoption risk. While the 0.8B model is CPU-runnable, full training and evaluation necessitate a GPU and installation of several heavy dependencies. The "5-minute tour" uses synthetic data and requires configuration adjustments for real-world model integration.