HYPIR is an open-source implementation for image restoration leveraging diffusion score priors, specifically tailored for researchers and practitioners in computer vision and generative AI. It offers a novel approach to enhance image quality by harnessing the power of diffusion models, providing a flexible and efficient solution for upscaling and restoration tasks.

How It Works

HYPIR utilizes a LoRA-based approach, fine-tuning Stable Diffusion 2.1 with diffusion-yielded score priors. This method allows for efficient adaptation of a powerful pre-trained diffusion model to image restoration tasks. The architecture is designed for high-resolution output (above 2k) and employs a patch-based processing strategy with configurable patch sizes and strides, enabling effective handling of large images.

Quick Start & Requirements

• Installation: Clone the repository, create a conda environment (conda create -n hypir python=3.10), activate it (conda activate hypir), and install requirements (pip install -r requirements.txt).
• Prerequisites: Python 3.10, PyTorch, Hugging Face diffusers, accelerate, polars, and CUDA-enabled GPU for inference and training.
• Pretrained Models: Download HYPIR_sd2.pth from HuggingFace or OpenXLab.
• Inference: Run python app.py --config configs/sd2_gradio.yaml --local --device cuda for Gradio demo. Detailed inference command provided in README.
• Resources: Training requires significant computational resources. Inference can be run on a free T4 GPU.

Highlighted Details

• Official implementation of the HYPIR paper.
• Supports upscaling to resolutions above 2k.
• Offers optional GPT-based prompt generation for inference.
• Integrated with Colab, OpenXLab, and Replicate.

Maintenance & Community

The project is maintained by XPixelGroup. Contact emails are provided for code/paper inquiries and commercial collaboration.

Licensing & Compatibility

The software is released under a non-commercial use only license. Commercial use, reproduction, or distribution requires prior written permission from Dr. Jinjin Gu. This restriction is in addition to any applicable open-source license.

Limitations & Caveats

The primary limitation is the strict non-commercial use clause, which may restrict adoption in commercial products or research requiring commercialization. The README also warns about unauthorized websites using their model and comparison images.