DiffPIR offers a novel approach to plug-and-play image restoration by leveraging denoising diffusion models as generative priors. This method targets researchers and practitioners in computer vision and image processing, aiming to achieve state-of-the-art results in tasks like super-resolution, deblurring, and inpainting with fewer neural function evaluations (NFEs) than prior diffusion-based methods.

How It Works

DiffPIR integrates diffusion models into the traditional plug-and-play iterative framework. Instead of relying on discriminative Gaussian denoisers, it uses pre-trained unconditional diffusion models to estimate $\mathbf{x}^{(t)}0$ from $\mathbf{x}{t}$ at each timestep $t$. This denoised estimate is then refined by solving the data sub-problem, effectively combining the generative capabilities of diffusion models with the iterative refinement of plug-and-play methods. This approach is advantageous as it inherits the strong generative prior of diffusion models while maintaining a manageable NFE count.

Quick Start & Requirements

• Install via pip install -r requirements.txt after cloning the repository.
• Requires downloading pre-trained models using bash download.sh.
• For motion blur, an additional download from https://github.com/LeviBorodenko/motionblur is needed.
• Inference can be performed using python main_ddpir_sisr.py, python main_ddpir_deblur.py, or python main_ddpir_inpainting.py, or via configuration files with python main_ddpir.py --opt configs/<task>.yaml.
• Official project page and paper are linked in the README.

Highlighted Details

• Achieves state-of-the-art performance on FFHQ and ImageNet datasets for super-resolution, deblurring, and inpainting.
• Demonstrates competitive or superior results compared to methods like DPS, DDRM, and DPIR.
• Maintains low NFEs (≤100) while achieving high reconstruction faithfulness and perceptual quality.
• Supports multiple image restoration tasks including super-resolution, Gaussian and motion deblurring, and inpainting.

Maintenance & Community

The project is associated with authors from ETH Zurich and is partly supported by the ETH Zurich General Fund, Alexander von Humboldt Foundation, and Huawei Fund. No specific community channels (Discord/Slack) or active development roadmap are mentioned in the README.

Licensing & Compatibility

The repository does not explicitly state a license. The code is based on OpenAI's Guided Diffusion and DPIR, which may have their own licensing terms. Users should verify compatibility for commercial or closed-source use.

Limitations & Caveats

The README indicates that training new diffusion models requires following OpenAI's Guided Diffusion repository. Compatibility with specific hardware or software versions beyond standard Python dependencies is not detailed. The project was presented at a 2023 workshop, suggesting it may be relatively new.