RePaint provides an official PyTorch implementation for image inpainting using denoising diffusion probabilistic models. It addresses the challenge of filling missing image regions by leveraging known image content, making it suitable for researchers and practitioners in computer vision and generative AI. The method generates coherent and contextually relevant content for masked areas, outperforming existing state-of-the-art methods in user studies.

How It Works

RePaint utilizes pre-trained denoising diffusion probabilistic models and conditions them during inference. The process starts with pure noise and iteratively denoises the image. In each step, the known image regions are resampled with noise corresponding to the current denoising step, ensuring consistency. This conditioned denoising allows the model to generate content for unknown regions that is harmonized with the known parts, a key improvement over standard diffusion models.

Quick Start & Requirements

• Install: pip install numpy torch blobfile tqdm pyYaml pillow
• Prerequisites: PyTorch (e.g., 1.7.1+cu110), Python.
• Models/Data: Download via bash ./download.sh.
• Run Example: python test.py --conf_path confs/face_example.yml
• More Info: Paper, Appendix

Highlighted Details

• Outperforms autoregressive and GAN-based SOTA methods in user studies (42/44 cases).
• Handles challenging masks, including "every second line" and super-resolution tasks.
• Allows customization of noise schedules and resampling parameters for inference speed and quality.
• Conditions pre-trained diffusion models without requiring retraining.

Maintenance & Community

The project is based on OpenAI's guided-diffusion repository. Support is available via GitHub Issues and Pull Requests.

Licensing & Compatibility

The repository is released under the MIT License, permitting commercial use and integration with closed-source projects.

Limitations & Caveats

The ImageNet model exhibits a bias towards inpainting dogs due to dataset composition. Some experiments may not have been re-evaluated after code refactoring.