This extension integrates Segment Anything (SAM) with Stable Diffusion Web UI for efficient, mask-based inpainting. It allows users to quickly generate masks by pointing at desired areas, significantly speeding up the inpainting workflow for artists and content creators.

How It Works

The extension leverages SAM's object segmentation capabilities to create masks directly within the Stable Diffusion Web UI. Users can select from various SAM models (including SAM 2, Fast SAM, and MobileSAM) and then refine masks using sketching tools. The generated masks are then used by Stable Diffusion's inpainting models to modify specific image regions based on user prompts.

Quick Start & Requirements

• Install via AUTOMATIC1111's Stable Diffusion Web UI: Extensions tab -> Install from URL -> https://github.com/Uminosachi/sd-webui-inpaint-anything.git. Restart UI.
• Requires AUTOMATIC1111's Stable Diffusion Web UI v1.3.0+.
• Optional: --xformers argument for memory efficiency.
• Model downloads are handled within the UI.
• Official Docs: https://github.com/Uminosachi/sd-webui-inpaint-anything

Highlighted Details

• Supports multiple Segment Anything models (SAM 2, Fast SAM, MobileSAM) with varying VRAM requirements.
• Offers mask refinement tools: expand, trim by sketch, add by sketch.
• Integrates with ControlNet for advanced inpainting control.
• Includes a "Cleaner" tab powered by Lama Cleaner.

Maintenance & Community

• Actively developed, with references to recent SAM research papers.
• No explicit community links (Discord/Slack) mentioned in the README.

Licensing & Compatibility

• Source code licensed under Apache 2.0.
• Compatible with AUTOMATIC1111's Stable Diffusion Web UI.

Limitations & Caveats

• Privacy extensions in browsers might interfere with mask retrieval.
• Older Gradio versions (<= 3.23.0) may display segmentation images small.
• Initial model downloads for inpainting can be time-consuming.