This ComfyUI custom node enables image segmentation using natural language prompts, leveraging GroundingDINO and Segment Anything (SAM) models. It targets users of the ComfyUI workflow manager who need precise object selection and masking capabilities within their image generation pipelines. The primary benefit is the ability to semantically identify and isolate any object in an image via text descriptions.

How It Works

The node integrates GroundingDINO for object detection based on text prompts and SAM for generating masks for detected objects. This two-stage approach allows for flexible and accurate segmentation, where GroundingDINO identifies potential objects matching the input text, and SAM refines these detections into precise masks. This combination offers a powerful way to interactively segment images using semantic understanding.

Quick Start & Requirements

• Install dependencies: pip3 install -r requirements.txt
• Models are automatically downloaded on first use or can be manually placed in ComfyUI/models/bert-base-uncased, ComfyUI/models/grounding-dino, and ComfyUI/models/sams.
• Requires Python dependencies listed in requirements.txt.
• Manual model downloads are linked in the README for faster setup.

Highlighted Details

• Implements core functionalities from sd-webui-segment-anything.
• Ensures output consistency with its predecessor for identical inputs.
• Supports multiple SAM model variants (vit_h, vit_l, vit_b, hq, mobile).
• Utilizes GroundingDINO with SwinT or SwinB backbones.

Maintenance & Community

• Project is based on work by continue-revolution.
• Open to contributions via pull requests.

Licensing & Compatibility

• License not explicitly stated in the README.
• Compatibility with ComfyUI workflows is the primary focus.

Limitations & Caveats

The README does not specify the license, which may impact commercial use. It also does not detail performance benchmarks or specific hardware requirements beyond standard Python dependencies.