This repository offers a simplified, more understandable implementation of Facebook's Segment Anything (SAM) models (v1, v2, v3). It targets developers and researchers seeking to grasp SAM's internals, providing enhanced capabilities like video tracking on arbitrarily long videos and adjustable input resolutions for faster inference.

How It Works

MuggledSAM deconstructs the original SAM codebase, removing complexity to clarify its architecture and algorithms. It leverages core SAM models but extends functionality, notably enabling robust video tracking across extended sequences by building upon SAMv2 and SAMv3. The project also allows input image resolution adjustments, a key optimization for inference speed.

Quick Start & Requirements

Installation requires Python 3.10+ and a virtual environment, followed by pip install -r requirements.txt. GPU acceleration needs specific PyTorch commands (e.g., pip3 install torch --index-url https://download.pytorch.org/whl/cu121), with a note on Windows c10.dll dependency requiring vc_redist.x64.exe. Model weights are downloaded separately from original SAM repos; SAMv3 weights require an agreement. Demo scripts (run_image.py, run_video.py, run_detections.py) are included. PyTorch installation details: pytorch.org/get-started/locally/.

Highlighted Details

• Enables video tracking on arbitrarily long videos using SAMv2 and SAMv3 models.
• Supports input image resolution adjustments to optimize inference speed.
• Provides three interactive demo scripts for image, video segmentation, and object detection.
• Includes a walkthrough for SAMv1 model structure; v2/v3 documentation planned.

Maintenance & Community

This project simplifies existing SAM models. The README lacks specific maintainer details, community channels, or a roadmap. Key "TODOs" include completing v2/v3 model documentation and addressing bug fixes.

Licensing & Compatibility

The muggled_sam repository does not explicitly state a license. As a derivative work simplifying official facebookresearch/segment-anything models, it is presumed to align with the Apache 2.0 license of the original repositories. Commercial use compatibility likely follows the terms of the underlying SAM models.

Limitations & Caveats

The run_video.py script is a "messy work-in-progress" with ongoing stability and feature updates. Downloading SAMv3 model weights requires signing an agreement. Documentation for SAMv2 and SAMv3 model structures is pending.