Summary This repository provides an official codebase for fine-tuning the Segment Anything Model (SAM) on customized medical imaging datasets. It offers a comprehensive empirical study and practical tools to adapt foundation models for specialized medical segmentation tasks, aiming to improve performance through optimized fine-tuning strategies and pre-training.

How It Works The project systematically evaluates fine-tuning SAM across diverse medical imaging scenarios (single/multiple labeled/unlabeled datasets), investigating combinations of backbones, components, and algorithms. Key findings highlight the superiority of parameter-efficient learning methods (Adapter, LoRA) applied to both encoder and decoder. Task-agnostic self-supervised pre-training before fine-tuning yields better downstream performance than standard SAM initialization, while network architecture shows minimal impact.

Quick Start & Requirements Installation via Conda (environment.yml) or pip (requirements.txt). Datasets require preprocessing into 2D slices with CSV lists. Users need SAM/MobileSAM checkpoints. Configuration involves selecting encoder architectures (vit_h, vit_b, vit_t) and fine-tuning methods (vanilla, Adapter, LoRA) for encoder/decoder. Training uses single/multi-GPU scripts, with options for GPU splitting. MedSAM and self-supervised pretraining (SSLSAM) are supported. Example scripts and visualization tools (Tensorboard, notebooks) are provided.

Highlighted Details

• Fine-tuning SAM offers a slight performance improvement over prior medical image segmentation methods.
• Parameter-efficient learning strategies (Adapter, LoRA) applied to both encoder and decoder demonstrate superior results.
• Task-agnostic self-supervised pre-training significantly enhances downstream task performance compared to standard SAM initialization.
• The choice of network architecture has a minimal impact on the final segmentation outcome.

Maintenance & Community Contact for issues is via email (hanxue.gu@duke.edu), with potential response delays noted. The codebase builds upon SAM, MobileSAM, MedSAM, Medical SAM Adapter, and LoRA for SAM. Development is supported by Duke University.

Licensing & Compatibility The repository's license is not explicitly stated in the README, requiring clarification for adoption decisions, especially concerning commercial use or integration into closed-source projects.

Limitations & Caveats Ongoing development includes prompt-based multi-class segmentation. Author notes potential delays in GitHub issue responses. Users must handle dataset conversion to 2D slices and ensure compatibility with required checkpoints.