SAM-Med2D offers a fine-tuned version of Meta's Segment Anything Model (SAM) specifically for 2D medical image segmentation. It addresses the need for robust segmentation across diverse medical modalities and anatomical structures by leveraging a massive dataset and an efficient adapter-based fine-tuning approach. This project is targeted at researchers and practitioners in medical imaging who require high-performance segmentation tools.

How It Works

SAM-Med2D adapts the SAM architecture by freezing the image encoder and introducing learnable adapter layers within each Transformer block. This allows the model to acquire domain-specific knowledge from medical imaging data. The prompt encoder is fine-tuned for point, bounding box, and mask inputs, while the mask decoder is updated through interactive training, enhancing its precision for medical segmentation tasks.

Quick Start & Requirements

• Install via pip (requires PyTorch).
• Dependencies: Python 3.8+, PyTorch, Apex (for mixed-precision training).
• Pre-trained models and the SA-Med2D-20M dataset (4.6M images, 19.7M masks) are available via Baidu Cloud.
• Official Demo: OpenXLab
• Notebook Demo: predictor_example.ipynb

Highlighted Details

• Achieves state-of-the-art performance on 9 MICCAI2023 datasets, outperforming the base SAM and FT-SAM.
• Trained on SA-Med2D-20M, the largest medical image segmentation dataset to date, covering 10 modalities and 31 organs.
• Supports ONNX export for encoder and decoder models for deployment.
• Offers interactive fine-tuning with point, box, and mask prompts.

Maintenance & Community

• Active development with recent updates in late 2023.
• Project led by OpenGVLab, Shanghai AI Lab.
• Hiring for researchers and engineers in medical AI.
• WeChat group for discussion.

Licensing & Compatibility

• Licensed under Apache 2.0.
• Permissive license suitable for commercial use and integration into closed-source projects.

Limitations & Caveats

• The project acknowledges anomalies in previously published test data (Table 4) and is working on updates.
• While SAM-Med2D focuses on 2D, a related project SAM-Med3D is available for 3D medical imaging.