Discover and explore top open-source AI tools and projects—updated daily.
mahmoodlabMultimodal foundation model for pathology image and text analysis
Top 88.5% on SourcePulse
TITAN is a multimodal whole-slide foundation model addressing limitations in clinical data for computational pathology. It enables patient- and slide-level analysis, especially for rare diseases, by extracting general-purpose slide representations and generating pathology reports. Pretrained via self-supervised learning and vision-language alignment, it generalizes to resource-limited scenarios without fine-tuning and outperforms existing models.
How It Works
TITAN (Transformer-based pathology Image and Text Alignment Network) combines visual self-supervised learning (SSL) and vision-language alignment for pre-training. It utilizes 335,645 whole-slide images (WSIs), over 182,000 pathology reports, and >423,000 synthetic captions. This dual strategy captures richer morphological semantics than single-approach models. Crucially, it avoids pretraining on large public histology collections (e.g., TCGA) to prevent benchmark data contamination.
Quick Start & Requirements
pip install -e ..huggingface_hub library. GPU recommended for inference.huggingface_hub.Highlighted Details
Maintenance & Community
Recent updates include the integration of TITAN slide feature extraction into TRIDENT (February 2025) and CONCHv1.5 feature extraction into CLAM (December 2024). The preprint and model weights were released in December 2024. The GitHub repository is the primary community hub.
Licensing & Compatibility
Released under CC-BY-NC-ND 4.0, strictly for non-commercial, academic research use with attribution. Commercial use, sale, or monetization requires prior written approval. Distribution and reproduction are restricted, necessitating individual registration and agreement to terms.
Limitations & Caveats
This is a "TITAN-preview" release. Decoder weights are removed from public release as a PHI leakage precaution, though encoder performance remains unaffected. Model access requires Hugging Face login and agreement to specific terms, including individual user registration and distribution restrictions.
4 weeks ago
Inactive
zdou0830
facebookresearch