Summary

Darwin is an open-source initiative building foundational large language models for natural sciences (physics, chemistry, materials science). It enhances LLM efficacy in scientific research by pretraining/fine-tuning on scientific literature and datasets, aiming to bridge the gap between specialized ML and generalist AI like GPT-4. Researchers benefit from a more capable tool for scientific discovery.

How It Works

Built on LLaMA, Darwin is pretrained/fine-tuned using scientific literature and FAIR datasets, augmented by Darwin-SIG generated instructions. A key strategy is QA + multi-task fine-tuning, proving most effective on LLaMA1. This approach integrates factual correctness and domain knowledge, outperforming generalist models in scientific Q&A and few-shot tasks.

Quick Start & Requirements

Install via pip install -r requirements.txt. Download Darwin weights from OneDrive. Inference requires >=10GB GPU memory (Darwin 7B). Fine-tuning examples use multi-GPU setups (e.g., 4x A100 80G). A Google Colab version is available.

Highlighted Details

• Outperforms GPT-3.5 fine-tuned models and GPT-4 in few-shot scientific Q&A.
• Achieved SOTA on MatBench for experimental bandgap prediction and metallic classification.
• LLaMA1 with QA + multi-task fine-tuning showed best performance among LLaMA variants.
• Full fine-tuning significantly surpasses LoRA; SFT effective on non-pretrained architectures.

Maintenance & Community

A UNSW AI4Science & GreenDynamics AI collaboration. Encourages feedback for safety improvements. Offers PhD/PostDoc positions; no explicit community channels listed.

Licensing & Compatibility

Dataset and model weights are CC BY NC 4.0 licensed. Usage is restricted to non-commercial, research purposes only. Models trained on this data are also limited to research use.

Limitations & Caveats

Under active development with unaddressed limitations. Not yet fine-tuned for maximum safety. Darwin 1.5 lacks the inverse design task for organic solar cells present in Darwin 1.0.