CareGPT is a comprehensive framework for developing, training, and deploying medical Large Language Models (LLMs). It aims to accelerate progress in the medical LLM field by aggregating numerous open-source datasets, models, and tools for training, evaluation, and deployment. The project is targeted at researchers and developers working with medical AI.

How It Works

CareGPT supports a full LLM training pipeline, including pre-training, supervised fine-tuning (SFT), reward modeling, and reinforcement learning (RLHF/DPO). It leverages techniques like LoRA and QLoRA for efficient fine-tuning and integrates with popular deployment tools like Gradio and ChatGPT-Next-Web. The framework emphasizes the importance of data quality over quantity, advocating for large-scale pre-training followed by smaller-scale supervised fine-tuning for optimal results.

Quick Start & Requirements

• Installation: Clone the repository and install dependencies using pip install -r requirements.txt after creating a Python 3.11 environment (conda create -n llm python=3.11).
• Prerequisites: LLaMA model weights need to be downloaded and converted to Hugging Face format if not already. GPU with sufficient VRAM is required for training and inference.
• Resources: Training commands are provided for various models (LLaMA, LLaMA-2, Baichuan, Qwen, InternLM) and fine-tuning stages.
• Documentation: Links to video tutorials, online demos, and detailed setup guides are available.

Highlighted Details

• Aggregates over 60 hospital department consultation materials and provides tools for distilling medical data from GPT-4/ChatGPT.
• Achieved top performance on the CMB leaderboard's IvyGPT benchmark, outperforming ChatGPT and other medical LLMs.
• Offers pre-trained and fine-tuned medical LLMs for direct download and use.
• Supports fine-tuning with LoRA, QLoRA, and reinforcement learning methods.

Maintenance & Community

The project is actively maintained and has been recognized in industry events and publications. It cites relevant research and encourages community contributions via GitHub issues.

Licensing & Compatibility

The repository is licensed under the MIT License. However, a disclaimer states that resources are for academic research only and strictly prohibited for commercial use.

Limitations & Caveats

The project explicitly states that generated content is for academic research and not for actual medical diagnosis. Commercial use is prohibited. The project does not perform Chinese word segmentation for its models, though it claims satisfactory results.