Zhongjing-LLaMA is a Chinese medical large language model designed to address the limitations of general-purpose LLMs in specialized domains. It offers a comprehensive solution for medical dialogue, aiming to provide professional-level responses and improve active medical inquiry capabilities for researchers and developers in the medical AI field.

How It Works

Zhongjing-LLaMA employs a full training pipeline, including pre-training, supervised fine-tuning (SFT), and reinforcement learning with human feedback (RLHF). This approach leverages a large-scale pre-training corpus and a unique 70,000-utterance multi-turn dialogue dataset (CMtMedQA) derived from real doctor-patient interactions. The RLHF stage, incorporating expert feedback, is crucial for enhancing safety and professional accuracy, distinguishing it from models solely fine-tuned on single-turn dialogues.

Quick Start & Requirements

• Install/Run: Modify scripts/cli_demo.sh with model paths and run.
• Prerequisites: Python, LLaMA base model, reward model, and LoRA weights (links provided in README).
• Resources: Training logs indicate effective convergence for pre-training, reward model, and PPO stages.
• Links: Paper, CMtMedQA Dataset

Highlighted Details

• First Chinese medical LLM with a complete pre-training to RLHF pipeline.
• Utilizes a 70,000-utterance multi-turn dialogue dataset (CMtMedQA) from real medical conversations.
• Includes a 1,000-utterance test set (CMtMedQA_test) for evaluating multi-turn dialogue and safety.
• Demonstrates near-professional doctor performance in certain dialogue scenarios.

Maintenance & Community

• Initiated by Zhengzhou University Natural Language Processing Laboratory.
• Key contributors: Songhua Yang, Hanjie Zhao, Senbin Zhu.
• Based on Ziya-LLaMA.
• Uses RLHF tools (link provided).

Licensing & Compatibility

• License details are not explicitly stated in the README.
• Model weights are provided for research purposes.

Limitations & Caveats

The model is intended for research use only, and users assume all medical risks. While significant progress has been made, there is still room for improvement in safety and professionalism, with the potential for unexpected responses in certain situations.