WiNGPT2 is a large language model specialized for the healthcare domain, offering intelligent medical Q&A, diagnostic support, and knowledge services. It aims to enhance efficiency and quality in healthcare by integrating professional medical knowledge and data. The project targets medical professionals and general users seeking reliable health information.

How It Works

WiNGPT2 is built upon Transformer architecture, leveraging models like Gemma and Llama 3 as base pre-trained models. It incorporates advanced techniques such as RoPE relative positional encoding, SwiGLU activation, and RMSNorm for improved performance. The model is trained on a vast, curated dataset of medical literature, guidelines, and clinical data, with specific optimizations for various healthcare scenarios and tasks.

Quick Start & Requirements

• Install/Run: Use Hugging Face transformers library.

  from transformers import AutoModelForCausalLM, AutoTokenizer
  model_path = "WiNGPT2-7B-Chat" # Or other available models
  tokenizer = AutoTokenizer.from_pretrained(model_path, trust_remote_code=True)
  model = AutoModelForCausalLM.from_pretrained(model_path, trust_remote_code=True).to("cuda")
  

• Prerequisites: Python, PyTorch, Hugging Face transformers. GPU with sufficient VRAM is recommended for optimal performance.
• Demo: A simple chatbot example is available (demo.py).
• Models: Available on Hugging Face Hub (e.g., WiNGPT2-7B-Chat, WiNGPT2-14B-Chat, WiNGPT2-Llama3-8B-Chat, and quantized versions).

Highlighted Details

• Offers specialized models like WiNGPT2-Gemma-2-9B and WiNGPT2-Llama3-8B.
• Supports quantized versions (AWQ, GGUF) for efficient deployment.
• Evaluated on custom benchmarks (WiNEval-MCKQuiz, WiNEval-MSceQA) showing competitive performance.
• Includes a roadmap for multimodal capabilities with WiNGPT-3.

Maintenance & Community

• Actively updated with new model releases and benchmark results.
• Community discussion via WeChat group (QR code provided in README).
• Contact via email (wair@winning.com.cn) or website (https://www.winning.com.cn).

Licensing & Compatibility

• Project licensed under Apache License 2.0.
• Model weights are subject to the base model's license (e.g., Qwen-7B). Compatibility for commercial use should be verified against base model licenses.

Limitations & Caveats

• While providing AI doctor functionalities, WiNGPT2's advice is for reference only and does not replace professional medical consultation.
• The project disclaims warranties for accuracy, reliability, or completeness, and users bear responsibility for decisions made based on its output.
• Some application examples mentioned (e.g., image report generation) may involve technologies beyond the core WiNGPT2 model.