This repository offers fine-tuned versions of the Qwen3-1.7B language model, specifically adapted for medical domain chat applications with an "R1 inference style". It targets developers and researchers seeking specialized medical LLMs, providing a dataset and scripts for both full parameter and LoRA fine-tuning to facilitate the creation of models adept at medical query response.

How It Works

The project fine-tunes the Qwen3-1.7B base model using either full parameter updates or the memory-efficient LoRA technique. It employs the delicate_medical_r1_data dataset and includes dedicated scripts (train.py, train_lora.py) for each training methodology. The objective is to achieve a distinct "R1 inference style" tailored for medical conversations, as illustrated by the provided example dialogue.

Quick Start & Requirements

• Installation: pip install -r requirements.txt
• Prerequisites: Python environment.
• Hardware: Full parameter fine-tuning requires 32GB VRAM; LoRA fine-tuning requires 28GB VRAM. Lower VRAM usage is possible with the Qwen3-0.6B model or by reducing MAX_LENGTH.
• Data Preparation: Execute python data.py for automated dataset download, preprocessing, and validation split.
• Training: Use python train.py (full parameter) or python train_lora.py (LoRA).
• Inference: Utilize python inference.py (full parameter) or python inference_lora.py (LoRA).
• Dependencies: SwanLab for logging, HuggingFace Transformers, and PEFT library.

Highlighted Details

• Supports both full parameter and LoRA fine-tuning methods.
• Internal testing indicates full parameter fine-tuning outperforms LoRA.
• Model is specifically tuned for an "R1 inference style" in medical contexts.
• Features automated data preparation and logging via SwanLab.

Maintenance & Community

• The README provides no specific information regarding maintainers, community channels (e.g., Discord, Slack), or project roadmaps.
• Mentions integration with SwanLab, HuggingFace Transformers, and PEFT libraries.

Licensing & Compatibility

• The README does not specify the license for the code or the fine-tuned model, leaving its terms of use and compatibility for commercial applications unclear.

Limitations & Caveats

• Significant VRAM requirements (28-32GB) are necessary for fine-tuning the 1.7B parameter model.
• The absence of explicit licensing information poses a potential adoption blocker for commercial use.
• Performance claims are based on internal testing results.