Summary

MedArk addresses the challenge of Large Language Models (LLMs) struggling with imperfect initial information in multi-turn dialogues, particularly in medical consultations. It enables LLMs to proactively ask clarifying questions and retrieve relevant knowledge, thereby improving the reliability and accuracy of responses. This project is targeted at researchers and developers seeking to enhance LLM collaboration in scenarios where initial user input is incomplete or ambiguous, offering a framework to mitigate hallucinations and foster more effective human-AI interaction.

How It Works

The core innovation is the "Ask and Retrieve Knowledge framework (Ark)," which empowers LLMs to self-reason and decide whether to "ask" for more information or "tell" an answer at each dialogue turn. This decision-making process is enhanced by retrieving knowledge pertinent to the user's input. By generating action paths for dialogues and training the MedArk model on this data, the framework equips LLMs with the ability to proactively seek missing information, mitigate medical hallucinations through active knowledge retrieval, and make informed decisions about subsequent actions.

Quick Start & Requirements

• Models & KB: Pre-trained models (MedArk-1.5B, MedArk-3B, MedArk-7B) and a local medical knowledge base are available on Huggingface (direct links are placeholders in the README).
• Prerequisites: Python, hashlib, TypedDict, torch, transformers, chroma-db, gradio. A Go environment is required for construction-sft/second_pass_gen.ipynb.
• Setup: Requires extensive manual data preparation, including creating tagged.pkl from dialogues, building a ChromaDB vector store, and generating SFT/DPO training data via multiple Python scripts and Jupyter notebooks. This is followed by SFT and DPO training stages.
• Deployment: A Gradio web UI can be deployed using deploy/depl_web_ui.py.
• Documentation: The SIGIR'25 paper citation is provided. Various notebooks and scripts for data construction, training, and inference are included within the repository structure.

Highlighted Details

• Proactive Questioning: Enables LLMs to intelligently ask clarifying questions when faced with incomplete information.
• Hallucination Mitigation: Actively retrieves medical knowledge to reduce the generation of incorrect or fabricated information.
• Self-Reasoning Actions: LLMs determine optimal dialogue actions ("ask" vs. "tell") through internal reasoning processes.
• SIGIR'25 Publication: The project is associated with research presented at the ACM SIGIR Conference.

Maintenance & Community

No specific community channels (e.g., Discord, Slack), active contributor information beyond the listed authors, or roadmap details are provided in the README. The project appears primarily linked to its associated academic publication.

Licensing & Compatibility

The README does not specify a software license. Users should verify compatibility for commercial use or integration with closed-source projects.

Limitations & Caveats

The current code is a prototype intended solely for demonstration purposes and is not production-ready. The usage instructions are described as not user-friendly, and significant manual effort is required for data preparation and model training pipelines. A Go environment is a specific, non-Python dependency for a key data generation step.