This project provides a GenAI-powered multi-agent system for medical diagnosis and healthcare research assistance, targeting healthcare professionals, researchers, and patients. It aims to streamline medical inquiries and research by integrating LLMs, computer vision, and advanced RAG techniques for more accurate and efficient information retrieval and analysis.

How It Works

The system employs a multi-agent architecture orchestrated by LangGraph, featuring specialized agents for diagnosis, information retrieval, and reasoning. It utilizes Docling for document parsing, extracting text, tables, and images for embedding. A sophisticated RAG system incorporates hybrid search (BM25 and vector embeddings via Qdrant), semantic chunking, query expansion, and confidence-based routing to agents like web search to mitigate hallucinations. Computer vision models are integrated for medical imaging analysis, with plans for brain tumor detection, chest X-ray classification, and skin lesion segmentation.

Quick Start & Requirements

• Installation: Docker or manual installation via pip install -r requirements.txt.
• Prerequisites: Python 3.11+, Docker (optional), API keys for LLMs (e.g., Azure OpenAI, OpenAI), Eleven Labs, Tavily, and Hugging Face. ffmpeg is required for speech services.
• Setup: Requires creating a .env file with API keys. Docker build and run commands are provided. Data ingestion is handled via ingest_rag_data.py.
• Links: Demo, LinkedIn, GitHub

Highlighted Details

• Multi-agent orchestration with structured graph workflows using LangGraph.
• Advanced RAG techniques including hybrid search, semantic chunking, and confidence-based agent handoff.
• Medical imaging analysis capabilities (Chest X-ray classification, Skin Lesion segmentation, Brain Tumor detection TBD).
• Real-time web search integration for up-to-date medical insights.
• Voice interaction via Eleven Labs API and input/output guardrails for safety.

Maintenance & Community

The project is actively maintained by souvikmajumder26. Contributions are welcome, with an issues tab available for feature requests. Contact information for the developer is provided via LinkedIn and GitHub.

Licensing & Compatibility

Licensed under the Apache-2.0 License. This permissive license allows for commercial use and integration into closed-source projects.

Limitations & Caveats

Some computer vision models, such as Brain Tumor Detection, are marked as "To Be Determined" (TBD). Initial runs may experience performance jitters due to model downloads. The system relies on external API keys, which may incur costs.