This repository provides a fine-tuned Falcon-7B Large Language Model (LLM) for mental health conversational applications. It addresses the need for accessible, empathetic AI support in mental well-being, targeting users seeking preliminary assistance or information. The project leverages QLoRA for efficient fine-tuning on a custom conversational dataset derived from reputable health sources.

How It Works

The project fine-tunes the sharded Falcon-7B LLM using the QLoRA technique, which significantly reduces memory requirements by quantizing model weights and applying low-rank adaptations. This approach allows for efficient training on consumer-grade hardware, such as an Nvidia T4 GPU, while achieving a reported training loss of 0.031 after 320 steps. The model is trained on a curated conversational dataset anonymized and pre-processed from mental health FAQs, blogs, and wiki articles.

Quick Start & Requirements

• Install/Run: Clone the repository and run the gradio_chatbot_app.ipynb notebook.
• Prerequisites: Nvidia GPU (A100 recommended, T4 usable with max_steps < 150), Python, Google Colab Pro (optional but recommended for faster training).
• Setup Time: Fine-tuning can take under an hour on an A100 or potentially longer on a T4. Inference is reported to take under 3 minutes per response.
• Links: Dataset, [Blog Post](Fine-tuning of Falcon-7B Large Language Model using QLoRA on Mental Health Dataset), Fine-tuned Model

Highlighted Details

• Fine-tuned Falcon-7B LLM using QLoRA.
• Utilizes a custom, anonymized mental health conversational dataset.
• Achieved low training loss (0.031) in under an hour on A100.
• Offers a Gradio-based chatbot interface for demonstration.

Maintenance & Community

The project is maintained by iamarunbrahma. Further queries can be directed via GitHub Issues or blog comments.

Licensing & Compatibility

The repository does not explicitly state a license. Compatibility for commercial use or closed-source linking is not specified.

Limitations & Caveats

The project explicitly states that the chatbot is not a replacement for professional mental health care. The dataset is derived from public sources, and the quality of responses may vary. No specific performance benchmarks against other models are provided.