Summary

This project, ChatKG, offers an intelligent Q&A system by integrating large language models (LLMs) with knowledge graphs (KGs) constructed from raw text. It targets users needing sophisticated information retrieval and conversational interfaces, automating KG construction, completion, and leveraging ChatGLM-6B for dialogue. The system provides precise information retrieval and interactive visualization capabilities.

How It Works

The system processes raw text through data preprocessing, then utilizes PaddleNLP's UIE model for entity and relation extraction to build an initial KG. This KG is iteratively refined and expanded using the SPN4RE model for relation extraction and graph completion. ChatGLM-6B powers a multi-turn dialogue service that queries the KG, complemented by a Vue.js frontend for chat and KG visualization.

Quick Start & Requirements

• Primary Install: Clone the repository, set up a Python 3.8+ virtual environment, and install backend dependencies (pip install -r requirements.txt). Frontend requires Node.js 16+ and npm install.
• Prerequisites: NVIDIA GPU (3090 recommended, 24GB+ VRAM, dual cards preferred), CUDA (version dependent on PyTorch/PaddlePaddle install), specific PyTorch (e.g., 1.11.0+cu113) and PaddlePaddle (e.g., 2.4.2.post112) versions.
• Setup: Requires careful installation of specific library versions to avoid conflicts. Configuration involves setting environment variables (.env) for model paths and GPU IDs. Data preparation involves placing raw text files in data/raw_data/.
• Links: Demo video: https://github.com/littlewwwhite/KnowledgeGraph-based-on-Raw-text-A27/assets/34784086/aa089b51-0b88-4d2b-b7fe-d892c503e55c

Highlighted Details

• Provides an end-to-end pipeline for KG construction from raw text.
• Leverages advanced NLP models like PaddleNLP UIE and SPN4RE for information extraction and relation completion.
• Features interactive knowledge graph visualization alongside LLM-powered chat.
• Supports streaming output for real-time conversational responses.

Maintenance & Community

The project is primarily maintained by the author, littlewwwhite. No specific community channels (like Discord/Slack) or major corporate sponsorships are highlighted. Contributions are welcomed via Pull Requests.

Licensing & Compatibility

The project is released under the MIT License. This license is permissive and generally compatible with commercial use and integration into closed-source projects.

Limitations & Caveats

The system has significant hardware requirements, specifically demanding high-end NVIDIA GPUs with substantial VRAM (24GB+). Installation requires strict adherence to specific library versions (PyTorch, PaddlePaddle) tied to CUDA versions, posing a potential setup hurdle. The project appears to be largely driven by a single maintainer, which could impact long-term support and development velocity.