This repository provides the code and datasets for Graph Chain-of-Thought (Graph-CoT), a method for augmenting Large Language Models (LLMs) with structured knowledge from graphs. It addresses LLM hallucination by enabling step-by-step graph traversal for information retrieval, targeting researchers and developers working on grounded LLM reasoning.

How It Works

Graph-CoT is an iterative framework that allows LLMs to navigate graphs to find relevant information. Each iteration involves three steps: LLMs reason about current knowledge and identify needed information, generate interaction requests to fetch data from the graph, and execute these requests. This process repeats until the LLM can formulate a final answer, enabling chain-of-thought reasoning directly on graph structures.

Quick Start & Requirements

• Install: Use conda create --name graphcot python==3.8, activate, then conda install pytorch==1.12.1 torchvision==0.13.1 torchaudio==0.12.1 cudatoolkit=11.3 -c pytorch and conda install -c pytorch -c nvidia faiss-gpu=1.7.4, followed by pip3 install -r requirements.txt.
• Prerequisites: Python 3.8, PyTorch 1.12.1, CUDA 11.3, FAISS-GPU 1.7.4.
• Data: Download graph environment files to data/processed_data/{data_name} and question answering datasets to data/processed_data/{data_name}/data.json.
• Links: ACL 2024 Paper

Highlighted Details

• Introduces Graph Reasoning Benchmark (GRBench) with 10 real-world graphs across five domains (academic, e-commerce, literature, healthcare, legal).
• GRBench includes questions requiring single-hop, multi-hop, and inductive reasoning.
• Graph-CoT enables LLMs to traverse graphs iteratively, fetching information contextually.
• Supports both rule-based (EM, BLEU, ROUGE) and GPT-4 based evaluation.

Maintenance & Community

• The project is associated with ACL 2024.
• Some code is adapted from ToolQA.

Licensing & Compatibility

• The repository does not explicitly state a license in the README.

Limitations & Caveats

• The setup requires specific older versions of PyTorch (1.12.1) and CUDA (11.3), which may pose compatibility challenges with newer hardware or software stacks.
• The README does not specify the license, creating uncertainty for commercial or closed-source use.