Summary: Graph-ToolFormer addresses the challenge of endowing Large Language Models (LLMs) with robust graph reasoning capabilities. It achieves this by integrating LLMs with Graph Neural Networks (GNNs) through prompt augmentation, enabling LLMs to leverage structured graph data for complex reasoning tasks. This project targets researchers and practitioners seeking to enhance LLM performance on tasks involving relational and structured information.

How It Works: The core approach involves a two-pronged strategy: fine-tuning LLMs on graph reasoning prompt datasets and coupling these fine-tuned LLMs with pre-trained GNN models. The LLM_Tuning component facilitates the LLM adaptation, while the Graph_Toolformer_Package provides a framework for executing reasoning tasks by orchestrating calls between the LLM and various GNN models (e.g., Graph-Bert, TransE). This synergy allows LLMs to query and interpret graph structures, improving their analytical power.

Quick Start & Requirements: Environment setup is managed via conda env create -f environment.yml. Key requirements include a GPU, preferably Turing or Ampere architecture (RTX 20s/30s/40s, A40-A100, T4+) for optimal performance, especially with 8-bit models, as CPU support for 8-bit mixed precision is noted as potentially problematic. Users must download ~100MB of graph datasets, ~5GB of fine-tuned LLM checkpoints, and ~5GB of GNN checkpoints from provided Google Drive links. The LLM fine-tuning script is python3 gtoolformer_gptj_script.py, and the demo execution is python3 ./src/Framework_Graph_Toolformer.py. Further details are available in the paper: IFMLab and arXiv.

Highlighted Details:

• Supports diverse graph datasets: bibliographic (Cora, PubMed), molecular (Proteins), social networks (Twitter), recommender systems (Amazon), and knowledge graphs (WordNet).
• Integrates multiple GNN architectures: Toolx, Graph-Bert, SEG-Bert, KMeans, BPR, TransE.
• Includes fine-tuning code for GPT-J, with plans to support LLaMA, GPT-2, OPT, and Bloom.
• Enables various reasoning tasks: property prediction, topic analysis, molecular function, community detection, and knowledge graph completion.

Maintenance & Community: The project appears actively developed, indicated by a detailed task list covering framework expansion, dataset integration, and model inclusion. However, specific details regarding maintainers, community channels (e.g., Discord, Slack), or release roadmaps are not provided in the README.

Licensing & Compatibility: No specific open-source license is mentioned in the provided README content. Users should verify licensing terms before integration, particularly for commercial applications.

Limitations & Caveats: CPU users may encounter issues with 8-bit model computations. Significant disk space is required for downloading large model checkpoints (up to 5GB each for LLM and GNNs). Several core framework release and service implementation tasks remain incomplete, suggesting the project is still under active development and may undergo changes.