This repository provides a unified toolbox for Natural Language Processing (NLP) tasks, integrating state-of-the-art Transformer models like BERT and GPT-2 with Graph Neural Network (GNN) architectures. It aims to simplify the implementation of complex NLP and GNN models for researchers and practitioners.

How It Works

The library offers pre-built layers for BERT, ALBERT, and GPT-2 within TensorFlow 2.0, allowing seamless integration into custom models. It also implements core GNN message-passing mechanisms for architectures such as GCN, GAN, GIN, and GraphSAGE, enabling their application to NLP tasks like text classification and sequence labeling.

Quick Start & Requirements

• Install: python setup.py install
• Prerequisites: TensorFlow-GPU >= 2.0, typeguard, gensim, tqdm, sentencepiece.
• Usage: python <script_name>.py (e.g., python bert_ner_train.py)
• Data Format: Specific input formats are detailed for NER and classification tasks.
• More Info: Examples are available in the tests directory.

Highlighted Details

• Integrates BERT, ALBERT, and GPT-2 as TensorFlow layers.
• Implements GCN, GAN, GIN, and GraphSAGE based on message passing.
• Supports various NLP tasks including NER, text classification, and text generation.
• Provides benchmark results for multiple models on standard datasets (e.g., CoNLL-2003, SST-2, Cora, Pubmed).

Maintenance & Community

• Last update mentioned is May 2020.
• Contact: zhoukaiyinhzau@gmail.com for questions.

Licensing & Compatibility

• The README does not explicitly state a license. Compatibility for commercial use is not specified.

Limitations & Caveats

• The project's last update was in May 2020, indicating potential lack of recent maintenance or updates for newer NLP advancements.
• Some GNN benchmark results are noted as being lower than cited papers.