This project provides a FastAPI-based API and Streamlit frontend for interacting with the GPT-J language model and its variants. It offers endpoints for text generation and multilingual zero-shot text classification, targeting developers and researchers who need to integrate or test GPT-J capabilities.

How It Works

The API leverages FastAPI for its backend, providing RESTful endpoints for model inference. It exposes a /generate endpoint for text completion and a /classify endpoint for zero-shot text classification using provided labels. The project utilizes a Streamlit application for a user-friendly web interface to test these functionalities.

Quick Start & Requirements

• Install dependencies: pip install -r requirements.txt
• Download GPT-J weights: wget https://the-eye.eu/public/AI/GPT-J-6B/step_383500_slim.tar.zstd
• Decompress weights: tar -I zstd -xf step_383500_slim.tar.zstd
• Run API: python3 serve.py
• Run Streamlit app: python3 -m streamlit run streamlit_app.py --server.port 8000
• Prerequisites: TPU VM (tested on v2-8, v3-8), zstd package.
• API documentation: http://api.vicgalle.net:5000/docs
• Streamlit app: http://api.vicgalle.net:8000

Highlighted Details

• Supports text generation with configurable parameters like token_max_length, temperature, and top_p.
• Includes a multilingual zero-shot text classification endpoint.
• Provides example Python and Bash code snippets for API interaction.

Maintenance & Community

• Contact: vgallegoalcala at gmail dot com
• Sponsor: Aspie96
• Acknowledgements: TPU Research Cloud (https://sites.research.google/trc/)

Licensing & Compatibility

• The README does not explicitly state a license.

Limitations & Caveats

• Deployment is specifically targeted at TPU VMs, with no clear instructions or support for other hardware or cloud platforms.
• The project relies on downloading large model weights (GPT-J-6B), which requires significant disk space and bandwidth.