This library provides ONNX transformer pipelines for Go, enabling AI use cases like text generation and image classification directly within Go applications. It targets Go developers and ML engineers seeking to deploy and scale Hugging Face transformer models efficiently on their own hardware, bypassing the need for Python RPC services.

How It Works

Hugot leverages ONNX models for compatibility with Hugging Face's ecosystem, allowing models trained in Python to be exported and run with identical predictions in Go. It supports pluggable backends, including a native Go implementation (GoMLX), ONNX Runtime (ORT), and OpenXLA, with ORT and OpenXLA supporting GPU acceleration via CUDA. The library prioritizes ease of use and performance for production environments.

Quick Start & Requirements

• Installation: Can be used as a library (import github.com/knights-analytics/hugot) or a CLI.
• Backends: Requires specific build tags (-tags ORT, -tags XLA, -tags ALL) and potentially downloading .so or .a files for ONNX Runtime and tokenizers, or using the provided Docker image.
• CUDA: For GPU acceleration, requires Nvidia drivers, CUDA toolkit, and compatible cuDNN libraries.
• Docs: Examples provided in hugot_test.go and within the README.

Highlighted Details

• Supports various pipelines: feature extraction, text classification, token classification, zero-shot classification, text generation, cross-encoder, and image classification.
• Offers both CPU and GPU (Nvidia CUDA) inference acceleration.
• Includes beta support for training and fine-tuning transformer pipelines (feature extraction only) using XLA.
• CLI tool available for running Hugging Face pipelines without Python or Go dependencies.

Maintenance & Community

• Brought to you by Knights Analytics.
• Contributions are welcome; see contribution guidelines.

Licensing & Compatibility

• The README does not explicitly state the license. Compatibility for commercial use is not specified.

Limitations & Caveats

The library and CLI are currently only built and tested on amd64-linux. Untested accelerator backends include TensorRT, DirectML, CoreML, and OpenVINO. Training is limited to the FeatureExtractionPipeline and requires the XLA backend.