This Rust library provides native, high-performance NLP pipelines and transformer models, porting Hugging Face's Transformers library. It targets developers needing efficient, ready-to-use NLP solutions in Rust, offering benefits like multi-threaded tokenization and GPU inference for tasks such as question answering, translation, and text generation.

How It Works

The library leverages tch-rs for bindings to LibTorch (PyTorch's C++ API) or onnxruntime for ONNX model execution, combined with rust-tokenizers for pre-processing. This approach allows for native Rust execution, avoiding the overhead of Python interop, and enables efficient multi-threaded operations and GPU acceleration.

Quick Start & Requirements

• Installation: Add rust-bert to Cargo.toml.
• Prerequisites: Requires LibTorch (v2.4 recommended). Automatic download is available via the download-libtorch feature flag. CUDA 12 is supported via TORCH_CUDA_VERSION=cu124. ONNX support requires enabling the onnx feature and potentially setting ORT_DYLIB_PATH.
• Resources: LibTorch downloads can be several GBs. Model weights are also substantial.
• Docs: crate documentation

Highlighted Details

• Supports a wide range of transformer architectures (BERT, GPT-2, BART, T5, etc.) and NLP tasks.
• Offers ready-to-use pipelines for common NLP tasks, simplifying integration.
• Includes a utility script for converting PyTorch model weights to a compatible Rust format.
• Benchmarks suggest significant speedups (2-4x) for text generation tasks compared to Python.

Maintenance & Community

The project is actively maintained by Guillaume Becquin. Further community interaction details are not explicitly provided in the README.

Licensing & Compatibility

The library is released under the MIT License, permitting commercial use and integration with closed-source projects.

Limitations & Caveats

Model parameter naming must align with the Rust schema when importing custom PyTorch models; otherwise, loading may fail unless load_partial is used. The ONNX export process for encoder-decoder models may result in multiple files, and omitting the "decoder with past" file can impact computational efficiency.