curated-transformers  by explosion

A PyTorch library providing state-of-the-art Transformer models and reusable components, targeting researchers and developers needing flexible, efficient, and well-annotated implementations of LLMs like Falcon and Llama. It simplifies model customization and integration, offering benefits like 4/8-bit inference and PyTorch meta device support.

How It Works

Models are constructed from a set of reusable building blocks, enabling features like 4/8-bit inference via bitsandbytes and PyTorch meta device for efficient memory management. This modular design facilitates easy addition of new architectures and ensures bug fixes or feature implementations benefit all models. Public APIs feature consistent type annotations for enhanced IDE support and code maintainability.

Quick Start & Requirements

• Install: pip install curated-transformers
• CUDA: PyTorch build with CUDA 11.7 is default. For Windows or Ada-generation GPUs on Linux, consider pip install torch --index-url https://download.pytorch.org/whl/cu118.
• Quantization: pip install curated-transformers[quantization]
• Documentation: Overview, Usage, API

Highlighted Details

• Supports state-of-the-art LLMs including Falcon, Llama, and Dolly v2.
• Modular design allows easy integration of new architectures and benefits from shared component improvements.
• Offers 4/8-bit inference and PyTorch meta device support for efficiency.
• Production-tested by Explosion and slated for spaCy 3.7 integration.

Maintenance & Community

• Developed by Explosion, the creators of spaCy and other NLP libraries.
• No explicit community links (Discord/Slack) are provided in the README.

Licensing & Compatibility

• License: MIT.
• Compatible with commercial and closed-source applications.

Limitations & Caveats

The README does not detail specific limitations, unsupported platforms, or known bugs. The library is presented as production-tested, implying a degree of stability.