fast-bert  by appvision-ai

SDK for BERT/XLNet-based NLP model training and deployment

created 6 years ago
1,899 stars

Top 23.4% on sourcepulse

GitHubView on GitHub
Project Summary

This library provides a simplified interface for fine-tuning and deploying BERT, RoBERTa, and XLNet models for text classification and language model tasks. It targets NLP practitioners and data scientists looking to leverage state-of-the-art transformer models with reduced boilerplate code, offering features like learning rate finding and FP16 training for efficiency.

How It Works

Fast-BERT abstracts the complexities of Hugging Face's Transformers library by providing BertDataBunch and BertLearner classes. BertDataBunch handles data loading, preprocessing, and tokenization, supporting both multi-class and multi-label classification. BertLearner encapsulates the training loop, optimization (including LAMB optimizer), metric calculation, and model saving/loading. It also includes a learning rate finder and supports FP16 training for faster execution.

Quick Start & Requirements

  • Install via pip: pip install fast-bert
  • Requires PyTorch and NVIDIA Apex.
  • Tested on Python 3.6+.
  • Official documentation and usage examples are available in the README.

Highlighted Details

  • Supports BERT, RoBERTa, XLNet, and DistilBERT models.
  • Includes a learning rate finder and LAMB optimizer for faster training.
  • Offers FP16 training for improved performance.
  • Provides functionality for both text classification (multi-class/multi-label) and language model fine-tuning.
  • Includes support for AWS SageMaker deployment.

Maintenance & Community

The project is actively maintained by kaushaltrivedi. Further community engagement details are not explicitly provided in the README.

Licensing & Compatibility

The library is released under the MIT License, allowing for commercial use and integration with closed-source projects.

Limitations & Caveats

The README mentions that the library is tested on Python 3.6+, implying potential compatibility issues with newer Python versions. Installation requires NVIDIA Apex, which can add complexity to the setup process.

Health Check
Last commit

11 months ago

Responsiveness

Inactive

Pull Requests (30d)
0
Issues (30d)
0
Star History
10 stars in the last 90 days

Explore Similar Projects

Feedback? Help us improve.