This repository provides a solution for running Hugging Face Transformers models, specifically DistilBERT and GPT-2, on Android devices using TensorFlow Lite. It targets mobile developers and researchers seeking efficient on-device natural language processing capabilities, enabling features like question answering and text generation without relying on cloud infrastructure.

How It Works

The project converts pre-trained models from the Hugging Face Transformers library into the TensorFlow Lite format. This conversion process allows for optimized inference on mobile hardware. The repository includes example scripts for model generation and provides demo applications for Android to showcase the functionality of DistilBERT for question answering and GPT-2/DistilGPT2 for text generation.

Highlighted Details

• On-device Question Answering with DistilBERT (97% of BERT's GLUE performance).
• On-device Text Generation using GPT-2 or DistilGPT2 (2x faster, 33% smaller than GPT-2).
• Includes Android demo applications for immediate testing.
• Model conversion scripts are provided.

Maintenance & Community

This project is part of the Hugging Face ecosystem. Further community engagement and support can typically be found through Hugging Face's official channels.

Licensing & Compatibility

Licensed under the Apache License 2.0. This license is permissive and generally allows for commercial use and integration into closed-source applications.

Limitations & Caveats

The model conversion scripts require a nightly version of TensorFlow, which may be unstable.