This repository provides Swift implementations of popular transformer models (GPT-2, BERT, DistilBERT) for on-device inference using Core ML 3. It targets iOS and macOS developers looking to integrate advanced natural language processing capabilities into their applications without relying on cloud services.

How It Works

The project converts PyTorch-trained models from the Hugging Face transformers library into the Core ML 3 format. It includes Swift implementations of tokenizers (WordPiece, Byte-Pair Encoding) and necessary utilities for tasks like question answering (SQuAD dataset) and text generation. The core advantage is enabling efficient, offline execution of these powerful NLP models on Apple hardware.

Quick Start & Requirements

• Requires git-lfs for downloading model files.
• Installation instructions are not detailed, but the project structure suggests using the provided conversion scripts and integrating the Core ML models into Xcode projects.
• Links to WWDC 2019 demos and Apple's Core ML models page are provided.

Highlighted Details

• Core ML 3 implementations of GPT-2, DistilGPT-2, BERT, and DistilBERT.
• Includes Swift tokenizers and SQuAD dataset utilities.
• Demo apps for on-device question answering and text generation.
• Models were demonstrated at WWDC 2019.

Maintenance & Community

This repository is not actively maintained and has been archived. Users are directed to swift-transformers for an in-development replacement.

Licensing & Compatibility

The license is not explicitly stated in the README. Given the association with Hugging Face and Apple, it's likely permissive, but users should verify for commercial use.

Limitations & Caveats

The project is archived and no longer actively maintained, indicating potential issues with compatibility with newer Swift or Core ML versions. The README explicitly states it is not actively maintained.