This project demonstrates edge inference of a Transformer NLP model directly within a web browser, targeting developers and researchers interested in on-device machine learning. It leverages Cloudflare Pages for efficient global delivery of a React application and an ONNX Runtime Web-based inference engine, enabling real-time emotion prediction from text without server-side processing.

How It Works

The application utilizes ONNX Runtime Web to execute a fine-tuned Hugging Face Transformer model (microsoft/xtremedistil-l6-h384-uncased, fine-tuned on GoEmotions dataset) within the browser. This approach minimizes latency and enhances privacy by keeping data local. The model is quantized to int8 for reduced size and faster inference. Multi-threading is employed for performance, though it was disabled for iOS compatibility due to a known bug.

Quick Start & Requirements

• Install Node.js/npm.
• Run npm start for development.
• Deploy via Cloudflare Pages.
• Requires specific HTTP headers (COOP/COEP) for multi-threaded inference, configured via public/_header or src/setupProxy.js.

Highlighted Details

• Demonstrates edge inference of a Transformer NLP model in the browser.
• Uses ONNX Runtime Web for efficient, client-side model execution.
• Leverages Cloudflare Pages for global CDN delivery.
• Model fine-tuned on the GoEmotions dataset for 27 emotion categories.

Maintenance & Community

• Open to PRs and feedback via GitHub issues.
• Blog post and YouTube video available for deeper dives.

Licensing & Compatibility

• The tokenizer is a copy of Google TFJS (Apache 2.0).
• Other licensing details are not explicitly stated in the README.

Limitations & Caveats

Multi-threading inference is disabled on iOS due to a bug. The project acknowledges potential biases in the pre-trained language model and the GoEmotions dataset, including representational limitations and potentially problematic content.