Telosnex/fonnx provides a Flutter library for running ONNX machine learning models natively across multiple platforms, including mobile, desktop, and web. It targets Flutter developers seeking to integrate AI capabilities like text embeddings and voice processing directly into their applications, offering benefits of privacy and low latency by enabling on-device inference.

How It Works

FONNX leverages the ONNX Runtime, integrating with platform-specific acceleration frameworks such as CoreML on iOS, Android Neural Networks API, and WASM SIMD on the web. This approach allows for efficient, native execution of ONNX models, which can be converted from popular frameworks like PyTorch and TensorFlow using tools like Hugging Face's optimum-cli.

Quick Start & Requirements

• Install: Add fonnx to your Flutter project's pubspec.yaml.
• Prerequisites: ONNX Runtime dependencies are handled automatically for most platforms. Web deployment requires specific server headers (Cross-Origin-Embedder-Policy: require-corp, Cross-Origin-Opener-Policy: same-origin) for optimal performance.
• Resources: Example app available at telosnex.github.io/fonnx/.

Highlighted Details

• Supports ONNX models for text embeddings (e.g., MiniLM L6 V2, MSMARCO MiniLM L6 V3) and voice processing (Whisper, Silero VAD).
• Achieves low latency inference, with benchmarks showing ~13ms on macOS (M2 Max) and ~41ms on Web WASM SIMD for a MiniLM L6 V2 embedding.
• Integrates via FFI for macOS/Windows/Linux, ONNX Objective-C library for iOS, and ONNX Android AAR for Android.

Maintenance & Community

• Recent updates include CI builds on Codemagic and support for Magika and Whisper across platforms.
• Community links are not explicitly provided in the README.

Licensing & Compatibility

• Dual-licensed: GPL v2 for the code on GitHub (requiring source code distribution for derivative works) and commercial licenses available upon contact. Commercial use is possible with a paid license, with terms favoring entities with successful, FONNX-dependent applications.

Limitations & Caveats

• The GPL v2 license may be restrictive for commercial applications without a separate commercial license.
• Web development requires specific server configurations and workarounds for WASM MIME types and SharedArrayBuffer usage in debug mode.