This repository provides the OLMoE.swift application, an offline-capable, privacy-focused AI experience for iOS devices. It allows users to interact with large language models directly on their device without data storage or internet connectivity requirements.

How It Works

The project leverages on-device inference for its AI capabilities. While the README doesn't detail the specific model architecture or inference engine used within the Swift application, it does provide Python code demonstrating how to load and run OLMoE models using the Hugging Face transformers library with PyTorch. This suggests a modular approach where the core AI logic can be accessed via standard Python libraries, with the Swift app acting as a user-friendly interface.

Quick Start & Requirements

• iOS App: Clone the repository, open the project in Xcode, set the target device (e.g., iPhone 15 Pro or higher), and run the project.
• Python/Hugging Face: Install transformers (>= 4.45.0) and torch. Run the provided Python code snippet to load and interact with models.
• Dependencies: CUDA is recommended for GPU acceleration if available.

Highlighted Details

• Fully offline and private AI experience.
• Runs directly on the device, no internet required.
• Supports model downloads within the app.
• Python examples provided for using OLMoE models with Hugging Face.

Maintenance & Community

The project is from Allen AI. Further community or maintenance details are not specified in the README.

Licensing & Compatibility

The project is open source, with the core application and its dependencies (LlamaCPP, ggml, MarkdownUI, HighlightSwift) licensed under the MIT License. This license permits commercial use and integration into closed-source projects.

Limitations & Caveats

The README specifies that the iOS app requires a target device of iPhone 15 Pro or higher, indicating potential hardware limitations for older devices. The exact model sizes and performance characteristics on mobile hardware are not detailed.