This project provides a native SwiftUI application for running local Large Language Models (LLMs) on Apple Silicon devices, leveraging the MLX framework. It targets developers and power users seeking a user-friendly interface to experiment with LLMs offline, offering real-time inference and a straightforward model management system.

How It Works

The application utilizes Apple's MLX framework, a high-performance machine learning library optimized for Apple Silicon. It provides a native SwiftUI frontend for seamless interaction with MLX-powered LLMs, enabling local execution of models like Llama and Mistral. The architecture focuses on on-device processing for privacy and real-time performance.

Quick Start & Requirements

• Installation: Open the Xcode project, select your signing team, set the destination to "My Mac," and click "Run."
• Prerequisites: Apple Silicon Mac (M1/M2/M3) with macOS 14.0+ or A-Series chip (iPad, iPhone) with iOS 17.2+.
• Model Download: Models are downloaded via the "Manage Models" feature within the app. Recommended starting model: Nous-Hermes-2-Mistral-7B-DPO-4bit-MLX.
• Documentation: MLX Swift Chat GitHub Repository

Highlighted Details

• Supports Mistral, Llama, Phi, and Gemma models (Gemma may have issues).
• Enables local LLM execution on Apple Silicon for real-time inference.
• User-friendly SwiftUI interface with model management and download capabilities.
• No data collection; all processing is done on-device.

Maintenance & Community

The project is under active development. Support for iOS is planned. The project acknowledges contributions from buh/CompactSlider, ml-explore/mlx-swift, and huggingface/swift-chat.

Licensing & Compatibility

The project's license is not explicitly stated in the README. Compatibility for commercial use or closed-source linking is not specified.

Limitations & Caveats

This project is not intended for production deployment. Some models may require additional implementation to function correctly. iOS support is listed as a future enhancement.