Summary

This project addresses the need for local, private AI processing on macOS by exposing Apple's Foundation Models and other MLX models through a unified, OpenAI-compatible API. It targets AI enthusiasts and developers seeking to leverage on-device LLMs without Python or cloud dependencies, offering significant privacy and performance benefits.

How It Works

Built entirely in Swift, the project leverages Metal for maximum GPU acceleration on Apple Silicon. It provides an OpenAI-compatible API endpoint that can serve Apple's on-device Foundation Models or any Hugging Face MLX model. The system also functions as an API gateway, aggregating requests for other local LLM backends like Ollama and LM Studio, and includes Vision OCR capabilities for image and PDF text extraction.

Quick Start & Requirements

• Primary Install: Homebrew (brew install scouzi1966/afm/afm) is recommended. Pip (pip install macafm) and building from source are also supported.
• Prerequisites: macOS 26 (Tahoe) or later, Apple Silicon Mac (M1/M2/M3/M4 series), Apple Intelligence enabled in System Settings, Xcode 16 (for source builds).
• Links: No direct documentation or demo links are provided within the README, beyond the GitHub repository itself.

Highlighted Details

• OpenAI API Compatibility: Seamless integration with existing OpenAI client libraries and applications.
• MLX Local Models: Supports running over 28 tested open-source MLX models from Hugging Face locally.
• API Gateway Mode: Auto-discovers and proxies requests to other local LLM backends (Ollama, LM Studio, Jan).
• Apple Foundation Models: Direct support for Apple's 3B parameter on-device language model.
• Vision OCR: Enables text extraction from images and PDFs via afm vision.
• Built-in WebUI: Provides a chat interface for easy testing and model interaction.
• Privacy-First: All AI processing occurs entirely on the user's device.

Maintenance & Community

The project welcomes contributions via pull requests. While no dedicated community channels like Discord or Slack are listed, the GitHub repository serves as the primary hub for issues and development discussions. Related projects like Vesta AI Explorer and AFMTrainer are also mentioned.

Licensing & Compatibility

The project is licensed under the permissive MIT License, allowing for broad compatibility, including commercial use and linking within closed-source applications.

Limitations & Caveats

The Apple Foundation Model is a 3B parameter model optimized for on-device performance. The software requires macOS 26 or later and Apple Intelligence to be enabled. Token counting for the Foundation model uses a word-based approximation, unlike proxied backends which report accurate counts. The nightly build is currently synchronized with the stable release.