This project provides a high-performance, native LLM inference server and macOS application specifically designed for Apple Silicon. It addresses the need for efficient, local LLM execution by offering OpenAI and Anthropic API compatibility without Python dependencies, targeting developers and power users seeking a streamlined, integrated experience on their Macs. The primary benefit is significantly faster inference speeds and advanced features like agent mode and multimedia generation, all within a native, non-Electron application.

How It Works

mlx-serve is built using the Zig programming language, leveraging Apple's MLX framework for native Metal acceleration on Apple Silicon. It also embeds llama.cpp to support the vast GGUF model ecosystem. This architecture avoids Python runtimes and Electron, resulting in a lean, fast binary. The server exposes familiar HTTP APIs (OpenAI, Anthropic, Ollama) allowing seamless integration with existing tools and workflows, while MLX Core provides a native macOS GUI for model management, chat, and agent functionalities.

Quick Start & Requirements

• Installation: Use Homebrew: brew tap ddalcu/mlx-serve, then brew install mlx-serve (CLI server) or brew install --cask mlx-core (GUI app).
• Prerequisites: macOS 26+ with Apple Silicon (M1/M2/M3/M4). Building from source requires Zig 0.16+ and mlx-c, libwebp.
• Resources: Models load on demand; RAM usage varies by model size and quantization.
• Links: ddalcu.github.io/mlx-serve, Download MLX Core.app, Changelog.

Highlighted Details

• API Compatibility: Supports OpenAI (chat/completions, embeddings, responses with WebSockets) and Anthropic Messages APIs. Also speaks the Ollama API for drop-in replacement.
• Model Support: Runs native MLX models and GGUF models via embedded llama.cpp. Includes support for DeepSeek V4 Flash via a dedicated engine.
• MLX Core App: Native macOS menu-bar app featuring model management, chat interface (with PDF/image support), agent mode with 10 built-in tools and sandboxed execution, and quick launcher.
• Performance: Claims up to +35% geomean speedup over LM Studio on identical weights, with speculative decoding further boosting performance on specific workloads.
• Multimedia Generation: Integrates native MLX models for image (FLUX.2), video (LTX-Video), and audio (Qwen3-TTS) generation.

Maintenance & Community

The project is actively maintained via its GitHub repository. No specific community channels (Discord, Slack) are listed, but the project encourages GitHub stars as a measure of utility.

Licensing & Compatibility

Distributed under the MIT License, permitting commercial use and integration into closed-source projects.

Limitations & Caveats

The software is strictly limited to macOS with Apple Silicon hardware. While it supports a wide range of models, performance and compatibility may vary depending on the specific model architecture and quantization. Building from source requires a development environment with Zig.