This repository provides the Apple MLX engine for LM Studio, enabling efficient large language model (LLM) inference on Apple Silicon hardware. It targets Mac users of LM Studio, offering a pre-bundled, optimized solution for running various LLMs, including text and vision models, with features like speculative decoding.

How It Works

The engine leverages Apple's MLX framework, a high-performance array library designed for machine learning on Apple hardware. It integrates with mlx-lm for text generation and mlx-vlm for vision-language tasks, facilitating direct inference without complex setup. The architecture supports features like speculative decoding to accelerate inference speeds by using a smaller, faster draft model to predict tokens.

Quick Start & Requirements

• Install: LM Studio 0.3.4+ includes mlx-engine pre-bundled. For standalone use, clone the repo and install dependencies: pip install -U -r requirements.txt.
• Prerequisites: macOS 14.0 (Sonoma) or greater, Python 3.11.
• Demo: Run python demo.py --model <model_name> for text models or add --images for vision models. Download models using the lms CLI tool.
• Docs: LM Studio CLI Documentation

Highlighted Details

• Supports text and vision LLMs (Llama-3.2-Vision, Pixtral, Qwen2-VL, Llava-v1.6).
• Implements speculative decoding for accelerated inference.
• Includes pre-commit hooks for code quality and pytest for testing.

Maintenance & Community

The project is maintained by lmstudio-ai. Community support and updates are typically channeled through LM Studio's official platforms.

Licensing & Compatibility

• mlx-engine itself is not explicitly licensed in the README, but its core components mlx-lm, mlx-vlm are MIT licensed, and Outlines is Apache 2.0.
• Compatible with LM Studio for Mac.

Limitations & Caveats

The engine is exclusively for macOS with Apple Silicon and requires a specific Python version (3.11). The standalone usage requires manual installation of dependencies.