Oosaurus is a native, Apple Silicon-only local LLM server designed to maximize performance on M-series Macs. It offers an OpenAI-compatible API, enabling seamless integration with existing tools and workflows, and provides a user-friendly SwiftUI interface for managing models and monitoring system resources.

How It Works

Oosaurus leverages Apple's MLX framework for optimized performance on Apple Silicon, utilizing MLXLLM for efficient LLM execution. It features a SwiftNIO server for handling requests and Server-Sent Events for low-latency token streaming. Key architectural choices include session reuse via KV cache for faster multi-turn conversations and automatic handling of chat templates from model configurations for accurate prompt formatting.

Quick Start & Requirements

• Install: Download the latest signed build from the Releases page.
• Requirements: macOS 15.5+, Apple Silicon (M1 or newer), Xcode 16.4+ (for building from source).
• Setup: Build and run the Oosaurus target in Xcode. Configure port in the UI (default 8080). Download models via the Model Manager.
• Docs: API Endpoints, Getting Started, Python examples.

Highlighted Details

• OpenAI-compatible API endpoints for /v1/models and /v1/chat/completions (streaming and non-streaming).
• Supports OpenAI-style function/tool calling with robust parsing and streaming deltas.
• Native MLX runtime with optimized session reuse (KV cache) for faster multi-turn chats.
• Model manager UI to browse, download, and manage MLX models from mlx-community.

Maintenance & Community

• Created by Dinoki Labs (dinoki.ai).
• Contributors: wizardeur (first PR creator).
• Community: Discord server available. Contributing Guide and Code of Conduct provided.
• Support channels and security policy are outlined.

Licensing & Compatibility

• License details are not explicitly stated in the README.
• Designed for Apple Silicon only; Intel Macs are not supported. Localhost only, no authentication.

Limitations & Caveats

The project is Apple Silicon only and does not support Intel Macs. /transcribe endpoints are placeholders pending Whisper integration. The README does not specify the license type, which may impact commercial use or closed-source linking.