LLMFarm is an iOS and macOS application enabling offline execution of various large language models (LLMs) and multimodal models. It targets developers and power users on Apple platforms seeking to test and deploy LLMs locally, leveraging the GGML library for efficient inference.

How It Works

LLMFarm utilizes the GGML library, a C library for machine learning, to run LLMs efficiently on Apple hardware. It supports Metal for GPU acceleration on Apple Silicon Macs, offering faster inference. The application provides a flexible interface for loading diverse model architectures and configuring various sampling methods for fine-tuned output generation.

Quick Start & Requirements

• Install via git clone --recurse-submodules https://github.com/guinmoon/LLMFarm.
• Requires macOS 13+ or iOS 16+.
• Metal acceleration is available on Apple Silicon Macs (Intel Macs are not supported for Metal).
• See FAQ for more details.

Highlighted Details

• Supports a wide range of LLM architectures including LLaMA, Gemma, Phi, Falcon, Mistral, Mixtral, Mamba, RWKV, and more.
• Includes multimodal capabilities with support for LLaVA, BakLLaVA, Moondream, and other vision-language models.
• Offers advanced sampling methods like Temperature, TFS, Mirostat, and Grammar-based sampling.
• Features include Apple Shortcuts integration and context state restoration.

Maintenance & Community

• The core inference engine (llmfarm_core) has been moved to a separate repository.
• Project relies on and integrates code from rwkv.cpp, Mia, LlamaChat, swift-markdown-ui, and similarity-search-kit.

Licensing & Compatibility

• The README does not explicitly state a license. Compatibility for commercial use or closed-source linking is not specified.

Limitations & Caveats

• Metal GPU acceleration is not supported on Intel Macs.
• The project's licensing status requires clarification for commercial adoption.