This Swift library provides a framework for loading and running large language models (LLMs) like Llama on macOS and iOS. It's designed for developers and researchers interested in on-device LLM inference, leveraging Metal for accelerated computation on Apple Silicon.

How It Works

The library is built upon ggml and llama.cpp, enabling efficient LLM execution. It supports various inference and sampling methods, including temperature, top-k, top-p, Tail Free Sampling (TFS), Locally Typical Sampling, Mirostat, and greedy decoding. The architecture is optimized for Apple's Metal framework for GPU acceleration, specifically targeting Apple Silicon hardware.

Quick Start & Requirements

• Install via Swift Package Manager: https://github.com/guinmoon/llmfarm_core.swift
• Requirements: macOS 13+, iOS 16+. Metal support requires Apple Silicon (Intel Macs are not supported for Metal).
• See examples in the Demo Project.

Highlighted Details

• Supports macOS (13+) and iOS (16+).
• Leverages Metal for GPU acceleration on Apple Silicon.
• Implements multiple sampling methods (Temperature, TFS, Mirostat, etc.).
• Based on ggml and llama.cpp.

Maintenance & Community

The project is under active revision and refactoring, with the author learning Swift during development. Feedback on code style and architecture is welcomed.

Licensing & Compatibility

The license is not explicitly stated in the README. Compatibility for commercial use or closed-source linking is not specified.

Limitations & Caveats

The code is in constant revision and may not be stable. Support for LoRA adapters (training, export, and context restoration) is currently missing. Metal acceleration is not functional on Intel Macs.