LLM.swift is a Swift library designed for easy local interaction with large language models across all Apple platforms. It targets developers building AI-powered applications on macOS, iOS, watchOS, tvOS, and visionOS, offering a simplified interface to models compatible with llama.cpp.

How It Works

LLM.swift acts as a lightweight abstraction layer over the llama.cpp project. This design choice ensures high performance and compatibility with any model supported by llama.cpp. It provides a flexible LLM class that can be subclassed, allowing customization of input preprocessing, output postprocessing, and real-time output updates via closures. Models can be loaded directly from bundled GGUF files or downloaded from Hugging Face.

Quick Start & Requirements

• Install via Swift Package Manager: .package(url: "https://github.com/eastriverlee/LLM.swift/", branch: "main")
• Requires Swift 5.5+ and llama.cpp compatibility.
• Model files (GGUF format) are required.
• See LLMTests.swift for usage examples.

Highlighted Details

• Supports macOS, iOS, watchOS, tvOS, and visionOS.
• Enables loading models from local bundles or Hugging Face.
• Provides customizable preprocess, postprocess, and update callbacks for fine-grained control.
• Offers built-in support for common chat templates like ChatML.
• Performance is tied to llama.cpp capabilities.

Maintenance & Community

• The project is maintained by eastriverlee.
• Source code is available on GitHub: https://github.com/eastriverlee/LLM.swift

Licensing & Compatibility

• The library is distributed under the MIT License.
• Compatible with commercial and closed-source applications.

Limitations & Caveats

The README notes that larger models (e.g., 7B parameters) may require significant memory and computational resources, especially on mobile devices, recommending smaller models (3B or less) for better performance. Model compatibility is dependent on llama.cpp support.