SiLLM is a toolkit designed to simplify the training and inference of Large Language Models (LLMs) on Apple Silicon hardware, leveraging the MLX framework. It caters to researchers and developers looking to experiment with LLMs locally on Macs, offering features for model loading, fine-tuning (LoRA, DPO), and deployment.

How It Works

SiLLM builds upon MLX, providing a streamlined interface for common LLM operations. It supports loading models from various formats (Huggingface, Torch, GGUF, MLX) and includes implementations for LoRA and DPO fine-tuning. The toolkit also offers experimental features like speculative decoding and beam search, aiming to make advanced LLM techniques accessible on Apple Silicon.

Quick Start & Requirements

• Install via pip: pip install sillm-mlx
• Requires Apple Silicon hardware.
• Clone repo for web app: git clone https://github.com/armbues/SiLLM.git
• Web app setup: cd SiLLM/app && pip install -r requirements.txt && python -m chainlit run app.py -w
• Official examples: SiLLM-examples

Highlighted Details

• Supports major model architectures: Llama 2/3, Mistral, Mixtral, Gemma, Phi, Qwen.
• Offers an OpenAI-compatible API server for deployment.
• Includes a Chainlit-based web app for a seamless chat experience.
• Provides tools for model conversion and quantization.

Maintenance & Community

The project is actively maintained by armbues and acknowledges contributions from the MLX community.

Licensing & Compatibility

Licensed under the MIT License, permitting commercial use and integration with closed-source projects.

Limitations & Caveats

The project includes experimental features, which may be subject to change or instability. Specific performance benchmarks are not detailed in the README.