Summary

This project provides pre-built containerized environments ("toolboxes") for running Large Language Models (LLMs) on AMD Ryzen AI Max “Strix Halo” integrated GPUs. It targets engineers and power users seeking a reproducible, flexible way to leverage AMD hardware for LLM inference using Llama.cpp across various compute backends.

How It Works

The project uses Toolbx containers for isolated LLM inference, powered by Llama.cpp. It supports multiple AMD backends: Vulkan (RADV, AMDVLK) and ROCm. This offers flexibility in choosing between stability, performance, or newer ROCm features, ensuring seamless integration and host system cleanliness. Containers automatically update with Llama.cpp changes.

Quick Start & Requirements

• Installation: Create toolboxes with toolbox create (e.g., `docker.io/kyuz0