This repository provides a Docker-based solution for running Ollama with Intel ARC GPU acceleration on Linux, targeting users with Intel ARC hardware who want to leverage local LLMs. It simplifies the setup process for using models like deepseek-r1 on Intel GPUs.

How It Works

The project utilizes Docker Compose to build a custom Ollama image with Intel Extension for PyTorch (IPEX-LLM) support, specifically leveraging the IPEX-LLM portable ZIP distribution. This approach allows Ollama to utilize Intel's optimized libraries for GPU inference, enabling local execution of large language models on compatible Intel hardware.

Quick Start & Requirements

• Install Docker and Docker Compose.
• Ensure Ubuntu 24.04 or newer is installed for optimal Intel ARC GPU driver support.
• Requires an Intel ARC series GPU (tested with A770 16GB and integrated Intel Core Ultra 5 125H).
• Clone the repository and run docker compose up.
• Access the web UI at http://localhost:3000.
• Official documentation: ipex-llm Quickstart

Highlighted Details

• Enables Ollama to run on Intel ARC GPUs via IPEX-LLM.
• Supports the deepseek-r1 model.
• Uses Docker Compose for simplified deployment.
• Allows selection of specific GPUs via ONEAPI_DEVICE_DELECTOR environment variable.

Maintenance & Community

• Defects related to IPEX-LLM should be reported to the intel/ipex-llm project.
• The project references IPEX-LLM releases, indicating reliance on its development.

Licensing & Compatibility

• The repository itself does not specify a license.
• Relies on Ollama and IPEX-LLM, which have their own licenses. Ollama is typically MIT licensed. IPEX-LLM is Apache 2.0 licensed.
• Compatibility for commercial use depends on the underlying licenses of Ollama and IPEX-LLM.

Limitations & Caveats

The project is described as an illustration and may not be production-ready. It is specifically tailored for Linux and Intel ARC GPUs, limiting its applicability to other operating systems or GPU vendors. Updates to IPEX-LLM require manual modification of the docker-compose.yml file.