This project provides a Docker-based solution to leverage Intel Arc GPUs on Linux for running AI workloads, including Large Language Models (LLMs) via Ollama, image generation with Stable Diffusion (SD.Next, ComfyUI), and automatic speech recognition with OpenAI Whisper. It targets users with Intel Arc hardware seeking to utilize their GPUs for these AI tasks through a streamlined, integrated setup, offering a unified interface via Open WebUI.

How It Works

The solution orchestrates multiple Docker containers, with a core focus on integrating Intel® Extension for PyTorch (IPEX) and IPEX-LLM to optimize performance on Intel Arc Series GPUs. It uses podman compose for deployment, enabling services like Ollama, Open WebUI, Stable Diffusion interfaces (ComfyUI, SD.Next), and Whisper. The setup prioritizes utilizing the xpu device, directing computation to the Intel Arc GPU via SYCL, which is a key advantage for users with this specific hardware.

Quick Start & Requirements

• Primary install / run command: Clone the repository (git clone https://github.com/eleiton/ollama-intel-arc.git), cd into the directory, and run podman compose up. Separate docker-compose.yml files are provided for ComfyUI (docker-compose.comfyui.yml), SD.Next (docker-compose.sdnext.yml), and Whisper (docker-compose.whisper.yml).
• Non-default prerequisites: Linux operating system, Intel Arc Series GPU, Podman.
• Links:
  • Repository: https://github.com/eleiton/ollama-intel-arc.git
  • Open WebUI: https://github.com/open-webui/open-webui
  • Intel® Extension for PyTorch: https://github.com/intel/intel-extension-for-pytorch

Highlighted Details

• Optimized for Intel Arc Series GPUs on Linux using Intel® Extension for PyTorch (IPEX).
• Integrates Ollama, Open WebUI, Stable Diffusion (ComfyUI, SD.Next), and OpenAI Whisper.
• Leverages IPEX-LLM for optimized LLM inference on Intel Arc GPUs.
• Exposes Ollama on port 11434 and Stable Diffusion UIs on ports 7860/4040.
• Supports automatic speech recognition and translation via Whisper with --device xpu.

Maintenance & Community

The README does not provide specific details on maintainers, community channels (like Discord/Slack), or a roadmap. It references official Intel documentation and GitHub repositories for underlying technologies.

Licensing & Compatibility

The README does not explicitly state a license for the ollama-intel-arc repository itself. It relies on official Docker images and underlying technologies, which may have their own licenses. Compatibility is explicitly for Linux systems with Intel Arc GPUs.

Limitations & Caveats

The solution is specifically designed for Linux environments and Intel Arc Series GPUs. It prioritizes cutting-edge features over stability, as stated in the README. Authentication for Open WebUI is turned off by default. No explicit license is provided for the wrapper project itself.