This project provides Docker images for oobabooga/text-generation-webui, simplifying deployment for users of various hardware configurations including Nvidia, AMD (unstable), Intel Arc (unstable), and CPU-only. It aims to be a Dockerized equivalent to the stable-diffusion-webui-docker project, offering pre-built images for easy setup and a template for custom configurations.

How It Works

The project leverages Docker Compose to manage the deployment of text-generation-webui. It offers pre-built images tagged by variant (e.g., default, base) and platform (e.g., nvidia, cpu, rocm, arc), allowing users to select the appropriate image for their hardware. Configuration is managed via docker-compose.yml and a structured config folder, with options to map volumes for persistent data and define extra launch arguments.

Quick Start & Requirements

• Install/Run: git clone the repository, navigate into the directory, and run docker compose up.
• Prerequisites: Docker, Docker Compose, and optionally NVIDIA CUDA Docker runtime.
• Setup: Minimal setup time, primarily dependent on Docker installation and image download.
• Links: Docker Hub: https://hub.docker.com/r/atinoda/text-generation-webui

Highlighted Details

• Offers variants for Nvidia (CUDA 12.1), CPU, ROCm 5.6 (unstable), and Intel Arc (unstable).
• Includes default (all extensions) and base (no extensions) variants.
• Supports TensorRT-LLM acceleration for Nvidia GPUs.
• Allows customization via EXTRA_LAUNCH_ARGS and runtime extension builds.

Maintenance & Community

Contributions are welcomed, with a focus on improving AMD/ROCm and Windows support. The project is actively seeking user reports and insights for unstable variants.

Licensing & Compatibility

The project's license is not explicitly stated in the README. Compatibility for commercial use or closed-source linking is not specified.

Limitations & Caveats

The ROCm and Intel Arc variants are experimental, unstable, and not regularly tested due to hardware limitations. Several extensions (multimodal, ngrok, silero_tts, superbooga/superboogav2) have known issues or crashes. Apple Silicon is not supported due to virtualization limitations.