This project provides a streamlined Docker Compose setup for deploying Ollama, a tool for running large language models locally. It targets developers and users who want an easy, containerized way to access and experiment with LLMs, offering GPU acceleration and integration with tools like Langchain.

How It Works

The setup utilizes Docker Compose to orchestrate multiple containers, including Ollama itself and a web UI. For GPU acceleration, it integrates with the NVIDIA Container Toolkit, allowing Ollama to leverage host GPU resources efficiently. An additional app container is provided for Langchain examples and as a devcontainer for development.

Quick Start & Requirements

• Install: git clone https://github.com/mythrantic/ollama-docker.git && cd ollama-docker
• Prerequisites: Docker, Docker Compose. Optional: NVIDIA Container Toolkit for GPU support.
• Usage:
  • CPU: docker compose up -d
  • GPU: docker compose -f docker-compose-ollama-gpu.yaml up -d
• Access: http://localhost:8080
• Docs: NVIDIA Container Toolkit

Highlighted Details

• Simplified deployment of Ollama and web UI via Docker Compose.
• Optional NVIDIA GPU acceleration with provided setup instructions.
• Includes an app container for Langchain examples and devcontainer functionality.
• run.sh script for setting up a local virtual environment as an alternative to Docker.

Maintenance & Community

• Contributions are welcomed via contribution guidelines.
• Contact: vantlynxz@gmail.com

Licensing & Compatibility

• License: RSOSL (Requires mention and credit).
• Compatible with commercial use, provided attribution.

Limitations & Caveats

The project is primarily focused on Docker deployment; alternative development environments are supported via a run.sh script but may require manual setup.