Kolo streamlines LLM fine-tuning by automating environment setup and providing a unified interface for popular tools like Unsloth, Torchtune, Llama.cpp, and Ollama. It targets AI researchers and developers seeking a rapid, hassle-free local fine-tuning experience, reducing setup time to minutes.

How It Works

Kolo leverages Docker to create a consistent, pre-configured environment, eliminating dependency conflicts. It integrates Unsloth for faster training and lower VRAM usage, Torchtune for PyTorch-native fine-tuning (including AMD GPU and CPU support), and Llama.cpp for GGUF conversion and quantization. Ollama manages model deployment, and Open WebUI provides a testing interface. This stacked approach offers flexibility and performance for local LLM experimentation.

Quick Start & Requirements

• Install: Requires Docker Desktop (Windows/Linux) and WSL2/HyperV on Windows. AMD GPU users need ROCm installed on Linux.
• Build: ./build_image.ps1 (or ./build_image_amd.ps1 for AMD).
• Run: ./create_and_run_container.ps1 (or ./create_and_run_container_amd.ps1 for AMD).
• Data: ./copy_training_data.ps1
• Train: ./train_model_unsloth.ps1 or ./train_model_torchtune.ps1
• Install Model: ./install_model.ps1
• Test: Access localhost:8080 in a browser.
• Docs: Fine Tune Training Guide (linked within README).

Highlighted Details

• Supports both NVIDIA (CUDA 12.1+) and AMD GPUs (Linux only via ROCm).
• Offers fine-tuning via Unsloth (faster, lower VRAM) and Torchtune (PyTorch native).
• Includes scripts for model installation, uninstallation, and listing.
• Provides SSH and SFTP access to the Docker container for advanced users.

Maintenance & Community

• Community support via Discord (link provided in README).

Licensing & Compatibility

• The README does not explicitly state the project's license. Compatibility for commercial use or closed-source linking is not specified.

Limitations & Caveats

• AMD GPU support is restricted to Linux; Windows WSL2 is not supported.
• Torchtune requires Hugging Face account and model access permissions.
• Re-training with the same output directory requires manual deletion via ./delete_model.ps1.