This project provides a C++ implementation for running MiniGPT-4, a multimodal large language model, with CPU inference capabilities using GGML. It targets developers and researchers seeking efficient, quantized execution of MiniGPT-4 on standard hardware without requiring GPUs.

How It Works

The core of minigpt4.cpp is its integration with the GGML library, enabling 4-bit, 5-bit, 6-bit, 8-bit, and 16-bit quantization of the MiniGPT-4 model. This approach significantly reduces memory footprint and computational requirements, allowing for inference on CPUs. The project facilitates model conversion from PyTorch to the GGML format, making pre-trained models accessible for local execution.

Quick Start & Requirements

• Install: Clone the repository with git clone --recursive https://github.com/Maknee/minigpt4.cpp.
• Build Library: Use CMake. For Linux: cmake . && cmake --build . --config Release. For Windows: cmake . && cmake --build . --config Release. Optional OpenCV support can be enabled via CMake.
• Models: Download pre-quantized models from Hugging Face or convert PyTorch checkpoints using provided Python scripts. Vicuna models are also required.
• Run: Execute inference via Python scripts (e.g., python minigpt4_library.py ...) or launch a web UI (python webui.py ...).
• Dependencies: CMake, C++ compiler, Python 3.x, PyTorch (for conversion), and optionally OpenCV.

Highlighted Details

• CPU-only inference for MiniGPT-4.
• Supports 4-bit, 5-bit, 6-bit, 8-bit, and 16-bit quantization via GGML.
• Includes scripts for converting PyTorch models to GGML format.
• Offers a web UI for interactive use.

Maintenance & Community

Information on maintainers, community channels, or roadmaps is not detailed in the provided README.

Licensing & Compatibility

The README does not explicitly state the license for minigpt4.cpp. However, it relies on GGML and potentially other libraries, whose licenses would apply. Compatibility for commercial use or closed-source linking is not specified.

Limitations & Caveats

The project appears to be a direct port, and performance characteristics compared to GPU-accelerated versions are not benchmarked. Obtaining and converting models requires familiarity with PyTorch and the original MiniGPT-4 repository setup.