This project demonstrates using a fine-tuned Llama 7B model (LoRA) with Unreal Engine 5 documentation to create a specialized, locally hosted documentation assistant. It targets developers and researchers seeking alternatives to cloud-based LLM APIs and vector databases for niche, context-aware information retrieval.

How It Works

The project fine-tunes Meta's Llama 7B model using a LoRA adapter trained on Unreal Engine 5.1 documentation. This approach allows for efficient, local adaptation of a base LLM to a specific domain, enabling it to answer queries about UE5 features like Nanite and Mass Avoidance with higher accuracy than a general-purpose model.

Quick Start & Requirements

• Requires oobabooga's text generation webui to be set up.
• Base model: Llama-7b (8bit mode recommended).
• Dataset: unreal_docs.txt (provided in repo) placed in text-generation-webui/training/datasets.
• Training on a NVIDIA 3090ti took approximately 8 hours.
• See oobabooga's text generation webui for setup.

Highlighted Details

• Fine-tuned model shows significant improvement over base Llama 7B for UE5-specific queries.
• Outperforms ChatGPT on newer UE5 features like Mass Avoidance, which were released after ChatGPT's training cutoff.
• Includes a web scraping script for documentation, though its implementation is noted as inefficient.

Maintenance & Community

• Relies on oobabooga's text generation webui and Stanford's Alpaca research.
• Key contributor to the training tab in the webui is mcmonkey4eva.

Licensing & Compatibility

• Licensed under the MIT License.
• Permissive license suitable for commercial use and integration with closed-source projects.

Limitations & Caveats

The model is prone to hallucinations and may generate incorrect information. Output quality could be improved with a UE5-tailored character YAML file or by formatting the dataset as instruction/response pairs. The included web scraping script is inefficient.