xTuring is an open-source library designed for the efficient and accessible fine-tuning of large language models (LLMs). It empowers users, from researchers to developers, to personalize LLMs like LLaMA, Mistral, and GPT-J with their own data, ensuring data privacy by enabling local or private cloud execution.

How It Works

xTuring employs memory-efficient fine-tuning techniques such as LoRA (Low-Rank Adaptation) and quantization (INT4, INT8) to significantly reduce hardware requirements and costs, potentially by up to 90%. This approach allows for faster training epochs and enables fine-tuning on less powerful hardware, making LLM customization more accessible. The library also supports scaling across multiple GPUs for accelerated training and includes features for data preprocessing, model evaluation with metrics like perplexity, and inference on both GPU and CPU.

Quick Start & Requirements

• Install: pip install xturing
• Prerequisites: Python, Hugging Face models. GPU recommended for fine-tuning.
• Demo: https://github.com/stochasticai/xTuring/blob/main/examples/models/llama/alpaca_data
• Docs: Tutorials available in README.

Highlighted Details

• Supports LLaMA 2, Mistral, Falcon-7B, GPT-J, GPT-2, OPT, Cerebras-GPT, Galactica, and Bloom models.
• Offers INT4, INT8, and LoRA fine-tuning for reduced memory footprint and cost.
• Includes CPU inference capabilities leveraging Intel® Extension for Transformers.
• Provides a UI playground for interactive fine-tuning and experimentation.

Maintenance & Community

• Active development with recent updates including LLaMA 2 integration, evaluation features, and INT4 precision.
• Community support via Discord: https://discord.gg/TgHXuSJEk6

Licensing & Compatibility

• Licensed under Apache License 2.0.
• Permissive license suitable for commercial use and integration with closed-source applications.

Limitations & Caveats

The roadmap indicates future support for INT3, INT2, and INT1 low-precision fine-tuning, suggesting these are not yet implemented. Stable Diffusion support is also listed as a future item.