Vigogne provides a suite of open-source French instruction-following and chat Large Language Models (LLMs), built upon foundational models like LLaMA, Llama-2, and Falcon. It aims to offer powerful, French-centric conversational AI capabilities for researchers and developers working with French language data.

How It Works

Vigogne models are fine-tuned using a combination of translated instruction datasets, self-instruct methods, and ChatGPT distillation techniques. The project emphasizes efficient training, incorporating methods like LoRA and QLoRA, and offers support for advanced acceleration libraries such as FlashAttention for improved performance and reduced memory usage.

Quick Start & Requirements

• Install via pip install . after cloning the repository.
• Optional dependencies for accelerated training: deepspeed, packaging, ninja, flash-attn.
• FlashAttention 2 installation may take 3-5 minutes on a 64-core machine.
• Official documentation and deployment guides are available within the repository.

Highlighted Details

• Offers both instruction-following and chat-optimized models.
• Supports deployment via Hugging Face Transformers, llama.cpp, FastChat, and vLLM.
• Integrates with application ecosystems like LangChain.
• Provides scripts for generating Orca-style and Self-Chat data.

Maintenance & Community

The project is actively updated, with recent additions including Llama-2 based models, FastChat/vLLM serving support, and LangChain integration. Links to community resources are not explicitly provided in the README.

Licensing & Compatibility

The README mentions commercial-friendly licenses for Falcon and MPT-based models, but the specific license for LLaMA/Llama-2 based models is not detailed. Compatibility for commercial use with all models should be verified.

Limitations & Caveats

Vigogne is under active development, and users should be aware of potential limitations, including the possibility of generating harmful or biased content, incorrect information, or unhelpful answers.