MFTCoder is a framework for high-accuracy, efficient multi-task fine-tuning of Large Language Models (LLMs), particularly for code-related tasks. It targets researchers and developers working with code LLMs, offering a robust solution for improving model performance across diverse coding challenges through its multi-task fine-tuning approach.

How It Works

MFTCoder employs a multi-task fine-tuning (MFT) strategy to train LLMs on multiple datasets simultaneously, aiming for improved generalization and performance on unseen tasks. It supports various efficient fine-tuning techniques like LoRA, QLoRA, and full-parameter training, enabling the fine-tuning of large models with reduced computational resources. The framework also incorporates methods like Self-paced Loss and the newer Convergence Balancer (CoBa) to manage convergence across different tasks, ensuring balanced training outcomes.

Quick Start & Requirements

• Install dependencies using bash sh init_env.sh.
• Requires CUDA (>= 11.4, preferably 12.1) and PyTorch (>= 2.1.0).
• Flash Attention (>= 2.3.0) is highly recommended.
• Official quick-start details are available within the respective framework READMEs (MFTCoder-accelerate, MFTCoder-atorch).

Highlighted Details

• Achieves state-of-the-art performance on HumanEval, with models like CodeFuse-DeepSeek-33B reaching 78.7% pass@1.
• Supports a wide range of base models including CodeLlama, StarCoder, Qwen, Mixtral, and DeepSeek-coder.
• Offers two training frameworks: MFTCoder-accelerate (with DeepSpeed/FSDP) and MFTCoder-atorch.
• Released two high-quality code instruction datasets: Evol-instruction-66k and CodeExercise-Python-27k.

Maintenance & Community

The project is actively developed by the AI Native team at Ant Group, with recent updates in October 2024 introducing preference alignment methods (DPO/RPO/ORPO) and new convergence balancing techniques. The team has a strong publication record at top AI conferences.

Licensing & Compatibility

The project appears to be open-source, with models available on HuggingFace and ModelScope. Specific licensing details for the framework and models are not explicitly stated in the README, but compatibility for commercial use would require verification.

Limitations & Caveats

While the framework supports various models and fine-tuning methods, specific details on the limitations of each supported model or the exact resource requirements for full-parameter training are not elaborated. The project is actively evolving, with new features and improvements being released regularly.