This repository provides a collection of LoRA models and training code for enhancing AI-driven software development efficiency. It targets engineers and researchers interested in fine-tuning large language models like LLaMA and ChatGLM for tasks such as user story generation, test code creation, code completion, and text-to-SQL conversion. The project offers pre-trained LoRA models and detailed tutorials for replicating the training process.

How It Works

The project leverages LoRA (Low-Rank Adaptation) to fine-tune pre-trained models on specific datasets tailored for software engineering tasks. It standardizes the AI-assisted development process by breaking down tasks into granular steps, feeding data for each step to the models. This approach aims to maximize the "copy-paste" effect of AI, generating accurate outputs for each micro-task. Datasets are prepared using OpenAI for generating user tasks and stories, and for creating code and test cases based on class information.

Quick Start & Requirements

• Installation: Primarily through provided Jupyter Notebooks (alpaca-lora.ipynb, chatglm-tuning.ipynb) and Python scripts.
• Prerequisites: Python, PyTorch, Hugging Face Transformers, and potentially CUDA for GPU acceleration. Specific model requirements (e.g., LLaMA-7B, ChatGLM-6B) are noted. Access to cloud GPUs (e.g., OpenBayes) is recommended for training.
• Resources: Training LoRA models can be resource-intensive, with training times varying from 25 minutes to several hours depending on dataset size and hardware.
• Links:
  • LLaMA Alpaca LoRA Training: https://github.com/tloen/alpaca-lora
  • ChatGLM Tuning: https://github.com/unit-mesh/unit-minions/blob/main/chatglm-tuning.ipynb
  • Data Preparation: https://github.com/unit-mesh/minions-data-prepare

Highlighted Details

• Focuses on practical AI applications in software development lifecycle.
• Provides specific LoRA models for user story generation, test code generation, code assistance, and text-to-SQL.
• Demonstrates a methodology for fine-grained task decomposition for AI training.
• Includes video tutorials and example outputs for various AI-assisted tasks.

Maintenance & Community

• The project is associated with unit-mesh.
• Sponsors include AIOS Club (for OpenAI Key) and OpenBayes (for Cloud GPU).
• Roadmap indicates completion of several key training tasks.

Licensing & Compatibility

• The README does not explicitly state a license for the repository's code or datasets. However, it mentions using OpenAI-generated data and publicly available projects, and notes that users are responsible for the consequences of their training. Compatibility for commercial use is not specified.

Limitations & Caveats

• The project relies heavily on OpenAI for data generation, which may incur costs and has usage policies.
• The quality of generated test cases by OpenAI is noted as potentially unreliable, suggesting a need for human review.
• Some datasets used for text-to-SQL and text-to-code are described as having low quality but being usable.