Test-Agent is an open-source project aiming to integrate Large Language Models (LLMs) into software testing workflows, targeting quality assurance engineers and developers. It provides specialized LLM capabilities for test case generation and assertion completion, enhancing test suite quality and efficiency.

How It Works

Test-Agent leverages a fine-tuned LLM, TestGPT-7B, built upon CodeLlama-7B. This model is specifically trained for tasks like multi-language test case generation (Java, Python, JavaScript) and automatic assertion completion for existing test cases. The project also includes an engineering framework for rapid local deployment and interaction via a ChatBot interface, prioritizing data privacy and local execution.

Quick Start & Requirements

• Install: Clone the repository, cd into it, and run pip install -r requirements.txt.
• Prerequisites: Python >= 3.8, Transformers == 4.33.2. Requires ~14GB VRAM for GPU acceleration. Supports various devices including Apple Silicon (mps), Intel XPU (xpu), Huawei NPU (npu), or CPU.
• Launch: Start controller (python3 -m chat.server.controller), model worker (python3 -m chat.server.model_worker --model-path models/TestGPT-7B --device mps), and web service (python3 -m chat.server.gradio_testgpt). Access the UI at http://0.0.0.0:7860.
• Models: Available on ModelScope and Hugging Face.
• Docs: ModelScope TestGPT-7B, Hugging Face.

Highlighted Details

• TestGPT-7B shows leading performance in Java, Python, and JavaScript test case generation (e.g., 48.6% Java pass@1) compared to other open-source models.
• Achieves 71.1% pass@1 for Java assertion completion with 100% strong validation.
• Offers a local, private deployment option to prevent data leakage.
• Future plans include support for Go/C++, domain knowledge Q&A, test copilot features, and larger model variants (13B, 34B).

Maintenance & Community

• The project is built upon FastChat.
• Community collaboration is encouraged for developing testing tools and intelligent assistants.

Licensing & Compatibility

• The specific license is not explicitly stated in the README, but it is an open-source project. Compatibility for commercial use or closed-source linking would require clarification of the license.

Limitations & Caveats

The project is actively under development, with plans for future language support and features. Performance metrics are provided for specific tasks and languages, and users should verify suitability for their specific testing needs.