This project provides a custom agent framework for local knowledge bases and large language models (LLMs), specifically targeting users who want to integrate LLMs like ChatGLM-6B or OpenAI with custom tools and web search capabilities. It aims to overcome limitations in existing agent prompts when using less powerful LLMs by offering a tailored approach for improved action and input field accuracy.

How It Works

The core of the project is a custom agent designed to work with LLMs that may struggle with complex prompt structures. It implements a custom agent logic that is more robust for models like ChatGLM-6B, achieving an estimated 80% accuracy in responding to instructions. This agent integrates with local knowledge bases and supports web retrieval, allowing for more sophisticated applications by combining LLM capabilities with external tools.

Quick Start & Requirements

• Install/Run: Execute python server.py to start the Flask server.
• Prerequisites:
  • Deployed ChatGLM-6B model accessible via API (or configure for OpenAI).
  • RapidAPI key for web search (e.g., Bing Web Search API).
  • Python environment.
• Setup: Run python helloworld.py to verify setup.
• Demo: python cli.py for an example interaction.
• Docs: https://zhuanlan.zhihu.com/p/635724707

Highlighted Details

• Custom agent implementation tailored for ChatGLM-6B, improving action/input accuracy.
• Supports local knowledge bases and web search (e.g., Bing).
• Flexible LLM integration (ChatGLM-6B or OpenAI).
• Flask-based server for API access.

Maintenance & Community

No specific information on contributors, sponsorships, or community channels (like Discord/Slack) is provided in the README.

Licensing & Compatibility

The project's licensing is not explicitly stated in the README. Compatibility for commercial use or closed-source linking is not specified.

Limitations & Caveats

The custom agent is noted to have an estimated 80% accuracy in instruction following, indicating potential for incorrect responses. The project does not appear to have extensive community support or clear licensing terms, which may impact long-term viability and commercial adoption.