Megabots simplifies the creation of production-ready LLM applications, specifically for answering questions over documents. It targets developers and researchers seeking to quickly deploy bots with features like document indexing, API exposure via FastAPI, and Gradio UIs, leveraging LangChain and Langchain-Serve.

How It Works

Megabots employs retrieval augmented generation (RAG). When a user asks a question, the system first queries a document index (created using FAISS) to find relevant information. This retrieved context, along with the original question, is then passed to an LLM (initially OpenAI models) to generate an answer. The library also supports conversational memory and integration with vector databases like Milvus.

Quick Start & Requirements

• Install: pip install megabots
• Prerequisites: OpenAI API key (OPENAI_API_KEY), Python 3.7+.
• Setup: Requires defining an index (either a directory of documents or a .pkl file).
• Docs: https://huggingface.co/spaces/momegas/megabots

Highlighted Details

• One-line bot creation for document Q&A.
• Automatic API exposure with langchain-serve and UI with Gradio.
• Supports custom prompts, conversational memory, and Milvus vectorstore.
• Indexing can be done from local directories or saved .pkl files.

Maintenance & Community

• Active development, indicated by a project board.
• Discord community available: https://discord.gg/zkqDWk5S7P

Licensing & Compatibility

• License: Not explicitly stated in the README.

Limitations & Caveats

• Described as a "work in progress" with potential API changes.
• A known issue exists where custom prompts with the sources=True flag cause the code to break.
• Currently only supports OpenAI models and the qna-over-docs task.