Loyal Elephie is a RAG-powered AI chatbot designed for local LLM deployment, offering a memory-enabled conversational experience with a focus on user control and hybrid search. It targets users seeking a private, customizable AI companion that can integrate with local LLMs and optionally external Markdown editors.

How It Works

The system employs a hybrid search strategy combining ChromaDB and BM25 for efficient retrieval, with specific optimization for date-relevant queries. It utilizes an XML-based LLM agent that minimizes token usage and is compatible with local LLMs via Llama.cpp or ExllamaV2. A key feature is its controllable memory, allowing users to save, edit, and manage conversational context as a "second-brain."

Quick Start & Requirements

• Install: Clone the repository, install frontend dependencies (npm i), and backend requirements (pip install -r requirements.txt).
• Prerequisites: Python, Node.js, npm. Local LLM setup (e.g., Llama.cpp, ExllamaV2) and an OpenAI-compatible API endpoint are required for backend functionality.
• Configuration: Set up user credentials in frontend/users.json and configure API URLs, keys, and model names in backend/settings.py.
• Run: Start the frontend with npm run build && npm run start and the backend with python app.py.
• Docs: https://github.com/v2rockets/Loyal-Elephie

Highlighted Details

• Supports 12 languages.
• OpenAI API compatible for both chat and embeddings.
• Optimized for local LLMs like Meta-Llama-3-70B-Instruct and Qwen2-72b-instruct.
• Optional integration with online Markdown editors for real-time knowledge updates.

Maintenance & Community

The project is a personal project by v2rockets. No specific community channels or roadmap are detailed in the README.

Licensing & Compatibility

The repository does not explicitly state a license. Compatibility for commercial use or closed-source linking is not specified.

Limitations & Caveats

The project was primarily designed for Linux; Windows and macOS compatibility are not fully tested, with WSL recommended for Windows users. The Markdown editor integration is optional and requires a separately hosted and configured editor.