Sensei Search is an AI-powered answer engine designed to provide users with synthesized information, similar to Perplexity AI. It targets users seeking an open-source alternative for AI-driven search and research, leveraging various open-source LLMs for enhanced capabilities.

How It Works

Sensei Search utilizes a hybrid approach combining a Next.js/Tailwind CSS frontend with a FastAPI backend. It integrates with multiple LLMs, including Command-R, Qwen-2-72b-instruct, and WizardLM-2 8x22B, alongside commercial options like Claude Haiku and GPT-3.5-turbo. Search functionality is powered by SearxNG and Bing, with Redis used for memory management. This architecture allows for flexibility in model selection and search source integration.

Quick Start & Requirements

• Local: cd sensei_root_folder/ && docker compose up
• Prerequisites: Requires reasonably good GPUs for running models like command-r, Qwen-2-72b-instruct, or WizardLM-2 8x22B. Assumes models are run via Ollama locally.
• Cloud (AWS): Requires an AWS account, requested GPU quota, and a Hugging Face token.
• Cloud Setup: pip install paka, make provision-prod, make deploy-backend, make deploy-frontend.
• Docs: Reddit Post

Highlighted Details

• Supports multiple open-source LLMs (Command-R, Qwen-2, WizardLM-2) and commercial models.
• Integrates both SearxNG and Bing for search queries.
• Utilizes Redis for memory management.
• Offers both local and cloud deployment options via AWS and paka.

Maintenance & Community

The project is maintained by jjleng. Further community or maintenance details are not explicitly provided in the README.

Licensing & Compatibility

The README does not specify a license. Compatibility for commercial use or closed-source linking is not detailed.

Limitations & Caveats

Running the larger open-source LLMs locally requires significant GPU resources. Cloud deployment necessitates AWS setup and Hugging Face token usage, implying potential costs and vendor lock-in. The project appears to be in active development, with no explicit stability guarantees mentioned.