sensei  by jjleng

AI-powered answer engine using open-source LLMs

created 1 year ago
449 stars

Top 68.0% on sourcepulse

GitHubView on GitHub
Project Summary

Sensei Search is an AI-powered answer engine designed to provide users with synthesized information, similar to Perplexity AI. It targets users seeking an open-source alternative for AI-driven search and research, leveraging various open-source LLMs for enhanced capabilities.

How It Works

Sensei Search utilizes a hybrid approach combining a Next.js/Tailwind CSS frontend with a FastAPI backend. It integrates with multiple LLMs, including Command-R, Qwen-2-72b-instruct, and WizardLM-2 8x22B, alongside commercial options like Claude Haiku and GPT-3.5-turbo. Search functionality is powered by SearxNG and Bing, with Redis used for memory management. This architecture allows for flexibility in model selection and search source integration.

Quick Start & Requirements

  • Local: cd sensei_root_folder/ && docker compose up
  • Prerequisites: Requires reasonably good GPUs for running models like command-r, Qwen-2-72b-instruct, or WizardLM-2 8x22B. Assumes models are run via Ollama locally.
  • Cloud (AWS): Requires an AWS account, requested GPU quota, and a Hugging Face token.
  • Cloud Setup: pip install paka, make provision-prod, make deploy-backend, make deploy-frontend.
  • Docs: Reddit Post

Highlighted Details

  • Supports multiple open-source LLMs (Command-R, Qwen-2, WizardLM-2) and commercial models.
  • Integrates both SearxNG and Bing for search queries.
  • Utilizes Redis for memory management.
  • Offers both local and cloud deployment options via AWS and paka.

Maintenance & Community

The project is maintained by jjleng. Further community or maintenance details are not explicitly provided in the README.

Licensing & Compatibility

The README does not specify a license. Compatibility for commercial use or closed-source linking is not detailed.

Limitations & Caveats

Running the larger open-source LLMs locally requires significant GPU resources. Cloud deployment necessitates AWS setup and Hugging Face token usage, implying potential costs and vendor lock-in. The project appears to be in active development, with no explicit stability guarantees mentioned.

Health Check
Last commit

9 months ago

Responsiveness

1 day

Pull Requests (30d)
0
Issues (30d)
0
Star History
15 stars in the last 90 days

Explore Similar Projects

Starred by Chip Huyen Chip Huyen(Author of AI Engineering, Designing Machine Learning Systems), Travis Fischer Travis Fischer(Founder of Agentic), and
1 more.

morphic by miurla

0.4%
8k
AI-powered search engine with generative UI
created 1 year ago
updated 18 hours ago
Starred by Andrej Karpathy Andrej Karpathy(Founder of Eureka Labs; Formerly at Tesla, OpenAI; Author of CS 231n), Alex Cheema Alex Cheema(Cofounder of EXO Labs), and
3 more.

Perplexica by ItzCrazyKns

0.3%
23k
AI-powered search engine alternative
created 1 year ago
updated 1 day ago
Feedback? Help us improve.