DistiLlama is a Chrome extension designed for users who want to privately summarize or chat with web pages and local documents using locally running Large Language Models (LLMs). It addresses the need for data privacy by keeping all LLM interactions and data on the user's machine, leveraging Ollama for local LLM execution.

How It Works

The extension utilizes Ollama as the backend for running LLMs locally. It extracts text content from the active browser tab using the Readability library to ensure cleaner, more relevant text. This extracted text is then processed by LangChain.js to perform summarization or chat functionalities. The results are displayed in a user-friendly popup window within the Chrome extension.

Quick Start & Requirements

• Install Ollama: Download and install Ollama, or run it via Docker. Start Ollama with OLLAMA_ORIGINS=* OLLAMA_HOST=127.0.0.1:11435 ollama serve.
• Pull Models: Use ollama pull llama2:latest or ollama pull mistral:latest (or other supported models from https://ollama.ai/library).
• Clone & Install: Clone the repository and run pnpm install.
• Load Extension: Enable developer mode in chrome://extensions/, then load the dist folder as an unpacked extension.
• Prerequisites: Node.js (for pnpm), Ollama.

Highlighted Details

• Supports local LLM summarization and chat with web pages/documents.
• Prioritizes user data privacy by keeping all processing local.
• Integrates with Ollama for easy local LLM setup and management.
• Uses Readability for improved text extraction quality.

Maintenance & Community

The project is maintained by shreyaskarnik. Further community engagement details are not specified in the README.

Licensing & Compatibility

The license is not explicitly stated in the README. Compatibility for commercial use or closed-source linking is not detailed.

Limitations & Caveats

The UI is noted as a work in progress. Features like configurable summarization chains, saving summaries, TTS support, and advanced prompt tuning are listed as future enhancements. The project is actively being developed with several "TODO" items.