This project provides an AI-powered search and chat interface for Paul Graham's essays, targeting users who want to query his writings. It leverages Retrieval Augmented Generation (RAG) to deliver contextual answers based on the essay content.

How It Works

The system uses OpenAI Embeddings (text-embedding-ada-002) to generate vector representations of text chunks from Paul Graham's essays. A Supabase PostgreSQL database with the pgvector extension stores these embeddings. User queries are embedded and matched against the database using cosine similarity to retrieve relevant passages. These passages are then used to construct a prompt for GPT-3.5-turbo, enabling a conversational Q&A experience.

Quick Start & Requirements

• Install dependencies: npm i
• Set up environment variables: OPENAI_API_KEY, NEXT_PUBLIC_SUPABASE_URL, SUPABASE_SERVICE_ROLE_KEY in .env.local.
• Scrape essays: npm run scrape
• Generate embeddings and store in Supabase: npm run embed (approx. 20-30 minutes).
• Run the app: npm run dev
• Requires an OpenAI API key and a Supabase account with PostgreSQL and pgvector.

Highlighted Details

• RAG implementation for querying essays.
• Uses OpenAI Embeddings and GPT-3.5-turbo.
• Data storage via Supabase PostgreSQL with pgvector.
• Cosine similarity for retrieval.

Maintenance & Community

• Project maintained by mckaywrigley.
• Contact available via Twitter.

Licensing & Compatibility

• All code and data are 100% open-source. (Specific license not detailed in README).
• Compatible with commercial use if underlying OpenAI and Supabase terms are met.

Limitations & Caveats

The project prioritizes simplicity over composability, with much of the application logic contained within a single homepage component.