Project Overview

Build a ligthweight Retrieval Augmented Generation (RAG) system that provides intelligent search and analysis of Paul Graham's essays from paulgraham.com/articles.html.

Tech Stack

• Frontend: Next.js with Shadcn UI components
• Database: Supabase (PostgreSQL) containerized with Docker/OrbStack
• Backend, choose one of:
  • TypeScript
  • Python

Core Components

1. Data Ingestion Pipeline (Python)

• [ ] Scrapes essays from Paul Graham's website systematically
• [ ] Stores processed data in Supabase (PostgreSQL + pgvector)

2. Search & RAG Interface (TypeScript)

Implement an intuitive interface that:

• [ ] Handles user queries and presents results in a chat UI in the Next.js frontend
• [ ] Performs search across the essay database
• [ ] [Optional] Generate summaries based on relevant essays using LLM
• [ ] [Optional] Build an evalset of questions and golden answers to benchmark the implementation

Example Interaction

User Query: "What makes a good founder?"

Response:

1. Relevant essay links with context:
   • Founder Mode
   • The Right Kind of Stubborn
   • How to Be a Founder