Most RAG tutorials stop at the same place: chunk your documents, embed them, retrieve the top-k, stuff it into the context window, call the LLM. That works for demos. It doesn’t hold up under real query distributions, large document sets, or production latency requirements.
This project is my attempt to build a RAG system the way I think it should actually be done — with adaptive retrieval, explicit agent orchestration, intelligent caching, and the infrastructure to measure and improve it over time.
Why “adaptive” and “multi-agent”
Naive RAG treats every query the same way. But a vague exploratory question needs different retrieval behavior than a precise factual lookup. A question about a topic with lots of conflicting documents needs verification logic that a simple similarity search can’t provide.
The multi-agent design addresses this by decomposing the retrieval process into specialized roles:
- Query Understanding Agent — disambiguates the query, rewrites it if needed, and classifies its type
- Retrieval Agent — executes the vector search against pgvector, adjusting strategy based on query type
- Context Optimization Agent — reranks and compresses retrieved chunks to fit the context window efficiently
- Verification Agent — cross-checks retrieved content for consistency before synthesis
- Response Synthesis Agent — generates the final answer grounded in the verified context
Each agent is a LangGraph node. The graph structure means the routing between agents is explicit and inspectable, not buried in a monolithic chain.
Infrastructure decisions
PostgreSQL + pgvector for the vector store — not a standalone vector database. The reasoning: pgvector lets you join semantic search with structured filters in a single query, which is essential for real document collections where you want things like “find relevant chunks from documents published after this date written by this author.”
Redis as a caching layer for two purposes: embedding cache (avoid re-embedding identical text) and query result cache for common queries. In a real deployment, a significant fraction of queries are near-duplicates.
FastAPI backend with async throughout — retrieval latency is dominated by I/O, so synchronous handling is a significant performance loss.
React + TypeScript + TailwindCSS frontend, containerized with Docker Compose for reproducible local development.
Current status
The architecture is scaffolded and the core agent graph is running. The vector ingestion pipeline, embedding manager, and basic retrieval flow are working. The evaluation dashboard, observability layer, and more sophisticated agent behaviors are in active development.
I’m building this incrementally — the goal is a platform that demonstrates real AI engineering practices, not just a proof of concept.
Stack
- Python · FastAPI · LangGraph · PostgreSQL + pgvector · Redis · SQLAlchemy · UV
- React · TypeScript · TailwindCSS
- Docker Compose · OpenAI-compatible embedding APIs
What I’m learning
The gap between “RAG demo” and “RAG system” is wider than I expected before starting this. The interesting engineering problems are almost all outside the core retrieval loop — caching strategy, async pipeline design, evaluation methodology, context length management. Building this has shifted how I think about AI engineering as a discipline, not just AI modeling.