RAG: Retrieval-Augmented Generation in Practice

RAG (Retrieval-Augmented Generation) supplies an LLM with relevant documents from a knowledge base — so it answers based on current, proprietary data instead of just its training knowledge.

Pipeline

1. Ingestion: split documents into chunks, generate embeddings, store in a vector database.
2. Retrieval: for a question, find the most similar chunks via vector or hybrid search.
3. Generation: pass question + chunks to the LLM, generate the answer.

Chunking

Chunk size and overlap strongly affect quality. Too large: much context but imprecise hits. Too small: context is lost. Proven: 300-800 tokens with 10-20% overlap, respecting semantic boundaries (paragraphs, headings).

Vector databases

pgvector, Qdrant, Weaviate, Milvus, Chroma. For small projects, SQLite with a vector extension suffices.

Pitfalls

• Relevance: bad chunks lead to bad answers (try re-ranking).
• Hallucination: instruct the LLM to answer only from chunks and cite sources.
• Metadata filters (date, author) improve retrieval.

See also: AI & Automation.