What components make up a RAG stack?

Retrieval-augmented generation pairs a model’s frozen training memory with a searchable memory of your own documents, an idea introduced by Lewis and colleagues in 2020.^[1] As a stack it comes down to four parts, and it helps to name each one by the single job it does.

The embedding model turns a piece of text into a vector, a list of numbers that places similar meanings near each other. The vector store holds those vectors and searches them fast; this can be a dedicated vector database or an extension that adds vector search to a database you already run, the way pgvector does for Postgres.^[2] The retriever is the logic that takes a user’s question, embeds it with the same model, and pulls back the closest chunks. The LLM then reads those chunks alongside the question and writes the answer. Strip out the first three and you have a plain chatbot answering from memory alone; add them and the model answers from your material instead.

The order matters because the parts run in two phases. Indexing happens once and offline: split documents into chunks, embed each, store the vectors. Retrieval happens at question time: embed the question, search, hand the results to the LLM. Frameworks like LlamaIndex exist mostly to wire these phases together so you are not hand-coding the chunking, the embedding calls, and the search loop yourself.^[3] Where this stack tends to break is the retrieval step. If the right chunk is not found, the model answers from training memory and may sound confident while being wrong, which is why grounding and how it fails gets its own page. That points at a structural weak spot in the handoff itself: the LLM only ever sees whatever chunks the retriever handed it, with no way to tell a well-matched chunk from a mediocre one. A bad retrieval and a bad generation can look identical from the outside, which is why checking what was actually retrieved is worth doing directly rather than trusting a wrong-sounding answer to explain itself.