What Is RAG? Retrieval-Augmented Generation Explained
Elexa AI Team · Updated 30 October 2026 · 7-minute read
Short answer
RAG (retrieval-augmented generation) is a way to make an AI model answer questions using your own documents. Before answering, the system searches your documents for the most relevant passages and gives them to the model along with the question. The answer is then based on your data instead of the model's general memory, and it can show which sources it used.
The problem RAG solves
Large language models such as GPT are trained on huge amounts of public text. They write well, but they have two limits. They do not know your private information, such as your company's policies, product manuals or contracts. And when they do not know something, they sometimes make up an answer that sounds right.
RAG fixes both. The model no longer has to remember your information; it reads the relevant parts at the moment it answers, a bit like an open-book exam.
How RAG works, step by step
A RAG system has two stages: preparing your documents once, and answering each question.
Prepare once
1. Split
Documents into small chunks
Prepare once
2. Embed
Turn each chunk into numbers that capture its meaning
Prepare once
3. Store
Save chunks in a vector database
Every question
4. Search
Find the chunks closest to the question
Every question
5. Generate
The model answers from those chunks and cites them
Preparing the documents
- Split each document into small chunks, usually a few paragraphs each, so the system can find precise passages.
- Create embeddings. An embedding model turns each chunk into a list of numbers that captures its meaning. Chunks about similar topics get similar numbers.
- Store the chunks and their embeddings in a vector database, such as PostgreSQL with pgvector.
Answering a question
- Search. The question is turned into an embedding too, and the database returns the chunks whose meaning is closest to it.
- Generate. The question and those chunks are sent to the language model with an instruction such as "answer only from these sources and cite them". The model writes the answer.
RAG vs fine-tuning
Fine-tuning means training a model further on your own examples. It changes how the model writes and behaves. RAG changes what the model knows at the moment it answers.
| RAG | Fine-tuning | |
|---|---|---|
| Best for | Answering from documents that change | Teaching a style, format or specialised behaviour |
| Updating knowledge | Add or replace documents; no retraining | Train again |
| Showing sources | Yes, each answer can cite its passages | No |
| Cost to start | Low | Higher: needs prepared training data and training runs |
For most "answer from our documents" problems, RAG is the right first choice. Some teams later combine both.
Where RAG is used
- Customer support: an assistant that answers from help-centre articles and hands hard questions to a person
- Internal knowledge: HR policies, IT guides and onboarding documents in one place to ask questions
- Product manuals: technicians asking how to fix or set up equipment
- Contracts and reports: finding clauses or figures across many long documents
What you need to build one
- Python and a web framework such as FastAPI
- An embedding model and a language model, for example from OpenAI
- A vector database, such as PostgreSQL with pgvector
- A set of documents to answer from, and a few test questions with known answers so you can measure quality
What makes RAG work well
Most RAG problems come from retrieval, not from the language model. If the search returns the wrong passages, even the best model gives a poor answer. Chunk size, the quality of the embeddings and how many passages you send all matter. Good systems are tested with real questions and improved step by step, rather than tuned by guesswork.
Next step
Build a RAG Chatbot walks you through building AskFlow, a complete RAG assistant with Python, FastAPI, pgvector and OpenAI, chapter by chapter with full source code. New to working with text? Start with NLP in Python, or see the AI Learning Path for the full order.