Retrieval-augmented generation, usually abbreviated RAG, is an AI architecture in which a generative model is supplied with information retrieved from an external knowledge source before or during generation. Instead of relying solely on information encoded in the model's parameters, the application retrieves relevant documents, passages, records, or other data and places selected material into the model's current context. The generator then uses that supplied context to formulate a response. RAG is particularly useful when information changes frequently, is domain-specific, is private to an organisation, or is too extensive to rely on solely through model pretraining. It separates information retrieval from language generation while connecting the two into one workflow.
A conventional RAG pipeline begins with a knowledge corpus such as documentation, policies, manuals, support articles, databases, or internal records. The corpus is prepared by extracting content, cleaning it, splitting it into chunks, generating embeddings or other search representations, and indexing the resulting records. When a user asks a question, the system processes the query and searches the index for relevant material. Retrieved passages are then selected, possibly reranked or filtered, and included in the prompt supplied to the generative model. The model generates an answer using the retrieved material as contextual evidence. Different RAG systems implement these stages differently.
Retrieval can use several techniques. Dense retrieval uses embeddings and vector similarity to find semantically related content. Sparse retrieval uses lexical methods that are particularly effective for exact terms and rare identifiers. Hybrid retrieval combines both. Reranking models can take an initial candidate set and reorder it according to more sophisticated relevance estimates. Metadata filters can restrict results by date, document type, permissions, department, language, or other properties. Query rewriting can transform a conversational question into a search-friendly query. These mechanisms can be combined because retrieval quality often depends more on the quality of the search pipeline than on the vector database alone.
Chunking is a major RAG design decision. Large documents generally cannot or should not be embedded as single vectors because retrieval needs sufficiently precise units of information. Chunks that are too small may lose necessary context, while chunks that are too large may contain multiple unrelated subjects and consume excessive context space. Overlapping chunks can preserve information across boundaries, but excessive overlap increases storage and retrieval redundancy. More advanced approaches can preserve document structure, use hierarchical retrieval, attach contextual metadata, or retrieve a parent document after identifying a relevant child passage. Chunking should therefore be evaluated against the actual document types and questions the system must answer.
RAG can reduce certain limitations of purely parametric language models, but it does not automatically eliminate hallucinations or guarantee factuality. If retrieval returns irrelevant, incomplete, stale, contradictory, or malicious information, the generator may produce an incorrect answer based on it. The model can also misunderstand correctly retrieved content. Strong RAG systems therefore use source attribution, retrieval evaluation, context validation, access controls, prompt design, and sometimes answer verification. Grounding an answer in retrieved documents can make its evidence easier to inspect, but the mere presence of citations or retrieved passages is not proof that the generated claims are supported.
RAG is particularly useful for knowledge that changes independently of model training. An organisation can update its indexed documents without retraining the underlying generative model. This can be advantageous for frequently changing policies, product catalogues, technical documentation, regulations, internal procedures, or customer-specific information. RAG can also enforce information-access boundaries by retrieving only records that the requesting user is authorised to see. However, authorisation must be implemented at the data and retrieval layers rather than assumed to emerge from the language model. Sensitive documents should not simply be inserted into prompts because a model is trusted to keep them separate.
RAG differs from fine-tuning in both mechanism and purpose. Fine-tuning changes model parameters by training on additional examples, whereas RAG leaves the generator's parameters unchanged during retrieval and supplies external information at inference time. Fine-tuning can teach behaviours, formats, terminology, or task-specific patterns, while RAG is particularly suited to providing changing or externally stored knowledge. The two approaches can also be combined: a fine-tuned model can use a RAG pipeline, and a RAG system can employ specialised models for embedding, reranking, query transformation, or generation.
The term retrieval-augmented generation describes a family of architectures rather than one fixed implementation. Some systems retrieve text from vector databases, while others retrieve structured database records, web pages, graph data, files, APIs, or combinations of sources. Some perform one retrieval step, while others use iterative retrieval, tool calls, query decomposition, or agentic workflows. Evaluation should therefore measure separate stages, including retrieval recall and precision, context relevance, answer correctness, citation or evidence support, latency, cost, and security. A well-designed RAG system is fundamentally an information-access architecture coupled to a generative model, not simply a prompt containing a few search results.
