Where does RAG come from?

The name comes from a 2020 paper by Patrick Lewis and 11 co-authors, "Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks" (arXiv, submitted 22 May 2020, read 1 November 2026). It describes models that "combine pre-trained parametric and non-parametric memory for language generation".

In plain words: parametric memory is what the model learned in training. Non-parametric memory is an index it can search. In the paper's version, "the non-parametric memory is a dense vector index of Wikipedia".

How does RAG work?

Anthropic gives a short version: "RAG is a method that retrieves relevant information from a knowledge base and appends it to the user's prompt" (Anthropic: contextual retrieval, 19 September 2024, read 1 November 2026). The same post lists the usual steps:

  1. Break the documents into smaller chunks of text. This is content chunking.
  2. Turn each chunk into an embedding: a list of numbers that encodes its meaning.
  3. When a question comes in, find the chunks closest in meaning to it.
  4. Add those chunks to the prompt, so the model answers from them.

Why does RAG matter for B2B SaaS marketers?

Most people who search for RAG are developers building AI apps. The marketing side is AI search. ChatGPT search, Perplexity, Gemini and Copilot retrieve web pages and pass what they find to a large language model before it answers. The pattern is the same; the knowledge base is the web.

Two things follow from that:

  • Retrieval comes before writing. A page the system cannot find or read cannot shape the answer. OpenAI states that "Sites that are opted out of OAI-SearchBot will not be shown in ChatGPT search answers" (OpenAI: overview of crawlers, read 1 November 2026).
  • The model works with passages, not whole sites. A clear paragraph that answers one question is easier to retrieve and quote than the same fact spread across a long page.

What does RAG look like? (hypothetical example)

A hypothetical accounting tool builds a support chatbot. Its help center is split into 2,000 chunks. A customer asks how to export VAT reports. The system finds the three closest chunks, the model writes an answer from them, and the bot links the help article. If that article is out of date, so is the answer. RAG makes answers traceable; it does not make them right on its own.

RAG vs grounding vs fine-tuning

  • Grounding is the aim: an answer that rests on sources. RAG is one common way to get there. Many writers use the two words as near synonyms.
  • Fine-tuning changes the model itself with extra training. RAG leaves the model as it is and changes what it reads at answer time.
  • An AI citation is what you see when a RAG system shows its sources: the link next to a claim.

How does RAG connect to third-party lists?

In AI search the knowledge base is everyone's pages, not just yours. For category questions, the passages retrieved often come from other sites. Foundation and AirOps looked at 5.1 million AI responses for 50 B2B brands: on unbranded questions, only 2.24% of citations went to the brand's own pages. When Prefer checked 188 answers in July 2026, 40% of the sources cited for buyer questions were independent lists.

So for a "best X" question, the passage that gets retrieved is often a list entry about you, written by someone else. Whether that entry exists, and whether it is accurate, matters as much as your own pages. The LLM SEO guide covers what brands can do about it.