Topic 4

Retrieval caps the answer

AI Engineering (Ch. 6, "RAG Architecture") puts it plainly: the success of a RAG system depends on the quality of its retriever. If the right passage never reaches the prompt, the best generator in the world is answering from memory.

A simple model makes that concrete. Say retrieval finds the right passage some fraction of the time (its recall). With the passage, the generator answers correctly most of the time; without it, rarely.

Predict first

Retrieval finds the right passage 70% of the time. With it, answers are right 90% of the time; without it, 20%. Which change helps more: a generator that's right 95% of the time with the passage, or retrieval that finds it 90% of the time?

The better generatorThe better retrievalThey help equallyNeither changes the total

How good can the answers be?

Illustrative probabilities · a simple model of a RAG system

Retrieval finds the right passage: 70%

Answer right, given the passage: 90%Answer right from memory alone: 20%

Right passage retrieved, answer right: 63%
No passage, answer right anyway: 6%
Wrong: 31%

End-to-end accuracy

69%

Ceiling with perfect retrieval

90%

End-to-end accuracy = recall × accuracy with the passage + (1 − recall) × accuracy without it. With perfect retrieval, the generator becomes the ceiling.

The numbers are illustrative, but the shape isn't. Drag recall to 100% and end-to-end accuracy is the generator's accuracy: that's the ceiling. Below it, every point of recall you add lifts the whole system. That's why error analysis asks "was the right passage retrieved?" first, and why lesson 5 measures retrieval and answers separately.

Quick check

Your RAG assistant gives wrong answers. Logs show the right passage was in the prompt for nearly every wrong answer. Where do you look?

The retrieverThe generator: the prompt, the model, or how the passages are presentedThe indexThe chunk size