Topic 6

Wrap-up, and the rest of the module

A RAG system is a retriever and a generator. Most of this module is about the retriever, because that's where most of the design decisions are, and because, as you saw, it caps what the generator can do.

Module 05, lesson by lessonStep 1 / 5

Scroll sideways to see the whole diagram

This lesson: the shape of a RAG system and when it's worth building.

Text version

This lesson: the shape of a RAG system and when it's worth building.
Next, the two families of retriever and how each computes a relevance score.
Then combining them, and reranking the candidates with a more precise model.
Then the indexing side: chunking, metadata, and rewriting queries before search.
Finally, how to measure retrieval and answers separately, and find which step failed.

Predict first

Which question does a RAG system have to answer well before generation can help?

Which model is largest?How long should the answer be?Which temperature to use?Which few passages, out of everything indexed, answer this question?

Wrap-up. From this lesson, you can now:

describe RAG as retrieve-then-generate, and where the pattern and the name come from;
explain what retrieval adds over training data alone: fresh, per-user, citable context;
decide when a knowledge base is small enough to put in the prompt, and when it needs retrieval;
separate indexing from querying, and say which changes require re-indexing;
explain why retrieval recall caps end-to-end accuracy;
route questions to no search, a document index or a table.

Quick check

Retrieval recall is 100% and answers are still wrong 10% of the time. What's the ceiling, and where's the fix?

The generator's accuracy with the passage is the ceiling; the fix is on the generation sideThe retriever; add more chunksThere's no ceilingThe index; re-embed everything

Wrap-up, and the rest of the module

Module 05, lesson by lessonStep 1 / 5

Scroll sideways to see the whole diagram

This lesson: the shape of a RAG system and when it's worth building.

Text version

This lesson: the shape of a RAG system and when it's worth building.
Next, the two families of retriever and how each computes a relevance score.
Then combining them, and reranking the candidates with a more precise model.
Then the indexing side: chunking, metadata, and rewriting queries before search.
Finally, how to measure retrieval and answers separately, and find which step failed.

Predict first

Which question does a RAG system have to answer well before generation can help?

Which model is largest?How long should the answer be?Which temperature to use?Which few passages, out of everything indexed, answer this question?

Wrap-up. From this lesson, you can now:

describe RAG as retrieve-then-generate, and where the pattern and the name come from;

explain what retrieval adds over training data alone: fresh, per-user, citable context;

decide when a knowledge base is small enough to put in the prompt, and when it needs retrieval;

separate indexing from querying, and say which changes require re-indexing;

explain why retrieval recall caps end-to-end accuracy;

route questions to no search, a document index or a table.

Quick check

Retrieval recall is 100% and answers are still wrong 10% of the time. What's the ceiling, and where's the fix?

The generator's accuracy with the passage is the ceiling; the fix is on the generation sideThe retriever; add more chunksThere's no ceilingThe index; re-embed everything