Senior interview questionAsked at Google, OpenAI, Lyft · interview write-up
Design an agent for searching internal documents that respects permissions, cites sources and abstains when it should.
Model answer
Permissions first: store each chunk with its source's access-control list and filter by the user's identity inside the retrieval call, so the model never sees what the user can't open. Sync ACL changes fast; stale permissions are a leak.
The agent plans searches, reformulates and opens documents, but every answer must cite passages, and a checker verifies claims against them. With no relevant evidence, or conflicting evidence, it abstains and suggests owners. Evaluate recall, citation accuracy, correct abstention, and permission tests with users of different access.
Practise the rest
22 RAG questions, 492 in all, each with a model answer. Free.
More from Google interview questions, OpenAI interview questions, Lyft interview questions.
More RAG interview questions
- What is retrieval-augmented generation, and how does it reduce hallucination?
- How do you choose a chunking strategy for RAG?
- Why combine keyword and embedding search, and where does a reranker fit?
- How do you evaluate a RAG pipeline?
- Your RAG chatbot keeps giving wrong answers. How do you find out why, and fix it?
- Design a RAG system that has to meet strict latency and cost targets while serving many tenants.