Senior interview questionAsked at Harvey · interview write-up
Design a question-answering agent over a law firm's legal memos, from ingestion and chunking to grounded, cited answers at scale.
Model answer
Ingest memos with the metadata lawyers filter on, such as matter, client, jurisdiction and date, plus ethical-wall permissions enforced inside the retrieval query. Chunk along the memo's structure, issue, analysis, conclusion, keeping the heading path, so citations land on a paragraph. Use hybrid search, since case names and statute numbers need exact match, then a reranker.
The model answers only from retrieved passages with pinpoint citations, and a verifier checks each claim against its cited span. Flag memos the law may have overtaken, and abstain on conflicting evidence.
Practise the rest
22 RAG questions, 492 in all, each with a model answer. Free.
More from Harvey interview questions.
More RAG interview questions
- What is retrieval-augmented generation, and how does it reduce hallucination?
- How do you choose a chunking strategy for RAG?
- Why combine keyword and embedding search, and where does a reranker fit?
- How do you evaluate a RAG pipeline?
- Your RAG chatbot keeps giving wrong answers. How do you find out why, and fix it?
- Design a RAG system that has to meet strict latency and cost targets while serving many tenants.