Senior interview questionAsked at Apple · interview write-up
Design the routing that decides whether a voice assistant query is answered by the on-device model or sent to a large server LLM.
Model answer
Route on what each query needs. The on-device model takes device actions, short answers and private requests: fast, offline, no per-query cost. Open-ended, knowledge-heavy or long reasoning requests go to the server model.
The router is rules for clear intents, then a small on-device classifier scoring difficulty, plus escalation when the local answer's confidence is low. Ask before sending personal context off the device. Track quality per cost and latency: share escalated, quality against always-server on a labelled set, and tail latency. Retrain on misrouted cases.
Practise the rest
9 Cost per request questions, 492 in all, each with a model answer. Free.
More from Apple interview questions.
More Cost per request interview questions
- How is an API request priced, and why is the reply often most of the bill?
- How do you turn a machine's hourly price into a cost per token, and how much can batching change it?
- (Illustrative numbers.) Model A costs $0.004 per request and is correct 50% of the time; model B costs $0.006 and is correct 95% of the time. Which is cheaper, and what should you track?
- You're deciding between a model API and self-hosting. How do you find the break-even, and what does the simple calculation leave out?
- How do you cut token costs at scale?
- When is a small open model good enough, rather than a frontier model?