Junior interview question
Which tool failures should a harness retry automatically?
Model answer
Transient ones: timeouts and dropped connections, with backoff. Not permanent ones like invalid arguments: return those to the model so it can fix the call. Overload responses mean slow down, not retry harder.
Scroll sideways to see the whole diagram
Not every failure deserves the same response.
Text version
- Not every failure deserves the same response.
- Transient failures may succeed if tried again a little later: retry, with backoff.
- Overload errors mean the service is struggling. Google's SRE book: return a specific overload status so callers back off instead of retrying.
- Permanent errors will never succeed: malformed requests, invalid arguments. Retrying the identical call only adds load.
- In an agent, many permanent errors come from the model's arguments. Returning the error to the model lets it correct the call.
- AI Engineering's tool failures include a tool simply returning the wrong output: no error is raised. Catching those needs checks on results (lesson 5).
Practise the rest
6 Agent failures questions, 492 in all, each with a model answer. Free.