Topic 4

Step through a trace

Here's the loop on a concrete task: a customer says they were charged twice. The agent has three tools: list_charges and get_policy, which only read, and refund, which moves money. The trace is written by hand to show the shape; no model was run.

Predict first

The agent will check the charges, check the policy, then refund. How many separate model calls does that take, including the final answer?

13410

A support agent, step by step

Illustrative trace · tools: list_charges, get_policy (read), refund (write)

step 1 of 11

Model calls

Tool calls

Write actions

Task
I was charged twice for order 4417. Can you refund the duplicate?

Everything above the current step is in the model's context when it produces the next one. The model only writes text; the harness parses each tool call, runs it and appends the result.

Things to notice as you step:

The context grows. The last model call reads the task, three thoughts, three calls and three results. On a long task, that growth becomes the problem of lesson 3.
Arguments come from results. ch_2 and 49 aren't in the task; the agent read them from the first tool result. AI Engineering (Ch. 6, "Plan generation") notes that parameter values are often extracted from earlier tool outputs, and that the model frequently has to guess when information is missing.
The policy check is a choice. Nothing forced the agent to call get_policy before refund. If it had skipped it, the refund would have gone through anyway. Lessons 2 and 6 are about making checks like this structural instead of hoping for them.
One step is different. refund is a write action. If the harness retried it after a timeout, the customer might be refunded twice. That's lesson 4.

Quick check

The book's tip for working with agents: always have the system report the parameter values it used for each function call. Why?

Because the values are generated by the model and can be wrong (a hallucinated ID, a wrong amount), and you can only check what you can seeFor billingTo make the trace longerBecause tools require it

Step through a trace

Predict first

The agent will check the charges, check the policy, then refund. How many separate model calls does that take, including the final answer?

13410

A support agent, step by step

Illustrative trace · tools: list_charges, get_policy (read), refund (write)

step 1 of 11

Model calls

Tool calls

Write actions

Task
I was charged twice for order 4417. Can you refund the duplicate?

Everything above the current step is in the model's context when it produces the next one. The model only writes text; the harness parses each tool call, runs it and appends the result.

Things to notice as you step:

The context grows. The last model call reads the task, three thoughts, three calls and three results. On a long task, that growth becomes the problem of lesson 3.

Arguments come from results. ch_2 and 49 aren't in the task; the agent read them from the first tool result. AI Engineering (Ch. 6, "Plan generation") notes that parameter values are often extracted from earlier tool outputs, and that the model frequently has to guess when information is missing.

The policy check is a choice. Nothing forced the agent to call get_policy before refund. If it had skipped it, the refund would have gone through anyway. Lessons 2 and 6 are about making checks like this structural instead of hoping for them.

One step is different. refund is a write action. If the harness retried it after a timeout, the customer might be refunded twice. That's lesson 4.

Quick check

The book's tip for working with agents: always have the system report the parameter values it used for each function call. Why?

Because the values are generated by the model and can be wrong (a hallucinated ID, a wrong amount), and you can only check what you can seeFor billingTo make the trace longerBecause tools require it