# Tool calls and constrained sampling interview questions

> 8 tool calls and constrained sampling interview questions for AI and ML engineer roles, each with a model answer.

Canonical: https://llmdojo.dev/interview/topic/tools-and-constraints
Last updated: 2026-10-23

8 questions: 1 junior, 4 mid-level and 3 senior. Taught in [Tool Instructions and Constrained Outputs](https://llmdojo.dev/learn/prompt-context-engineering/tools-and-constrained-outputs).

## [In tool calling, who executes the tool, and what does the model need in order to use the result?](https://llmdojo.dev/interview/q/in-tool-calling-who-executes-the-tool-and-what-does-the-model)

Level: Junior

For tools you define, **your application**. The model **only emits a name and arguments**, such as `lbs_to_kg` with `{"lbs":40}`.

**Diagram: The tool-call loop**

1. The context holds the tool definitions and the user's question.
2. The model emits a structured call: a tool name and JSON arguments. With Anthropic's API the reply stops with stop_reason "tool_use".
3. The call is text the model generated. Your code checks it before anything runs.
4. Your code executes the function. The model has no hands: it only asked.
5. The output goes back into the context (a tool_result block) and the model runs again, until it answers instead of calling.

After AI Engineering, Ch. 6 "Function calling", and Anthropic, "Tool use with Claude" (fetched 2026-10-21). Illustrative call.

Your code runs it and sends the output back, a `tool_result` in Anthropic's API, in the next request. Every result becomes context: **billed tokens** the model reads like any other text.

## [The model keeps calling search_orders for refund questions that another tool handles. What do you change first?](https://llmdojo.dev/interview/q/the-model-keeps-calling-search-orders-for-refund-questions-that-another-tool)

Level: Mid-level

The two **tool descriptions**. The model **never sees your function**: the API builds a system prompt from the definitions, billed as input tokens, and the model chooses from that text. Anthropic calls detailed descriptions "by far the most important factor in tool performance" and suggests at least **3 to 4 sentences**: what the tool does, when to use it and when not, what each parameter means, and its limits.

## [Explain one step of constrained sampling. What does it guarantee, and what doesn't it?](https://llmdojo.dev/interview/q/explain-one-step-of-constrained-sampling-what-does-it-guarantee-and-what)

Level: Mid-level

Score every token, **mask the ones the rule forbids**, renormalise the survivors, then sample as usual. It guarantees **form, not truth**.

**Diagram: One constrained sampling step**

1. The model scores every token, as in Module 01. Here GPT-2 continues "The largest planet in our solar system is".
2. Before sampling, every token that would break the rule is removed: its probability becomes exactly 0.
3. The allowed tokens are rescaled to sum to 1. Their order doesn't change: " Earth" stays ahead of " Jupiter".
4. Sampling then works as before, but can only pick an allowed token. Format is guaranteed; content is not.

After AI Engineering, Ch. 2 "Structured Outputs" > "Constrained sampling". Probabilities: GPT-2 small, lib/labs/data/sampling.json.

On "The largest planet in our solar system is", GPT-2's recorded favourite is " about" (9.4%). Allowing only planet names, " Earth" (2.8% before masking) edges out " Jupiter" (2.7%); renormalised, 24.8% to 23.7%, upper bounds since only the top 50 tokens are named. Renormalising **keeps the survivors' order**.

## [Your agent has a delete_records tool. What has to happen between the model emitting a call and the delete running?](https://llmdojo.dev/interview/q/your-agent-has-a-delete-records-tool-what-has-to-happen-between)

Level: Senior

Treat the arguments as **a stranger's input**: an allowlist of tools, a schema check, then app-side policy (who may delete what, how many, after which confirmation). Code catches an invalid tool or invalid parameters, **not a wrong value**: `lbs` = 100 when the user said 120 passes every check. Confirmation is app state, the user's click, never a `"confirmed": true` the model wrote. Strict tool use guarantees **shape, not values**: a schema-valid delete is still a delete.

## [Should you turn on grammar-constrained decoding for every structured output? What would you watch?](https://llmdojo.dev/interview/q/should-you-turn-on-grammar-constrained-decoding-for-every-structured-output-what)

Level: Senior

**Not blindly**. It needs a grammar per format, can add latency, and output cut off at the length limit is still unparsable. It can also **override the model**, so watch **the mass on allowed tokens**. On "I opened the box and found", the digits in GPT-2's top 50 held 1.5%, and the top one, " 2", was its 20th choice; " two" had 7.7 times the probability of " 2". When that mass is small, ask for the format the model writes naturally, or convert afterwards.

## [Implement a client that spreads LLM calls across several providers, respecting their rate limits and failing over when one is down.](https://llmdojo.dev/interview/q/implement-a-client-that-spreads-llm-calls-across-several-providers-respecting-their)

Level: Mid-level

Try providers in preference order, and **bench** one for its retry-after time when it rate-limits or errors.

```python

class RateLimited(Exception):
    retry_after = None

class Pool:
    def __init__(self, providers, cooldown=30.0):
        self.ps = providers  # in preference order
        self.until = {p.name: 0.0 for p in providers}
        self.cooldown = cooldown

    def call(self, req, rounds=3):
        for r in range(rounds):
            for p in self.ps:
                now = time.monotonic()
                if self.until[p.name] > now:
                    continue
                try:
                    return p.complete(req)
                except RateLimited as e:
                    wait = e.retry_after or self.cooldown
                    self.until[p.name] = now + wait
                except (TimeoutError, ConnectionError):
                    self.until[p.name] = now + self.cooldown
            time.sleep(2 ** r + random.random())
        raise RuntimeError("all providers failed")
```

Interviewers look for **jittered backoff**, never retrying a bad request, timeouts on every call, and no silent failover mid-stream once tokens reach the user.

## [Design an AI chat for a spreadsheet product that turns commands like sum column K or delete row 5 into validated tool calls on the sheet.](https://llmdojo.dev/interview/q/design-an-ai-chat-for-a-spreadsheet-product-that-turns-commands-like)

Level: Mid-level

Give the model a few **typed tools**, like sum a range or delete rows, plus a compact view of the sheet: headers, size, a few rows.

**Diagram: The tool-call loop**

1. The context holds the tool definitions and the user's question.
2. The model emits a structured call: a tool name and JSON arguments. With Anthropic's API the reply stops with stop_reason "tool_use".
3. The call is text the model generated. Your code checks it before anything runs.
4. Your code executes the function. The model has no hands: it only asked.
5. The output goes back into the context (a tool_result block) and the model runs again, until it answers instead of calling.

After AI Engineering, Ch. 6 "Function calling", and Anthropic, "Tool use with Claude" (fetched 2026-10-21). Illustrative call.

It emits a tool call; your code **validates** it against the schema and the live sheet, checking column K exists and row 5 is in range, then executes. Destructive actions show a **preview and need confirmation**, and everything is undoable. Evaluate on commands with expected calls.

## [Design a scalable service where an AI agent plans work and calls approved tools without ever bypassing deterministic controls. Cover prompt injection, approvals, budgets and loop termination.](https://llmdojo.dev/interview/q/design-a-scalable-service-where-an-ai-agent-plans-work-and-calls)

Level: Senior

The model proposes; **deterministic code decides**. Every tool call passes a policy layer the model can't change: a tool allowlist, argument schemas, the user's own permissions, and rate and spend limits. Irreversible or high-value actions need human **approval**, bound to the exact arguments shown.

**Diagram: Defences in layers**

1. A model can be trained to rank instructions by where they come from, so a tool output loses to the system prompt.
2. The prompt can name what is off-limits and mark untrusted text as data. Both rely on the model complying.
3. System-level defences don't depend on the model: a tool it doesn't have can't be misused, and a send that needs approval can be declined.
4. Layers reduce risk; none removes it. Plan for an attack that gets through.

Levels from AI Engineering, Ch. 5, "Defenses Against Prompt Attacks"; measures from OWASP LLM01:2025.

Tool outputs are untrusted: they can't grant permissions, and sensitive data can't reach outbound tools. Loops end at step, token and time budgets, or on repeated actions. Scale with stateless workers over a durable task queue, logging every step for **audit and replay**.

## Related
- [All interview questions](https://llmdojo.dev/interview)