Topic 3

Least privilege

The most reliable limit is what the agent physically can't do. OWASP's mitigations for excessive agency start there:

Only the tools the agent needs. Every tool is a capability a mistake or an attacker can use.
Only the functions each tool needs. A tool for reading a file doesn't also need to delete one.
Narrow tools over open-ended ones. OWASP's example of open-ended is shell access: it can do almost anything.
Minimum permissions downstream. Read-only database access when the agent only reads.
Act in the user's context. Run actions with the identity and scope of the person the agent serves, not a shared privileged account.
Complete mediation. Enforce authorisation in the downstream systems on every request, instead of relying on the model to decide what's allowed.

Predict first

A support agent needs to look up the current customer's orders. Which design limits damage best?

A run_sql tool on the admin account, with a prompt saying “only query this customer's orders”A list_orders() tool that takes no customer ID: it runs with the signed-in customer's credentials and returns only their ordersA run_sql tool on a read-only account for all customersAccess to the database's admin console

Narrow tools, narrow permissionsStep 1 / 4

Scroll sideways to see the whole diagram

A general tool lets a mistake or an attack do anything that tool can do: any SQL, any recipient.

Text version

A general tool lets a mistake or an attack do anything that tool can do: any SQL, any recipient.
Purpose-built tools limit the damage to what the task needed. OWASP: prefer narrow tools to open-ended ones like shell access, and grant the minimum permissions, such as read-only database access.
OWASP also recommends running actions in the user's own context and scope, so an agent can't reach other customers' data.
And enforcing authorisation in the downstream systems themselves, on every request (“complete mediation”), rather than trusting the model to decide.

Narrow tools help reliability as well as safety. Lesson 5's call log found most mistakes on the general refund tool; a refund_duplicate(order_id) that finds the later charge itself can't refund the wrong one. Anthropic's advice to design the agent-computer interface carefully points the same way: tools shaped so the right call is easy and the wrong one is hard, or impossible.

AI Engineering (Ch. 6, "Capability extension") warns that code interpreters, the most open-ended tool of all, come with the risk of code injection attacks. If an agent must run code, contain where it runs: the last page.

Quick check

Why is “run actions with the user's own permissions” a strong defence?

It's fasterHowever the model is manipulated, it can't do anything the user couldn't do themselves; another customer's data is out of reachIt makes logs shorterIt removes the need for tools

Least privilege

The most reliable limit is what the agent physically can't do. OWASP's mitigations for excessive agency start there:

Only the tools the agent needs. Every tool is a capability a mistake or an attacker can use.

Only the functions each tool needs. A tool for reading a file doesn't also need to delete one.

Narrow tools over open-ended ones. OWASP's example of open-ended is shell access: it can do almost anything.

Minimum permissions downstream. Read-only database access when the agent only reads.

Act in the user's context. Run actions with the identity and scope of the person the agent serves, not a shared privileged account.

Complete mediation. Enforce authorisation in the downstream systems on every request, instead of relying on the model to decide what's allowed.