What an AI agent should never be allowed to do on its own

Most AI agent failures we read about are not really AI failures. The model did roughly what it was asked. The problem was what it had been allowed to do: an agent with full access to the mailbox, the database and the payment account, running with nobody checking. OWASP calls this "excessive agency", and it sits in its top ten risks for AI applications.

We design every agent the way you would set up a new employee on their first day. They get their own login, they can only open the folders their job needs, and a manager signs off on anything important.

Never on its own

Whatever the agent does, these actions should always wait for a person to approve them:

• Moving money: payments, refunds, credits, changes to bank details.
• Deleting anything that cannot be restored.
• Sending messages outside your organisation: to customers, suppliers or the public.
• Changing permissions or settings, including its own.
• Anything a regulator, auditor or court might later ask "who decided this?" about.

The agent can still do most of the work. It prepares the refund, drafts the email, lists the records to delete. A person clicks approve. That click usually takes seconds, and it is the difference between a mistake and an incident.

How we limit what an agent can reach

• Its own account. Never a shared login and never an admin user. When something goes wrong, the logs show exactly which actions were the agent’s.
• A short list of tools. The agent can call the five functions its job needs, not the whole API. "Create draft invoice" is a tool; "run any database query" is not.
• Read and write kept apart. Many agents only need to read most things and write to one place. We give them exactly that.
• Limits on volume and spend. A cap on how many actions per hour and how much it can spend on model calls, so a loop or an attack cannot run up a bill or flood a system overnight.
• A full record. Every input, decision and action is logged with the reason the agent gave, so you can always answer "why did it do that?".

Draft first, then act

For a new agent we almost always start in draft mode. For the first few weeks it prepares everything and does nothing. People compare its drafts with what they would have done. Once the drafts are consistently right, we let it act on the low-risk cases and keep approval on the rest. Trust is earned per type of task, not granted all at once.

Questions to ask whoever builds your agent

• Which account does the agent use, and what exactly can that account do?
• Which actions need a person to approve, and who is that person?
• Where is the log of what it did, and who reviews it?
• What stops it running a thousand times in a loop?
• How was it tested against someone trying to trick it?

If the answers are vague, the agent is not ready for real data.