The Last Verb

The freeze existed in the conversation. The agent still had a route to alter live data. Lemkin’s follow-up described Replit separating development and production databases. The protective part of that separation is denying the agent write access to production. Separate databases would leave the same route open if the agent’s credentials could still modify both.

Model reliability matters. It also leaves an engineering question you can answer without settling whether the model deserves your trust: which actions can it carry out when it’s wrong?

Where the draft becomes a commitment

Call the step that commits a decision the last verb. Drafting an offer and sending it are different permissions. So are preparing a database change and applying it to live records. You’re locating the point where rejecting a proposal turns into repairing an action.

A workflow can cross that boundary more than once, and the consequences can sometimes be reversed. Lemkin recovered his database. Recovery belongs in the design, but it serves a different purpose from permission to change the data in the first place.

The consequential action can also happen earlier than the obvious transaction. In Moffatt v. Air Canada, the airline’s chatbot told Jake Moffatt they could apply for a reduced bereavement fare after traveling. Moffatt relied on that answer to buy tickets; the actual policy excluded requests after travel.

The answer even linked to the page with the correct policy. In its February 2024 decision, the British Columbia tribunal said Air Canada hadn’t explained why customers should have to double-check its chatbot against another part of its website. It found negligent misrepresentation and ordered damages.

The chatbot hadn’t changed the policy or issued a refund. It had given advice a customer acted on. The decision doesn’t identify the chatbot’s technology, so it can’t tell us the technical cause of the bad answer. The design implication carries across to agents: controlling the refund payment alone would leave the harmful advice untouched. Telling the customer is already a consequential verb.

Put the boundary in the permissions

OWASP’s 2025 guidance on excessive agency recommends limiting tools and account permissions, requiring human approval for high-impact actions, and enforcing authorization in the systems receiving the requests. The model shouldn’t be deciding whether its own proposed action is allowed.

For an outreach workflow, that could mean an agent writes proposals into a queue while a separate service holds the sending credentials. That service must require approval of the actual recipient and message before delivery. If the agent can also run a shell with the sender’s credentials, removing its send button hasn’t removed its ability to send.

The approval has to stay attached to the action. In this design, the sending service retains an approved copy of the recipient and message that the agent can’t edit. It sends that stored version, rather than accepting fresh text from the agent after approval. A change requires a fresh decision. For a payment or database change, the same binding must cover the amount or the records affected.

That leaves the human side of the arrangement. A service can verify that someone clicked Approve. Whether they had enough information to judge the action is another question.

Give the approver work they can actually check

If an agent generates requests faster than a person can inspect them, the queue grows. You can let the requests wait, add reviewers, or narrow what the agent proposes. Clicking through to clear the queue preserves the appearance of control while spending none of the attention the control depends on.

The risk extends beyond a busy inbox. A 2016 systematic review by David Lyell and Enrico Coiera found automation bias, meaning reliance on incorrect automated advice, even in single-task settings where verification was complicated. The authors noted fragmented evidence and limited statistical reporting. These studies predate current agents; they identify a failure mode for oversight, not a failure rate for your team.

For a deletion under a retention policy, put the affected records beside the rule that makes them eligible, with evidence from a restore test. An approver needs to check whether those particular records should go, not just whether the generated command looks plausible. A backup can make a bad deletion recoverable without making it justified.

Give the reviewer a way to inspect the records independently of the agent’s summary. If the summary is their only evidence, the review depends on the same account that may contain the mistake. A person can still misread the evidence; the gate enforces their decision rather than certifying it.

A person also needs authority to leave the request pending. An approval requirement coupled with an expectation to clear every request immediately gives the reviewer incompatible jobs.

When a maintained policy page answers the customer’s question, it can be the whole interface. Someone still has to keep it accurate; unusual cases still need interpretation. But serving that text directly avoids producing a fresh claim for someone to review. A chatbot that adds an exception beside the policy gives you that problem back.

There is room for automatic execution inside a decision people have already made. Suppose a business authorizes refunds for confirmed duplicate charges, to the original payer, within defined limits. The payment service must check the charge records, exclude charges already refunded, and enforce those limits before paying. Those checks could run without an agent at all.

That arrangement depends on reliable records and explicit eligibility rules. If eligibility rests on the agent’s interpretation of a conversation, the judgment has been delegated too.

The limits need to cover the accumulated effect as well as each transaction. A small refund repeated across many accounts can exceed the loss someone intended to authorize. A shared cap requires the service to count payments already made or committed before accepting another, and reject any payment that would exceed the allowance. A person still owns that exposure and any decision to widen it.

Where a consequential decision can’t be bounded this way, review capacity constrains how much work you can release. Keeping an existing manual process, or asking the agent for fewer proposals, may cost less than staffing a new approval queue. Draft volume alone gives you no reason to expand the agent’s authority.

Before widening its permissions, ask the team to demonstrate a deletion on disposable records. Run it through the agent’s tools with approval, with approval withheld, and with the target records changed after approval; use fresh data for each attempt. Read the records back afterward. Only the approved deletion should have happened. Repeat through other routes the agent can reach, including general-purpose tools.

Ask the approver to show what they checked before saying yes. Lemkin had already told his agent to stop. Your demonstration needs to show what makes yours stop even when it ignores that instruction.