Ask almost any team how they manage the risk of an AI agent, and the answer arrives quickly: "There's a human in the loop."
It's a reassuring phrase. It suggests judgment, oversight and a person who will catch the mistake before it matters.
In practice it often means something much thinner: someone, somewhere, clicks "approve."
A human in the loop is not a design. It's a placeholder for one.
Three ways the loop fails
The human becomes a rubber stamp. An agent that produces forty items a day for review gets careful attention in week one. By week six it produces four hundred, the reviewer has a queue and a deadline, and nearly everything is approved. This isn't negligence. It's what happens to any person asked to check a system that is right most of the time. Attention fades exactly as trust in the system grows, and the review becomes a delay rather than a control.
The human is in the wrong place. Many loops put the person at the end: here is the output, approve it. By then the agent has already chosen which data to read, which records to change, which customer to contact. The reviewer sees the answer, not the decisions that produced it. A review of the output is not a review of the action.
The human absorbs accountability without authority. When something goes wrong, two explanations become available at once: "the agent did it" and "the reviewer approved it." Each points at the other. The reviewer had no real way to see what the agent did, and the agent can't be held accountable at all.
An agent can be given an identity, credentials, permissions and a task. That answers the question "what acted?" It can never answer "who is accountable?" Only a named person can, and only if they had the information and the authority to decide.
Review is not governance
Review is an activity. Governance is a design.
Review asks: did someone look at this? Governance asks something harder: who was allowed to decide what, at which point, on what evidence, and where is that decision recorded?
A system can have a great deal of review and very little governance. That's the pattern in most early agent deployments: approvals everywhere, and no one able to say afterward why a particular action was allowed.
What designed intervention looks like
Putting people in the loop well is less about adding approvals and more about deciding where human judgment changes the outcome. We think about it in five principles.
1. Place intervention points by consequence, not habit. Not every step deserves a person. The ones that do share a few traits: they're hard to reverse, they affect people outside the team, they move money or change a system of record, or they're new territory for the agent. Put human judgment there, and let the agent run everywhere else.
2. Give the person what they need to decide. A reviewer who sees only the final answer is guessing. A reviewer who sees what the agent was asked to do, what it looked at, what it intends to do next and what the alternatives were is exercising judgment. The quality of the intervention depends on the quality of the context.
3. Give the person real options. "Approve" and "reject" are not enough. People need to be able to amend, escalate, ask for more information, or stop further actions. That last one needs care. The instinct in most safety frameworks is a kill switch, but stopping an agent halfway through a payment or a record update can leave things in a worse state than the problem you were trying to stop. What operations need is the ability to stop new actions at the right scope, let work in flight reach a clean stopping point, and route everything after that through a person.
4. Record every intervention as a decision. Who decided, when, on what information, and why. That record is what turns "someone approved it" into accountability. It is also what an auditor, a regulator or your own incident review will ask for first.
5. Check whether the humans are adding anything. If a reviewer overrides the agent zero times in a thousand, one of two things is true: the step doesn't need a human, or the human isn't really reviewing. If they override it constantly, the agent isn't ready for the work. Either way, the pattern tells you something, but only if you look.
The right question
"Is there a human in the loop?" is the wrong question, because the answer is almost always yes.
The better question is: what is the human for?
If you can say exactly which decisions each person owns, what they see when they make them, what they're able to do, and where their decision is recorded, you have a design. If you can't, you have a person standing next to an automated system, carrying the accountability for it.
People deserve better than that. So do the systems they're asked to vouch for.
In the next article in this series: why accuracy, the number every AI evaluation starts with, is the wrong metric to stop at.