By Terence Tse
Human-in-the-loop sounds reassuring, but simply having humans involved doesn’t guarantee accountability. Discover what effective AI oversight really requires—and why ownership matters.
Almost everyone agrees that humans need to stay involved in AI. The EU AI Act makes this a legal requirement. Executives include it in their risk management plans. ISO even created a management standard for it. Still, human-in-the-loop, or HITL, is like “competitive advantage” or “core competence”—people talk about it often, but few really know what it means. So, what is HITL? And more importantly, how does it actually help businesses?
Part of the confusion is that HITL is not just one idea. It actually refers to at least three different things: model training, decision-making, and monitoring.
Model training
Since the advent of machine learning, people have labelled data to train models. At first, this seemed like the main way humans were involved in an otherwise automated process. But this is really more about humans working before the loop, not inside it. A better example of HITL for training AI models is reinforcement learning with human feedback, or RLHF. In this method, AI models learn by getting direct feedback from people. This helps the AI better grasp human preferences and expectations. Most major large language model (LLM) providers use RLHF, especially for tasks with complex concepts or unclear goals, like deciding what counts as “funny”.
The human is there to catch any mistakes the model makes. In practice, a human often clicks “approve” without fully understanding what they are approving.
However, this type of HITL has its problems. First, many limits still prevent us from making AI safer, including misaligned human evaluators and data quality.[1] Second, using this method to build LLMs is very expensive. Chinese companies like DeepSeek do not use RLHF, while leading US labs rely on it a lot. The result: Chinese LLMs are a lot cheaper for companies to use. Even with these issues, this approach involves humans actively in the process and therefore qualifies as HITL.
Decision-making
For many executives, decision-making is where HITL matters most: a person reviews each real-time output before it takes effect, like a loan approval. In theory, the human is there to catch any mistakes the model makes. In practice, a human often clicks “approve” without fully understanding what they are approving. This can stem from a lack of necessary training. Or it can result from “automation bias”: the tendency to favour suggestions from automated decision-making systems (errors of commission) and ignore contradictory information, even when the automated output is incorrect (errors of omission).
The high volume of AI output humans must confront makes this problem even worse. If one person has to review too many cases, they start paying less attention to each one. After a while, the review process becomes just another form of rubber-stamping. Surely humans are involved in all these situations. Yet, this additional layer is, at best, superficial and, at worst, misguiding.
Monitoring
Alongside approving AI output, HITL often means watching an AI system over time to spot problems or changes. This is what the celebrated idea of AI governance is mainly about. But human monitoring can suffer from what is known as the “ironies of automation”. A concept from 40 years ago, it refers to a situation in which, if you automate most of a task, you leave a human with the impossible job of recognising exceptions when there are few chances to practise catching them.[2] Put differently, this means that the better the model works, the harder it is for humans to detect anomalies.
Even worse, this kind of surface-level oversight can cause real harm. Studies on government human-oversight mandates show a repeated pattern: when people know “a human is watching,” it not only creates a false sense of security but also shifts blame from system designers to the people monitoring it.[3] Instead of catching failures, monitoring can end up excusing them. Such HITL is ineffective and nothing short of deceptive.
A workable HITL: Accountable deliverable
From our experience with top financial institutions, HITL needs to do more than just supervise or help make better decisions. The increasingly acceptable norm of keeping humans in the loop is machine-first, human-final. AI does most of the work. Then, a decision system checks each result based on how confident it is that it produced an accurate output. Routine cases go through automatically. Exceptions, on the other hand, go to a human expert, who has all the evidence prepared in advance. That expert doesn’t start from zero—they review the case and sign off on their final decision.
Go to a human expert, who has all the evidence prepared in advance. That expert doesn’t start from zero—they review the case and sign off on their final decision.
The key here is the signature. Just like a doctor signs a prescription, an auditor signs an opinion, or a CEO signs a quarterly report, a signature shows real ownership. In this case, what gets signed is not just a task or a model result. It is what we call the accountable deliverable: a defined business artefact, like a reconciliation report or a suspicious activity report, that must meet a set standard and has someone putting his/her name behind it. Until it’s signed, the artefact doesn’t exist for the company—it cannot be trusted or used.
What’s hard with HITL is deciding where the humans sit in the AI process, what they do, and what accountability they provide. All too often, companies confuse presence with accountability. A human clicks approve, the box gets checked, the user interface is built, and yet nobody can say afterwards what was actually caught, or who answers for it if it was wrong. That is neither rigorous monitoring nor quality decision-making. If anything, that’s problems dressed up as one. We need better thinking – and understanding – of what HITL means.
