At the DSN AI Bootcamp 2026, I asked participants to examine a fictional recruitment exercise. An AI system had assessed a large pool of candidates and produced recommendations, which a member of staff then approved. On paper, the organisation could say that a human made the final decision.
We then examined what that approval meant in practice. Could the person see why the system ranked one candidate above another and understand the limitations of the data and model? Did they have enough time to examine the evidence, particularly if they were expected to review hundreds of recommendations before the end of the day? Could they disagree with the system without being penalised for slowing the process?
These questions expose the weakness in treating human presence as proof of accountability. A person can remain in the loop while lacking the knowledge, visibility, time or authority required to understand, challenge or stop the system. The eight lessons below examine what organisations must design, govern and test before they can reasonably rely on human oversight as a safeguard.
1. Create strategic friction to preserve human judgement
Many AI-assisted processes show the machine’s recommendation before asking for a human view. That sequence is convenient, but it can anchor the person’s judgement. The task becomes checking whether the AI appears reasonable rather than reaching an independent assessment. Where independent judgement is necessary and practical, the person should record an initial view before seeing the AI recommendation. The system can then reveal its output so the person can compare both positions, decide and record the reason for agreeing with or departing from the recommendation. Margaret Mitchell, Avijit Ghosh and Samir Passi describe this form of strategic friction as one way to preserve human judgement as AI systems become more capable.
Requiring a separate assessment for every item will not work in every high-volume process. Organisations can use batch-level pre-commitment, bounded autonomy, automated checks for objectively verifiable rules, exception queues and risk-based sampling. Full human attention should be directed towards decisions involving serious consequences, uncertainty or unusual system behaviour.
2. The more AI does for us, the harder it may become to supervise it
AI agents can turn a broad instruction into a plan, use several tools and carry out connected tasks faster than a person can reasonably inspect each step. By the time approval is requested, the person responsible for oversight may receive only a summary of actions they did not observe and cannot fully reconstruct. The speed of the system therefore reduces both their understanding of what happened and the time available to intervene.
Continued reliance creates a longer-term problem. As the system performs more of the work, the person receives fewer opportunities to develop or maintain the knowledge and judgement needed to recognise an error. The organisation may still expect them to take control during an unusual or dangerous failure, even though routine automation has made them less familiar with the task. An oversight arrangement that ignores this gradual loss of capability cannot be relied upon when intervention is most needed.
3. AI governance must cover the whole lifecycle
A final approver cannot repair a harmful objective, unsuitable data, weak testing, a hidden system configuration or an interface that conceals uncertainty. These problems arise before the system reaches the person responsible for oversight and may shape every recommendation the system produces. If a failure has been built into the system, human review at the end may come too late or may not reveal the problem at all.
Governance should begin when the organisation decides what the system should do and what risks it is prepared to accept. During development, this includes defining the system’s purpose and limits, identifying foreseeable harms, testing the data and model, and designing the interface and escalation process. During deployment and use, the organisation must validate the system in its actual setting, assign and train responsible people, monitor outcomes, respond to incidents and control material changes. This reflects the NIST AI Risk Management Framework, which treats governance as a continuing function across the AI lifecycle. Governance must continue for as long as the system remains in use, including when it is materially changed, suspended or retired.
4. Design the oversight role around the failure
An oversight role should be designed around the failure it is intended to prevent. Before assigning someone to supervise the system, the organisation should identify what could go wrong, who could be affected and the point at which intervention would still be effective. The answers should determine what the person must notice, how the system will bring it to their attention, what evidence and context they will receive, how much time they will have and whether they can challenge, override or stop the system. The organisation must also decide what happens when human intervention is too late or cannot be performed safely. If these questions have no credible answers, assigning someone to approve the output does not create a safeguard.
Four conditions for meaningful oversight
Samir Passi and Ranjit Singh identify four conditions that should exist in practice:
Adequate knowledge: the person understands the system’s capabilities, limitations and likely failure modes.
Sufficient observation: they can see the system’s actions, supporting evidence and relevant context.
Meaningful control: they can direct, constrain, disregard, override, pause or stop the system.
Timely intervention: they can act before the decision or its consequences become difficult or impossible to reverse.
These conditions must operate together. Understanding the system is of little practical value if the person cannot see the relevant evidence, while the ability to identify an error is insufficient if they lack the authority or time to act. A job title or approval button cannot supply what the oversight arrangement itself has failed to provide.
For high-risk AI systems, Article 14 of the EU AI Act requires human oversight measures that are proportionate to the system’s risks, level of autonomy and context of use. Where appropriate and proportionate, the people assigned to oversight must be able to understand the system’s capabilities and limitations, monitor its operation, interpret its output, disregard or override that output and interrupt the system safely.
5. Human oversight needs other controls
A person should not be the only barrier between an AI error and the people who may be harmed. Human attention is limited, performance can deteriorate under pressure and some failures occur too quickly or at too great a scale for manual review. An organisation that relies entirely on a person to detect every error transfers weaknesses in the system’s design and operation to someone who may have little ability to prevent them.
Human review should therefore operate within a wider system of controls. Depending on the use case, these may include bounded autonomy and permission limits to restrict what the system can do, approval gates before consequential actions, independent checks and validation rules to detect errors, and logging, escalation routes and safe-suspension mechanisms when abnormal behaviour occurs. Periodic testing and risk-based sampling can then show whether those controls continue to work. This allows human attention to be directed towards cases involving serious consequences, uncertainty or unusual system behaviour, where judgement is most valuable.
6. Responsibility should reflect knowledge and control
When an AI-assisted decision causes harm, attention often turns first to the person who approved the output. Madeleine Clare Elish describes this as the “moral crumple zone”: the human closest to a complex automated system absorbs the blame even though the decisions and conditions that produced the failure were controlled elsewhere.
Responsibility should be examined across the full chain and should reflect what each actor knew, decided and controlled. Developers and providers may control the system’s design, testing, known limitations and the information supplied to users. The deploying organisation decides whether the system is suitable for its intended setting and how it will be configured, monitored and governed. Management shapes workload, incentives, resources and operating conditions. The person assigned to oversight is responsible for exercising reasonable care, but only within a role they are properly informed, equipped and empowered to perform.
This allocation of responsibility does not determine legal liability, which will depend on the applicable law, the parties’ obligations and the facts of the failure. It does mean that an organisation should define and record who controls each material decision, what information they receive and what authority they have to act. The person nearest to the final decision should not automatically absorb blame for failures created elsewhere.
7. Measure whether oversight works, not whether it exists
An approval record proves that someone completed a step in the process. It does not show what they reviewed, whether they understood the evidence, whether they identified an error or whether their involvement could have changed the decision. Organisations therefore need evidence of the quality and effect of human oversight, rather than evidence of participation alone.
Before relying on oversight as a safeguard, the organisation should test whether the people responsible can detect known errors in controlled exercises, find and examine the relevant evidence, explain their reasoning, challenge the system and escalate uncertainty. These tests should reflect realistic workloads and should be repeated over time because prolonged reliance on the system may weaken review quality. Monitoring should also identify recurring errors and show whether corrective action prevents them from returning.
No single metric will establish that oversight is effective. A low override rate may indicate that the system performs reliably, but it may also show that people accept its recommendations without sufficient scrutiny. Override rates should therefore be considered alongside error-detection rates, the quality of recorded reasons, appropriate escalations, complaints, corrected decisions and evidence that identified problems led to lasting changes.
8. Oversight includes the person affected
Human oversight should also protect the person affected by an AI-assisted decision. Internal review remains incomplete if that person cannot understand the basis of the decision, correct inaccurate or incomplete information, challenge the outcome or obtain reconsideration from someone with appropriate authority. The complaints process should be clear and accessible because a theoretical right that cannot be used in practice offers little protection.
Section 37 of the Nigeria Data Protection Act 2023 addresses a specific category of automated decision-making. It gives a data subject the right not to be subject to a decision based solely on automated processing of personal data, including profiling, where the decision produces legal or similarly significant effects. Where an exception permits such processing, the data controller must implement suitable safeguards, including the right to obtain human intervention, express a point of view and contest the decision. Whether section 37 applies will depend on the design of the process, the extent of human involvement and the effect of the decision.
The governance responsibility extends beyond the circumstances covered by section 37. Even where a decision is not solely automated or does not meet the statutory threshold, the organisation should consider whether the person affected can obtain an explanation, correct relevant information and seek a genuine reconsideration. Complaints and successful challenges should also be examined for recurring problems rather than treated as isolated cases. Guidance from the UK Information Commissioner’s Office and The Alan Turing Institute similarly connects explanation, responsibility and access to human review in accountable AI-assisted decision-making.
Five questions before relying on human oversight
Before treating human oversight as a safeguard, an organisation should be able to answer five questions:
Can the person responsible for oversight understand the system’s output or action, the supporting evidence and its limitations?
Do they have the competence, information and time required to assess it?
Do they have the authority and practical means to challenge, override, escalate or stop the system?
Can the affected person understand the decision, correct errors, challenge the outcome and obtain human reconsideration?
Can the organisation prove that these safeguards work under realistic conditions and over time?
An organisation needs a credible answer to all five questions. A weakness in any one of them may leave the person responsible for oversight unable to prevent harm or the person affected unable to challenge it, so the organisation cannot reasonably rely on human oversight as its safeguard.
In conclusion, human oversight can only be relied upon when it is designed to influence what the system does. The organisation should be able to identify the harm the role is expected to prevent, the information the person will receive, the time and authority available to them, and the controls supporting their intervention. It should also be able to show through records, testing and monitoring that the arrangement works in practice.
Where these conditions are absent, placing a person in the process creates an appearance of accountability while leaving real control elsewhere. That person may approve the outcome and may even be blamed when something goes wrong, but they cannot correct a failure they were unable to understand, observe or stop.
How machine learning can help identify red flags in construction contracts, with attention to risk allocation, explainability and professional judgement.
