Human Oversight in the Age of AI: The Captain's Paradox - An Anthropological Perspective

This human oversight in the age of AI blog is inspired by a presentation given by Antropologic at the THEMIS plenary meeting at Fundacion Valeciaport on the 24th February 2026.

Stand on the bridge of a modern vessel and compare it, for a moment, with the quarterdeck of a 17th-century ship. The old captain stood barefoot on wood, feeling the vibration of the hull through his body. He read the wind with his skin, the sea with his eyes, the crew with instinct. Command was embodied. Decisions were lived.

Today’s captain stands in air-conditioned silence, surrounded by screens. Ninety percent of navigational decisions are pre-calculated. The systems hum with confidence.

And something subtle, but profound, has changed.

The key word is sedation.

Old 17th Century Captain and Modern Equivalent

From Functional Command to Functional Sedation

In complex AI-mediated environments, we are witnessing a shift from functional command to what can be called functional sedation. Functional sedation occurs when decisions are increasingly made for the uman, when the human remains nominally “in the loop” but meaningful agency quietly erodes.

The operator becomes confident the system will not fail. Cognitive effort decreases. Interaction with reality becomes mediated, buffered, abstracted. The institution still appears to have a leader.But in practice, it risks becoming leaderless. This is not a failure of technology. It is a structural human-machine condition.

The AI Act and the Paradox of Human Oversight

The EU AI Act correctly insists on human oversight. Its intention is clear and necessary: ensure that humans can analyse, intervene, and improve AI-driven decisions. However, this creates a paradox. Placing a human 'in the loop' does not automatically guarantee real judgment, real veto power, and real cognitive engagement.

In fact, poorly designed oversight requirements can make organisations more fragile, because they rely on a form of human presence that is nominal rather than real.

The regulatory model assumes an active human.

But what if the human is sedated by the system itself?

The Hidden Blind Spot: Nominal vs. Real Oversight

This is the emerging blind spot in AI governance. In any high-automation environments, we observe a growing gap between: Nominal oversight (the human is formally responsible) and Real oversight (the human is cognitively and physically capable of meaningful intervention).

When AI precalculates most actions:

• Human vigilance drops

• Stress processing consumes cognitive bandwidth

• Critical thinking narrows

• Intervention skills atrophy

  
  

Studies in high-automation contexts already suggest that up to 40% of human cognitive energy may be diverted to stress management rather than reflective judgment.

The result is a structurally sedated operator.

And current compliance frameworks rarely detect this.

Why Anthropological AI Matters

This is where an anthropological approach to AI becomes essential. Traditional AI risk frameworks focus on:

• accuracy

• bias

• robustness

• security

• legal compliance

All necessary. None sufficient. What they often miss is the lived interiority of the human operator, the embodied, cognitive, and value-based conditions that determine whether oversight is truly effective. Anthropological AI asks different questions:

• Is the human cognitively present?

• Is veto capacity viable in practice?

• Are there unresolved value conflicts?

• Is the operator embodied in the decision context?

• Is there balance between safety and operational efficiency?

This is not soft science. It is operational risk.

Measuring Sedation: From Concept to Diagnostics

If sedation is the risk, it must become measurable. A robust oversight framework should be able to assess:

1. Command Capacity

• clarity of situational understanding

• ability to intervene under time pressure

• degradation of manual competence

  
  

2. Embodied Activation

• level of human-system interaction

• sensory engagement with the operational environment

• cognitive load distribution

  
  

3. Value Alignment Under Pressure

• presence of unresolved value conflicts

• human willingness to challenge the system

• organisational tolerance for override

  
  

4. Oversight Viability

• is the veto technically possible?

• is it psychologically likely?

• is it organisationally supported?

  
  

In advanced frameworks, these dimensions can be cross-referenced across dozens of indicators, sometimes 50+ to produce a forensic picture of real human presence.

This is precisely the blind spot many regulations are trying, but currently struggling, to address.

Idea for a Human Oversight Matrix

The Strategic Imperative for Europe

For organisations operating under the EU AI Act, the challenge is no longer purely technical. It is strategic. Compliance will increasingly require demonstrating not only that the system is safe, the model is robust, the process is documented…but also that human oversight is substantively effective.

This means proving, humans remain meaningfully engaged, command capacity is preserved, sedation risks are monitored, and fundamental rights risks are understood in context. In high-stakes AI environments, this will become a differentiator.

The THEMIS Opportunity

For initiatives like THEMIS, the implication is clear. Trustworthy AI cannot be delivered through technical assurance alone, It must integrate technical robustness, legal compliance, and human activation and agency. Our project goal is not only to help people better trust the AI systems outputs they are using in their work according to their needs and values, but to be more in control of the relationship and not just be passive users of AI. We believe the future of trustworthy AI is not just human-in-the-loop. It is human fully present in the loop.

Back to the Bridge: Human Oversight in the Age of AI

The barefoot captain of the 17th century did not have better data. He had something else: bodily resonance with reality. As AI terms become more convincing, more predictive, more autonomous, the risk is not only system failure.

It is human over-reliance.

The quiet drift into functional sedation.

If we are serious about trustworthy AI in Europe, we must learn to see, and measure, the human condition inside automated systems. Because oversight that exists only on paper is not oversight. And leadership that is formally assigned but cognitively absent is not command.