concept
Meaningful Human Control
Control that links autonomous decisions to relevant human reasons and informed human responsibility.
Meaningful human control asks what makes an automated action something humans can responsibly govern. The 2015 autonomous-weapons open letter uses it to qualify a proposed prohibition. Santoni de Sio and van den Hoven’s account proposes two necessary conditions (pp.6–9).[2][1]
| Condition | Question for the whole system | Editorial teaching example |
|---|---|---|
| Tracking | Does behavior respond to relevant human reasons and facts, including when circumstances change? | An approval process fails this test if it treats a reason to cancel as irrelevant. |
| Tracing | Is there a relevant person who understands the system’s capabilities and effects and recognizes responsibility for its use? | A named approver may understand their duty while overestimating what the system can do. |
The examples illustrate the account, rather than report measured failures. Tracing concerns informed responsibility across design and use, beyond identifying someone to blame. Tracking is broader than recording a person’s button press. The authors allow control through earlier design decisions and appropriate operating restrictions, rather than requiring a person to initiate every action in real time.[1]
Control and moral acceptability
The authors warn: “Meaningful human control is a necessary but not sufficient condition for a system to be morally or societally good.” (Tracking, p.8.) Control can serve wrongful goals. Which people’s reasons matter, and what level of responsiveness is adequate, remain questions about norms and context.[1]
This connects Machine Ethics to the responsibility of designers and users, and Society-in-the-Loop to whose reasons count. These are editorial comparisons, not evidence that the proposals are equivalent. The philosophical conditions supply a design vocabulary, rather than a benchmark, certification or proof of safety.
A willing agent and a responsible human
Corrigibility asks whether an agent preserves and assists correction. The Off-Switch Game studies a narrower incentive to wait for an informative human decision. These are editorial connections to tracking and tracing: an agent’s willingness to accept intervention does not establish that the human understands the system or that the relevant people’s reasons govern it. Conversely, assigning informed responsibility does not prove the agent will preserve the intervention channel.[3][4]
Sources
- Meaningful Human Control over Autonomous Systems: A Philosophical Account · Source record src-201 · Back to claim ↑1 ↑2 ↑3
- Autonomous Weapons: An Open Letter from AI & Robotics Researchers · Source record src-199 · Back to claim ↑1
- Corrigibility · Source record src-202 · Back to claim ↑1
- The Off-Switch Game · Source record src-203 · Back to claim ↑1
Pages that link here
Last updated 2026-10-10