Al.Qaeda.net

49. A Human Override

136
49. A Human Override
Journal Entry
The hand above the machine must belong to a person with evidence,
time, authority, and the protected right to say no.
A human override is not a red button.
It is an entire culture arranged around the possibility that the
system is wrong.
The button matters.
So does the person authorized to press it.
So does the training that teaches when.
So does the log that records why.
So does the protection against retaliation.
So does the investigation after the fact.
So does the budget for testing the override under real conditions
instead of assuming it works because the diagram says it does.
Governability is often described as the ability to disengage or
deactivate an AI system that behaves unexpectedly. [6]
That is the technical layer.
The political layer is whether leaders can stop a capability after
investing prestige in it.
The economic layer is whether a company will disclose a
dangerous failure when a contract depends on confidence.
The military layer is whether an operator can interrupt a mission
without being treated as the failure.
The psychological layer is whether a person believes his own
perception when the machine disagrees.
The human override must exist at every layer.
A worker can challenge an automated employment decision.
A defendant can challenge an algorithmic risk score.
A doctor can reject a clinical recommendation without losing
access to the system.
A commander can slow an AI-supported operation.
A citizen can discover when automated systems materially
affected a government decision.
An independent body can inspect high-risk models.
A legislature can prohibit a use even when the technology is
impressive.