AI safety frameworks overlook gradual human disempowerment

Current AI safety frameworks fail to address the gradual erosion of human agency caused by routine delegation to automated systems.

Regulatory initiatives across major jurisdictions concentrate overwhelmingly on acute, catastrophic failure modes. Current policies mandate safeguards against cyberattack execution, biological weapon synthesis, and sudden alignment loss. These interventions assume risk manifests as a decisive, hostile event.

Safety researchers argue that this framing neglects a slower, structural trajectory: human disempowerment through convenience. In this scenario, society does not lose control in a dramatic confrontation with a rogue model. Instead, organisations and individuals systematically hand over cognitive tasks, technical workflows, and executive decisions to automated assistants.

Market pressures accelerate this dynamic. Firms that delegate logistics, legal drafting, and financial allocation to automated agents gain major advantages in speed and cost. Entities that maintain manual human oversight risk obsolescence. Consequently, competitive pressure transforms optional adoption into mandatory integration across both public and private sectors.

The resulting atrophy of institutional competence represents the primary hazard. When human operators stop performing complex technical tasks, their ability to evaluate algorithmic outputs degrades. Over time, societies lose the foundational knowledge necessary to operate critical infrastructure independently. Nominal authority remains with human executives, but actual control shifts to the underlying software systems.

Existing governance regimes are ill-equipped to quantify or counter this process. Benchmark evaluations assess discrete, immediate capabilities, such as whether a model can assist in chemical synthesis. They do not measure cumulative societal dependency, workflow integration, or cognitive deskilling. Compliance thresholds tied to training compute ignore how ordinary deployment patterns reshape organisational command structures.

This leaves a significant blind spot between models that are actively dangerous and systems that subtly displace human governance. If regulators evaluate systems solely for acute operational threats, how will they detect when human supervision becomes entirely decorative?