Correction at Civilizational Scale

A civilizational control loop can fail even when no single component misbehaves, because the failure is in the aggregate's correctability, not in any one actor's intent.

What decision changes?

Do not stop at asking whether any single lab, model, or regulator behaved badly. Ask whether the loop connecting them still lets humans notice and redirect it.

Treating superintelligence risk as a story about a single rogue model misses the failure mode the civilizational frame is built to catch: alignment failures without villains. Every visible actor — lab, regulator, user, model — can behave defensibly while the loop connecting them drifts toward a state nobody can correct.

This is why the book treats model-level evaluations as insufficient on their own. A model can pass every benchmark while embedded in a loop whose selection pressures — competitive deployment speed, procurement dependence, user habituation — erode civilization’s actual capacity to notice and redirect it.

Correction at civilizational scale is therefore not solved by any single actor doing better. It requires the loop itself — training, deployment, incentives, and successor creation together — to remain inside a basin where correction is still real.