Institutional Genesis, Memory, and Decay: Historical Case Studies
Every safety institution we now take for granted — drug approval, aircraft certification, independent accident investigation, constitutional limits on power — was once missing. Someone built it, under specific pressures, and it either survived or it didn’t. This appendix reads eleven such histories and asks three plain questions of each: how did the institution come to exist, what kept it honest and working once it existed, and what exactly broke when it failed?
The answers cluster. Institutions that check power or catch errors are almost never designed from theory in advance. They are built in one of three ways: because someone’s own money was already exposed to hidden risk (Lloyd’s ship registry, born from insurers’ losses); because a disaster killed enough people to force reform (American drug regulation, built one body count at a time); or because the threat was constant and could not be forgotten (the Dutch water boards, which never needed a founding scandal because the sea never stopped rising). Once built, they stay alive only through active maintenance: preserving evidence before anyone has the power to act on it (flight recorders came before the modern aviation regulator had teeth), refusing to let anything operate without passing inspection (certification backed by insurers who will not cover the uncertified), writing the rules into the thing itself so successors inherit them (the GPL software license travels with the code, whatever the new owner intends), and deliberately re-teaching rare dangers to people who never lived through them (Venice renegotiated the doge’s contract at every succession, for centuries).
And they fail in a small number of recurring ways. Rules decay when the generation that remembers why they were written retires — Depression-era banking constraints eroded on roughly a human career’s timescale, and 2008 followed. Regulators born with two jobs — promote the industry and police it — reliably sacrifice the policing, as the U.S. Atomic Energy Commission did until Congress split it in two in 1974; this is arguably the single most direct historical warning for how AI oversight bodies are being chartered today. And sometimes a new kind of power simply appears faster than any existing check can adapt: Rome’s constitution had survived three and a half centuries of stress, but it had no mechanism for a general whose legions were loyal to him personally, and Caesar crossed the Rubicon.
One asymmetry matters more than all the parallels. The strongest safety regimes humanity has built were paid for in disasters that were survivable — the polity endured, learned, and reformed. That path assumes you get to fail first and legislate afterward. If a failure with a sufficiently capable AI system is unrecoverable, the most productive route to safety institutions in human history is unavailable precisely when it is needed most. That is why the cases that did not require a catastrophe — self-interested vigilance, constant visible threat, memory that renews itself, and rules entrenched against their own removal — carry more weight for AI governance than their share of the historical record would suggest.
Each card below covers one mechanism through one historical case. For the complete narrative with sources, read the full appendix on site or download the PDF.
Historical case studies (11)
Genesis from Money Already at Risk
Lloyd's Register (1760) shows the easiest correction mechanism to build: one a self-interested counterparty would build anyway, because their own capital is exposed to hidden quality.
Genesis from the Catastrophe Ratchet
U.S. pharmaceutical regulation (1906, 1938, 1962 Acts) was assembled one body count at a time, each expansion of regulatory reach following, never preceding, a demonstration that the previous reach was insufficient.
Genesis from Chronic, Self-Refreshing Threat
Dutch water boards, some tracing to the thirteenth century, never needed a founding scandal because flooding was continuous, not occasional — the hazard refreshed faster than institutional memory could decay.
Evidence Preservation Before Authority
Flight recorders and the independent NTSB investigative function, plus blameless near-miss reporting (ASRS), preceded and outlasted the FAA's full enforcement authority — a weak correction system becomes a stronger one primarily by preserving evidence and widening plurality before it hardens any handle.
Selection Gating and the Certified Basin
Airworthiness certification paired with the near-universal requirement of insurance produces a genuine selection basin: an aircraft that fails certification cannot be deployed, because no insurer will cover it and no airport will schedule it.
Constraint Inheritance Across Successors — the GPL
The GNU General Public License is the clearest existing engineering solution to a narrow successor problem: the constraint travels with the artifact through copyright, a strong distributed enforcement lever, rather than depending on the successor's stated intent.
Memory Refresh Through Succession
Venice's roughly thousand-year persistence came from converting a rare, long-horizon hazard into frequent, short-horizon surrogate events — the doge's promissione ducale renegotiated at every succession, and lot-and-vote elections designed to make office capture impractical.
Entrenchment and the Corrigibility Paradox
The 1933 Enabling Act shows a correction channel used, with complete formal validity, to abolish itself; postwar Germany's Article 79(3) (Ewigkeitsklausel) responds by placing the correction channel's own integrity conditions, not any current policy, outside the ordinary amendment process.
Failure: Reform Decay
Glass-Steagall era banking constraints, built from Depression-era catastrophe, eroded on roughly the timescale over which the generation that lived through the founding catastrophe left the relevant institutions — culminating in repeal in 1999 and a reproduced failure in 2008.
Failure: Dual-Mandate Genesis
The Atomic Energy Commission combined the mandate to develop nuclear technology with the mandate to regulate its safety in one agency, and predictably subordinated safety to development until the 1974 split into the NRC and ERDA — arguably the single most consequential historical lesson for present-day AI governance.
Failure: Capability Jump Outruns Correction Latency
The Roman Republic's correction architecture was stable for roughly three and a half centuries until the Marian military reforms created a new class of causal power — legions personally loyal to a general — for which no correction channel existed, demonstrated decisively by Caesar crossing the Rubicon.