Failure: Dual-Mandate Genesis
The Atomic Energy Commission combined the mandate to develop nuclear technology with the mandate to regulate its safety in one agency, and predictably subordinated safety to development until the 1974 split into the NRC and ERDA — arguably the single most consequential historical lesson for present-day AI governance.
What decision changes?
Where a body both promotes and constrains the same technology, do not audit the combined body harder — split the promotional and safety functions structurally at founding, with independent funding and statutory (not negotiated) evidence access.
Most institutional failures involve decay: something that once worked stops working. This one is different — some watchdogs are compromised from the day they are founded, because the same body is charged with both promoting a technology and policing it. No capture needs to occur, no drift needs to be detected; the conflict is in the founding charter.
The flagship case is the United States Atomic Energy Commission. From 1946, one agency held both the mandate to develop nuclear technology — a national priority with enormous prestige and budgets attached — and the mandate to regulate its safety. The outcome was exactly what the structure predicts: for nearly three decades, safety concerns were consistently subordinated to development goals inside the same organizational hierarchy, not through any conspiracy but through ordinary bureaucratic gravity — promotion had the mission, the money, and the career paths. In 1974 Congress gave up on reforming the combined body and simply split it: a Nuclear Regulatory Commission for safety, a separate administration for development. Japan repeated the original error and paid for it decades later: the pre-Fukushima “nuclear village” of interlocking utility, regulator, and ministry personnel is a textbook dual-mandate arrangement. The private-sector version is Arthur Andersen, auditing Enron’s books while collecting lucrative consulting fees from the same client — remedied afterward not by more auditing but by creating a structurally separate, differently funded overseer (the PCAOB).
The pattern’s remedy is as consistent as the pattern itself: structural separation at founding — different bodies, different funding, different appointment paths — never merely auditing the combined body harder. Adding oversight layers inside the same incentive structure adds gears without changing what the machine does.
This is arguably the single most direct historical warning for AI governance being built right now, because the pre-1974 arrangement is being reproduced in several load-bearing places at once. Frontier labs evaluate the safety of the very systems they are racing to deploy. National AI safety institutes sit inside ministries whose statutory mission is industrial growth — the U.S. institute’s 2025 renaming to the Center for AI Standards and Innovation made the promotional half of its mandate explicit. Standards bodies drafting the technical rules for regulatory compliance are substantially staffed by the regulated industry. And third-party evaluators typically receive model access through negotiated agreements on the developer’s terms, rather than anything like subpoena power. Each of these can be defended individually; together they reconstruct, with some fidelity, the arrangement nuclear regulation had to be broken apart to escape.
One of eleven historical case studies in Institutional Genesis, Memory, and Decay — see the overview for the full life-cycle map, or read the complete appendix.
What would count as evidence?
Arthur Andersen's simultaneous audit and consulting relationship with Enron produced the same failure privately, remedied by the PCAOB as a differently funded, differently appointed corrector rather than a second audit layer inside the same incentive structure; the pattern currently recurs in lab self-evaluation of frontier systems, national AI safety institutes housed in growth-mandated agencies, industry-populated AI-standards bodies, and negotiated (rather than statutory) third-party model evaluator access.