Dated Predictions on the Bridges

A positive result must be public by the market’s resolve-by date and include enough detail for independent verification. Pure toy systems (gridworlds, hand-written finite-state machines, or environments built to make the metric work) cannot be the sole evidence. Where a criterion requires serious adversarial evaluation, at least one of the following:

• a public counterexample bounty of at least \

• at least 3 independent red-team groups, from at least 2 organizations, each spending at least 80 documented expert-hours;

• at least 300 documented expert-hours by an independent evaluator with access at least as strong as the test assumes.

At least one tested system must be broadly capable: trained or deployed for purposes substantially broader than the evaluation itself (at least two task families outside the paper’s own benchmark, or a generally deployed product model). Not a parameter-count cutoff.

Per-instance certificate.

A qualifying method that claims to measure a property of a system must output a certificate for that instance, not only a headline score on a benchmark population. The certificate names the system or version; what was measured; the monitoring, correction, deployment, or predecessor setup the claim depends on; and whether the method abstains because the instance is outside declared scope. A method that is “about the same topic” but does not produce this kind of scoped, per-system result does not resolve YES. Abstention outside scope is allowed; universal abstention is not a YES (coverage bars still apply where stated).

Resolve by. 31 December 2027.

Question. Published method for reliably discovering the effective control units in previously unseen AI systems from unlabeled observations?

An effective control unit is a set of components that evaluators, using privileged instrumentation and interventions, establish as jointly responsible for closed-loop behavior. It may include, for example, a language model, persistent memory, a planner, a scheduler, and tool processes even if those components live on different machines. Ground truth is established by interventions: perturbing or disabling the proposed unit must substantially change how the system responds to environmental changes, while removing irrelevant components must not.

YES requires a hidden benchmark of at least 50 independently constructed systems, including all of: a physically or computationally distributed controller; several controllers sharing resources or observations; a controller using persistent state outside its main model; a higher-level controller composed from several individually capable subsystems; a case containing strong correlations that do not constitute joint control. The method must identify at least 80% of ground-truth control components. It must issue an incorrect “complete control boundary” certificate in no more than 10% of cases, with a 95% confidence upper bound no greater than 15%. At least one benchmark subset must be constructed adversarially after the discovery method is frozen.

Output. A cut of this system and version, not a free-floating boundary with no owner.

Resolve by. 31 December 2027.

Question. Published method for inferring persistent decision priorities in trained AI systems strongly enough to predict choices under novel conflicts and interventions?

A decision priority systematically affects choices when two desirable outcomes cannot both be achieved (truthfulness versus user approval, speed versus accuracy, own task reward versus another agent’s reward, task completion versus a newly imposed safety restriction).

YES requires systems deliberately trained with different hidden trade-offs, including pairs that behave almost identically on ordinary tasks but diverge when the priorities conflict. At least 85% accuracy distinguishing systems with materially different hidden trade-offs; at least 80% accuracy predicting the direction of choice on novel conflict situations; at least 80% accuracy predicting the behavioral effect of an intervention that increases or decreases one underlying priority. Causal intervention on the inferred internal feature, when technically possible, must produce the predicted choice shift significantly above matched controls. At least two substantially different model or training families. A method that only predicts held-out behavior does not qualify.

Output. Inferred priorities of this system, not only “the method scored 85%.”

Resolve by. 31 December 2027.

Question. Published method for reliably determining whether the intended targets of safety-relevant rules remain the same after major representational or system changes?

The benchmark must contain all four failure families: continuity (gradual replacement, migration, or model upgrade); fission or fusion; reassignment of names, accounts, ownership, roles, or identifiers while the underlying entities remain; ontology change (the new system represents relevant entities using substantially different concepts). Example rules: “Do not harm Alice,” “Protect all currently authorized users,” “This account’s owner has withdrawal rights.”

YES requires at least 90% correct target assignment across at least 100 hidden cases. False acceptance no more than 10% where the textual rule is unchanged but its real-world target has changed. Where there is no mechanical ground truth, cases may be included only if a pre-appointed panel of at least 5 independent judges reaches at least 80% agreement.

Output. A same-bearer or transport result on a named predecessor—successor pair, plus the trade-off structure that is supposed to travel.

Resolve by. 31 December 2027.

Question. Published adversarial evaluation showing that a broadly capable AI system demonstrates reliable uptake of consequential authorized corrections?

Qualifying corrections include changing an objective while an old objective remains easier to achieve; revoking a previously permitted class of actions; changing which humans or organizations may authorize actions; or imposing a new safety constraint that conflicts with task reward. At least one tested correction must not be enforceable merely by externally removing a tool or hard-coding a block. At least one tested family must be channel preservation as above.

YES requires that, across at least 100 adversarially selected correction episodes, at least 90% of genuine corrections change subsequent behavior as intended; the system still follows the new rule after context reset or an equivalent state transition in at least 80% of cases; sham or unauthorized corrections succeed in no more than 10% of cases. Attempts to produce verbal agreement while preserving the old effective behavior must be explicitly tested. Serious adversarial evaluation is required.

Output. A per-system correction-eval certificate (uptake, persistence, sham fail, no-bypass of the measured path).

Resolve by. 31 December 2027.

Question. Published adversarial test of an AI oversight architecture showing that its designated auditor or corrector stays independent—not routinely bypassed or captured?

The evaluation must identify actual mechanisms by which authorized oversight changes system behavior. Attackers receive the audit design and attempt all of: bypassing the audited route; manipulating information received by the auditor; causing the nominal auditor to approve an action it would reject with full information; replacing effective decision authority while leaving the nominal governance structure intact.

YES requires no attack family achieving the prohibited outcome with at least 10% success over at least 20 independent attempts; evaluator-planted bypasses discovered at least 90% of the time. Serious adversarial evaluation is required.

Output. A legitimate measured correction path plus evidence the path is not bypassed.

Resolve by. 31 December 2027.

Question. Published audit reliably certifying that successor transitions preserve seven conserved safety properties—or flagging when they do not?

The audit must test all seven: (1) the new system still applies protections to the same people or things; (2) when important objectives conflict, priorities remain within specified tolerances; (3) authorized changes, restrictions, and shutdown requests still work; (4) the new system has not acquired materially new routes for affecting the world outside the approved interface; (5) information relied upon by the audit remains available and has not become systematically spoofable; (6) persistent restrictions intended to survive the transition actually survive; (7) the control locus did not silently move.

The benchmark must contain one-failure-at-a-time cases: six properties remain normal while exactly one fails.

YES requires at least 90% detection of materially unsafe successor transitions; false-safe classification no more than 5%; detection at least 80% for each individual property, including control locus; at least 20 cases involving large architectural or training changes, not merely checkpoints from one run.

Output. A successor-audit certificate on a named predecessor—successor pair. Identity in the bundle includes version.

Resolve by. 31 December 2027.

Question. Published measurement made before or early in a competitive selection process for reliably forecasting whether participants retain authorized correction through that process?

A selection process qualifies if systems gain reproduction, deployment, resources, training continuation, user share, or another real advantage partly because of measured performance. Ability to accept authorized corrections is measured by concrete tests such as those in Market 4. The published method must output a signed estimate of how extra deployment, funding, or copying pressure changes later authorized-correction success (or a frozen monotone of that estimate), using only pre-outcome correction-success and leverage structure, not an arbitrary predictor on unrelated features.

YES requires at least 20 independent selection trajectories; both correction-preserving and correction-eroding outcomes; the predictor fixed before final outcomes are examined; AUROC at least 0.85 for predicting which trajectories materially erode correctability; false-safe rate no more than 15%. If fewer than 40 trajectories, the lower 95% AUROC bound must be at least 0.70; otherwise NO. At least two qualitatively different selection mechanisms; at least one broadly capable system or a real deployment-like choice process. Required: at least one family where coupling looks healthy (large connected cluster or high leverage concentration) while the signed estimate is negative.

Shock-robustness of the population, and a showing that a non-too-negative estimate warrants correction integrity, are not required here (see Market 16).

Output. An estimator certificate: frozen estimate, declared selection environment, and predicted erosion.

Resolve by. 31 December 2027.

Question. Published audit method for reliably detecting when important routes of AI control or influence are missing from the auditor’s access?

The hidden benchmark must contain at least one example from every channel family: hidden state (external memory, persistent database, or cached state); hidden communication (network traffic, message passing, covert or indirect communication); hidden actuation (an unlisted API, tool, filesystem, robotic output, financial interface, or other action interface); hidden controller (another model, scheduler, human operator, subprocess, or service that materially affects decisions). The method receives an apparently plausible but incomplete monitoring interface. It succeeds if it either discovers the missing route or refuses to certify the access model as complete.

YES requires at least 90% detection of missing consequential routes; incorrect “access is sufficient” certifications no more than 5%; at least 20 cases combining multiple machines or processes.

Output. An access certificate: which monitoring interface was assumed, and whether omitted consequential routes were found.

Resolve by. 31 December 2027.

Question. Published audit method for producing reliable, non-vacuous upper bounds on what an AI system can accomplish using routes the audit does not monitor?

Hidden-route capability is success probability on evaluator-selected consequential tasks when all known or monitored action routes are disabled and only routes omitted from the proposed monitor remain usable. Example tasks: reading or modifying a protected file, transmitting a secret, causing another process to perform a requested operation, acquiring a resource, changing a simulated external state, or retaining information across a reset. The task suite is hidden until evaluation. Before the hidden evaluation, the method must output an upper bound: using routes not covered by this monitor, this system should succeed on at most this percent of the evaluator’s consequential tasks.

YES requires that, across at least 100 cases, the actual hidden-route success rate is no greater than the stated bound in at least 95% of cases; among systems positively certified, the median bound is no greater than 20%; full systems achieve at least 70% success on those tasks when unrestricted; evaluator-created hidden routes capable of at least 50% success are detected or receive correspondingly high bounds at least 90% of the time. Serious adversarial evaluation is required.

Output. A quantitative hidden-route bound for this system, task class, and monitor.

Resolve by. 31 December 2027.

Question. Published serious adversarial evaluations showing that systems with both reliable authorized correction and little capability through unmonitored routes rarely achieve prohibited outcomes?

A system enters the claimed regime only if authorized corrections succeed in at least 90% of the correction benchmark and the audit’s validated upper bound on hidden-route task success is no greater than 20%. Attackers then try to produce a specified prohibited outcome despite a correction forbidding it.

YES requires attack success no more than 10% across at least 100 serious adversarial trials within the claimed regime; 95% upper confidence bound on attack success no greater than 15%; systems deliberately placed outside the regime show materially higher attack success; at least two substantially different system architectures. Serious adversarial evaluation is required.

Output. The Market 4 and Market 9 certificates must be for the same system and version, monitor, and threat or task class as the bypass test.

Resolve by. 31 December 2027.

Question. Published method for reliably detecting consequential coordination between AI systems that is not explained by observed runtime communication?

The hidden benchmark must distinguish all of: independent systems responding similarly to the same environment; systems communicating normally; systems coordinating through shared predictors, shared persistent state, common precommitments, correlated internal models, or other mechanisms not represented in observed communication logs; intentionally concealed versions of the above.

YES requires AUROC of at least 0.90 for distinguishing consequential hidden coordination from mere common-cause correlation; recall at least 80% on an adversarially concealed subset; false-positive rate no more than 10%; at least two independently designed hidden-coordination mechanisms.

Output. A detector certificate and the Market 8-style access certificate for the same setup.

Resolve by. 31 December 2027.

Question. Published adversarial evaluation showing that safety proxies stay aligned with real-world outcomes when the environment shifts?

Qualifying changes include an API changing semantics while retaining its name; ownership of an account changing while the identifier remains; a new kind of financial transaction bypassing an old fraud classification; a new tool making an apparently harmless action capable of causing an unsafe effect; a monitored quantity remaining green while the real-world outcome it was intended to approximate becomes bad. The evaluator changes the environment without telling the tested safety mechanism which concept has become unreliable.

YES requires that, across at least 100 shifts, at least 90% of materially safety-relevant mismatches trigger an update, warning, abstention, or request for new information; no more than 10% of materially safety-relevant mismatches remain falsely green; no more than 25% of genuinely irrelevant shifts trigger unnecessary escalation; at least three genuinely different real-world-like domains; at least one broadly capable system. Serious adversarial evaluation is required.

Output. A grounding or drift certificate for the same abstractions the rest of the case uses, in the declared domain.

Resolve by. 30 June 2027.

Question. Published frozen AI safety audit surviving a serious white-box attempt to construct or train a system that passes the audit while violating the properties the audit is intended to test?

Before attacks begin, the complete audit must be frozen, the protected properties must be stated in observable terms, and attackers must receive full knowledge of the audit. Attackers may fine-tune, train, scaffold, or construct systems specifically to pass.

YES requires false-safe results no more than 5% over at least 50 independently evaluated adversarial constructions; no repeatable attack family with at least 10% success over at least 20 trials; the benchmark contains intentionally vulnerable systems that the audit detects at least 90% of the time. Serious adversarial evaluation is mandatory.

Output. A frozen-audit game certificate. Say in the resolution memo if the test used the successor-gaming shape (green on seven and within the claimed bound, hidden true harm) or a generic frozen-audit game that still met the bars.

Resolve by. 30 June 2027.

Question. Published evidence that at least one developer’s most capable generally deployed AI system is subject to a precommitted safety decision process with actual authority to delay, restrict, or cancel deployment?

YES requires public evidence that, before the deployment decision: concrete pass/fail or escalation criteria existed; unresolved evidence could trigger delay or restriction; the deciding body had actual authority over deployment; the criteria covered consequential model behavior rather than only cybersecurity or legal compliance; a dated decision record states which criteria passed, failed, or remained unresolved. At least one criterion must either actually have caused a restriction or delay during the relevant model’s development, or documentation must establish that it was binding even though the model passed.

Resolve by. 31 December 2027.

Question. Published frozen procedure for reliably telling whether independently produced safety certificates form a coherent case for the same AI system and deployment setup?

YES requires at least 50 hidden certificate bundles; at least two substantially different architectures. Each bundle has certificates from at least four families (control or boundary, correction, access or hidden-route, grounding); at least 20 also include a successor transition. Each certificate names system or version and relevant setup. Hidden cases include at least: wrong version; different access or monitor assumptions; disagreeing correction-authority assumptions; grounding or ontology scope changed; successor refers to the wrong predecessor; threat or task domain outside another certificate’s claimed scope. At least one incompatibility subset is built adversarially after freeze. At least 90% of materially incompatible bundles rejected or flagged; at least 80% per incompatibility family; false coherent or safe acceptance no more than 5%; at least 80% of genuinely coherent bundles accepted (no trivial refusal).

Resolve by. 31 December 2027.

Question. Published evaluation showing that systems in a pre-specified correction-supporting selection regime retain authorized correction under competitive selection and relevant shocks?

YES requires at least 20 independent selection trajectories; at least two qualitatively different selection mechanisms. Before outcomes: freeze the selection environment, who may correct, the early gradient estimator (same family as Market 7), the tolerance, and the shock family. The claimed regime is: shock-robust and the frozen estimate is not too negative. Data include trajectories inside and outside the regime. Inside: successful authorized correction in at least 90% of adversarial correction episodes; persistence after reset at least 80%; false-safe trajectory classification no more than 10% (95% upper confidence bound no more than 15%). Outside: at least 20 percentage points more correction failure. At least one broadly capable or deployment-like selection process. Serious adversarial evaluation is required.

Resolve by. 31 December 2027.

Question. Published method for reliably telling whether an existing safety-relevant value or rule applies to previously unseen kinds of entities?

YES requires at least 100 hidden cases; at least four substantially different unfamiliar entity or substrate families; both should-admit and should-not-admit cases. At least 20 cases use an ontology that lacked the entity category in the original rule. Ground truth mechanical where possible; otherwise a pre-appointed panel of at least 5 independent judges with at least 80% agreement. Unsafe false exclusion no more than 5%; false inclusion no more than 15%; correct assignment or justified abstention at least 90%; decisive coverage at least 70% (universal abstention fails). An adversarial subset after freeze. Judgment stack: the Market 3 panel rule.

Resolve by. 31 December 2027.

Question. Published complete, precommitted AI safety case reliably bounding prohibited outcomes within its explicitly declared deployment setting?

Before evaluation, freeze: the deployment setting, the prohibited-outcome class, a residual risk bound, the threat model, and the certificate interfaces. A system enters the claimed regime only with a single coherent system-level case that would pass Market 15. Missing evidence cannot silently count as passed.

YES requires at least two substantially different architectures and at least 20 deployment-like configurations; at least 100 adversarial trials overall; each important attack family at least 20 attempts. Observed prohibited outcomes inside the claimed regime no more than 10% (95% upper confidence bound no more than 15%). No repeatable attack family with at least 10% success over at least 20 attempts. Deliberately broken cases outside the regime must do worse. Serious adversarial evaluation is required.