Safety engineers and eval builders
Engineer / Evals Path
A route through the measurement and artifact questions that could change deployment practice.
Experiment line · start here for operational hooks
Toy simulation — capture theater
The fastest map from bridge name to CCI refusal signal and minimum handle level — including the case where reported correction acceptance stays high while true uptake collapses.
What this path is for
You build evals, traces, or deployment gates and want operational hooks — what to measure, what falsifies a pass, where bridges still carry weight.
You will leave knowing:
- How boundary discovery and grounding viability show up in audit design.
- What correction-channel integrity means as a causal test, not a survey.
- How the experiment lines bound (not prove) bridge claims — build order and negative ledgers matter.
- What a deployment gate artifact expects in the worked example (Appendix D).
Time: ~2–3 hours following the ordered list above.
PDF companion: Parts II and VI (boundaries, correction) → safety-case / artifact chapters → Appendix D.
Read in this order
Cards, chapter toys, experiment lines, and book chapters — curated for this audience. Experiments are sanity checks only; negative results are part of the reading.
What the Book Is Not Claiming
Sanity checks and bounds — not frontier certification.
Alignment as a Measurement Problem
Find the effective optimizer before evaluating goals.
Boundary Discovery
Trace the operative controller — model, memory, tools, loop.
The Boundary Error
Align the object whose dynamics matter — not just the model.
Chapter 1 — Scaffold Misuse
Model-only eval passes; scaffold misrepresents the world and repurposes output.
Scaffold Misuse
Good model, bad loop — input and output control without jailbreaks.
Composite Agency
Unit discovery when the agent is a coalition.
Chapter 9 — UAD coalition board
Hands-on composite boundary discovery.
Grounding Viability
Dashboards that stay green while bearers drift.
Correction-Channel Integrity
Causal uptake tests, not compliance surveys.
A Minimal Certification Schema for Moving Boundaries
Six conditions before grow/split/merge/successor.
Adversarial Agency Tests
Perturbation battery for strategic opacity.
Deployment Gate
Traces, handles, thresholds, rollback authority.
A Worked Example: The BioShield Deployment Gate
BioShield — concrete gate artifact walkthrough.
Bridge Assumptions
Which MB bridges your eval must discharge or bound.
Toy simulation
CCI refusal semantics and minimum handle level.
Embedded audit simulation
UAD + scoped CCI + deploy gate — negative ledger first.
Goal-agent simulation
Light tier collapse, honest twins, escalation ladder.
Lab-layer simulation
Pipeline lab — deep vs light tier under pre-registered batteries.
Evidence and Uncertainty
Record negatives; bound what the eval may claim.