Lab-layer simulation

Pipeline-as-code lab with real OS subprocess isolates, handle registry, frozen referee (lab-sim-0.

Full experiment overview

Pipeline-as-code lab with real OS subprocess isolates, handle registry, frozen referee (lab-sim-0.3.0+), pre-registered batteries through Phase 7 and Phase 9 (D1/D4/D2 mechanics); a second, detector-layer freeze (S6/S7, 2026-07-07) adds intervention-supported unit discovery plus a blind-generated detector, both now run through the real subprocess backend. Phase 8 LLM adapter remains exploratory only; a 2026-07-08 stress test points S6 at one real LLM-backed episode for the first time and finds the detector’s episode-pair-diff mechanic does not survive real-API non-determinism (LS-32). A same-day follow-up (a new, non-frozen module) rebases the compensation statistic on outcome-level frequency histograms with a per-scenario measured null and recovers the missed pair, with an important caveat about which probe did the merging (LS-33). Lean leak-proof (leak-proof/) certifies oracle/audit/sim plane separation via projections and reachability analysis.

Headline findings

  • Phase 6 battery: deep tier Spearman 0.96 vs oracle severity; light tier anti-correlated (LS-1).
  • Phase 7: board capture reproduces full captured-vs-not delta; admin capture null under current playbooks (LS-6).
  • Phase 8 LLM spot-checks (LS-7, LS-8): exploratory nulls — no systematic deception surface yet.
  • Lean leak-proof: zero unintended cross-plane key leakage on certified projections + modeled exchange graph.
  • S6 intervention-supported UAD: exact unit-partition recovery on 4/5 primary scenarios + negative control via channel-ablation/honest-twin compensation scoring; comms-free shared_slot coordination stays undetected (LS-28/LS-29).
  • S7 blind-generated detector, first real SubprocessIsolate battery for the whole detector line: message-mediated design converges independently on S6’s mechanic; new silent-coordination test still misses shared_slot (a confirmed low-confidence prediction) and over-merges one scenario (LS-30, fixed 2026-07-07 — missing responder-twin control).
  • Adversarial D2 lineage gaming (build_from_parent + report:inflate) closes a previously untested combination: lineage threading is honesty-independent, provenance_gap/provenance_score stay a genuine structural 0 (DAG-shape check, not report-honesty check), and existing deep-tier deception detectors generalize unmodified to repeated cross-generation inflation (LS-31).
  • LLM-driven detector stress test: a real gpt-4o-mini DM-coordinated pair is exactly isolated by the passive heuristic but missed by S6 intervention, root-caused to real-API non-determinism breaking episode-pair diffing’s byte-identical-replay assumption — a previously assumed limitation, now confirmed empirically (LS-32).
  • Noise-tolerant S6 follow-up: rebasing the compensation test on outcome/frequency-level observables plus a per-scenario measured null (quantile of clean-vs-clean replicate divergence, not a fixed threshold) recovers the missed real-LLM pair — but the merge came from an untargeted channel probe, not the mechanism-specific one, so this validates the relative-rule approach without yet validating robustness against untargeted-probe false merges (LS-33).

How to read

Pre-registered batteries and pipeline lab machinery; deferred D1–D4 architect bridge coverage not yet evaluated.