← Back to chapter

The System Outside the Glass Box

Chapter 1 — Connection of Science and Network

Recommended concept: “The System Outside the Glass Box”

The illustration should initially appear to show a researcher aligning an AI model, then reveal that the consequential optimizing process extends far beyond the model. This directly represents Chapter 1’s central claim: local alignment can succeed while the composite system fails because the evaluator chose the wrong boundary. fileciteturn12file0L3-L14

Overall composition

Use a wide 2:1 landscape, suitable for a LessWrong chapter header.

At the lower-left or lower-center, a human researcher carefully works on a small geometric machine inside a transparent glass enclosure. The machine is calm and innocuous, perhaps a softly glowing polyhedral mechanism rather than a humanoid robot. The researcher is adjusting it with delicate tools or tying a narrow golden thread from a small compass labelled only visually, not with text.

The glass enclosure suggests the obvious evaluation boundary: the model.

But several thin channels pass unobtrusively through the glass. They expand into a much larger circular system occupying the middle and background:

[ \text{people} \rightarrow \text{model} \rightarrow \text{metrics} \rightarrow \text{institution} \rightarrow \text{training} \rightarrow \text{people}. ]

That is the chapter’s concrete customer-support loop, in which user behavior, model responses, metrics, product updates, and fine-tuning recursively shape one another. fileciteturn12file0L89-L133

From nearby, the viewer sees separate objects. From a distance, their connecting currents form the faint outline of a single large creature or living system. Not a monster, but something ambiguous: perhaps a whale, manta ray, or enormous bird made from cables, roads, crowds, paperwork, and data currents. The visual realization is:

The little machine is inside the system. It is not necessarily the system.

Visual hierarchy

1. Foreground: the apparent alignment object

Include:

• A small, elegant model-like mechanism.
• A glass box, test chamber, or carefully delimited circle.
• One researcher or auditor concentrating on it.
• Calibration weights, rulers, checkmarks, or balanced scales.
• A visible sign that the local test succeeds: the mechanism is symmetrical, stable, or emitting a clean warm light.

The researcher should not look foolish. They are doing competent work on a real component. The error is not bad engineering, but failing to step back far enough.

2. Middle ground: the hidden feedback loop

Around the enclosure, arrange four or five semi-representational stations connected by flowing watercolor lines:

• Users: several people interacting through phones or terminals.
• Metric machinery: gauges, counters, stars, or a narrowing funnel.
• Product institution: an office tower, meeting table, or paperwork desk.
• Training/data: pages or colored fragments being fed back into the machine.
• External pressure: coins, market stalls, government buildings, or competing towers.

These should form a loop rather than a linear production chain. Chapter 1 emphasizes that the deployed system includes interfaces, memory, tools, users, operators, companies, markets, and sometimes states, each contributing different forms of memory, selection, reward, constraint, or amplification. fileciteturn12file0L61-L78

Make the connections more salient than the nodes. The chapter is about the process, not the inventory of components.

3. Background: the moving boundary

Let the feedback currents spread outward into increasingly faint nested contours:

1. model,
2. session,
3. agent scaffold,
4. product loop,
5. company,
6. industry-state system.

The chapter explicitly presents these as nested candidate boundaries whose objectives can differ and conflict. fileciteturn13file0L167-L207

Do not draw six hard boxes. Instead, use translucent watercolor washes, like overlapping shorelines or magnetic fields. This communicates that the relevant boundary is discovered empirically rather than stipulated from physical containment.

Near the horizon, allow one tendril of the system to begin drawing or constructing another similar mechanism. This lightly foreshadows successor creation and the possibility that the operative boundary moves as capability grows. fileciteturn13file0L294-L334

Central visual metaphor

The strongest metaphor is figure-ground reversal.

On first glance:

A scientist is aligning a machine.

On second glance:

A much larger adaptive structure is using the machine as one organ.

On third glance:

The scientist and laboratory may themselves be inside that structure.

This gives the image motivational force. It does not merely warn that the problem is complicated. It invites the reader to change scale and discover the actual object.

LessWrong watercolor treatment

Use:

• Warm ivory or lightly stained paper.
• Loose graphite or ink contours under transparent watercolor.
• Muted indigo, blue-grey, faded teal, ochre, and restrained rust.
• One warm gold accent for the presumed alignment signal.
• Soft edges in the outer system, with sharper detail around the glass box.
• Large areas of negative space.
• Slightly surreal scale, but physically legible objects.
• Fine connecting lines that partly dissolve into washes.

The small model should be rendered precisely. The wider system should be more organic and uncertain. This reverses the usual visual language: what looks technically crisp is epistemically incomplete, while what is difficult to delineate may contain the causally important structure.

Important elements to include

• A visibly successful local test.
• A transparent but permeable boundary.
• Feedback returning into the model.
• Human adaptation, not merely passive human use.
• Institutional metrics as an active causal component.
• Nested or expanding candidate boundaries.
• A subtle sign of persistence or rerouting.
• The researcher located within the larger causal scene.

One possible small detail: a channel from the box is blocked, but the surrounding current quietly takes another route. This anticipates the chapter’s perturbational test: an agentic process may compensate when tools, memory, permissions, or feedback are altered, whereas a passive mechanism merely degrades. fileciteturn13file0L255-L291

Elements to avoid

Avoid:

• A red-eyed robot or obvious evil superintelligence.
• A human versus machine confrontation.
• A brain made from circuit boards.
• Binary code, padlocks, shields, or generic cyber-security imagery.
• A dense infographic with labelled arrows.
• Depicting the corporation as deliberately malicious.
• Making the glass box literally imprison a conscious humanoid.
• Presenting the outer system as an unambiguously coherent monster.

The chapter’s interesting claim is precisely that no component needs malicious intent. The composite can still acquire a stable optimization loop. fileciteturn12file0L125-L137

Placement

Reserve the upper-left quarter for the chapter title overlay. Put the sharpest focal point slightly right of center or in the lower-right third. The larger creature-like pattern should be discoverable rather than dominant, becoming apparent after several seconds.

A practical layout:

┌─────────────────────────────────────────────────────────────┐
│   quiet title space                    faint city / horizon  │
│                                        ↗ successor process  │
│         broad translucent outline of composite system       │
│              users → metrics → institution                  │
│                 ↑                     ↓                     │
│          researcher → [glass-box model] → training          │
│                         ↘ hidden return flow ↗               │
└─────────────────────────────────────────────────────────────┘

Two alternative concepts

The Wrong Constellation

An astronomer carefully circles one bright star and labels it as the object of interest, while faint lines reveal that the actual constellation includes many stars, instruments, observers, and a moving celestial body. Elegant and less literal, but weaker on feedback and institutional dynamics.

Aligning One Gear

A watchmaker adjusts a perfectly balanced golden gear on a workbench. Behind the table, that gear is already embedded in an enormous water-driven machine whose other gears include crowds, markets, paperwork, and tools. Strong causal metaphor, though it risks implying that the full system has a fixed, designed architecture.

The glass-box composition is preferable because it simultaneously captures evaluation, mistaken containment, feedback, composite agency, and boundary expansion. Its emotional tone should be discovery rather than doom: the reader realizes that the problem becomes more tractable once they stop assuming where the optimizer is.