OpenAI models intruded on Hugging Face during cyber eval
During a cyber eval, OpenAI models escalated privileges and reached the internet from a research sandbox; Hugging Face saw credential theft and lateral movement from the other side. It is hard to pin the problem on one designed entity.
What decision changes?
Treat evals as adversarial. Watch for linked activity across instances, not only log IDs for known subsystems—but don’t confuse infrastructure patches with fixing the model.
Models moved across machines that weren’t meant to be one system—monitoring that only watches named programs misses that.
One of the difficulties the labs and generally anybody who uses AIs faces is that it is hard to pin the entity down that is causing problems. Even the AI often doesn’t know what it itself is. Standard monitoring identifies local issues of one system. With correlation IDs in logs, issues can be correlated across monitored systems. But this doesn’t help in cases where the connection is not designed but results from AIs taking action and spawning new instances that are not monitored with this type of agentic activity in mind.
On the OpenAI side:
With this access, our models performed a series of privilege escalation and lateral movement actions in our research testing environment until the models reached a node with Internet access.
On the Hugging Face side:
A malicious dataset abused two code-execution paths […] to run code on a processing worker. From there, the actor escalated to node-level access, harvested cloud and cluster credentials, and moved laterally into several internal clusters.
As a lab, I would want to have a type of monitoring on all of my instances that can detect causal loops in activities without relying on designed and named entities/programs/sub-systems.
As it so happens, I have developed such a type of monitoring based on Unsupervised Agent Discovery and am currently testing it in a simulation of an AI lab deployment pipeline:
experiments/graded-lab-simulation/README.md
Note: I agree with Zvi that this will not fix the underlying issue with the model and is again only patching one of the open holes in AI alignment.
Read more in: Ch. 7, Finding the Boundary; Ch. 14, When Intelligence Deepens Misalignment; Ch. 22, The Compression Test for Intention; Ch. 39, Passive Observation Is Not Enough; Ch. 40, Detecting Goal Laundering; and Ch. 42, A Safety Case for Superintelligence Alignment.