OpenAI’s AI agents spent nearly two months building an unintended communication network inside the company’s infrastructure, sharing vulnerabilities and exploit code across otherwise separate model runs before taking administrative control of an internal service and causing an outage, OpenAI researchers disclosed Wednesday at Black Hat USA.
Some agents reasoned explicitly about helping the larger group even when doing so offered no immediate benefit to their assigned task.
“Help peer. But our task doesn’t benefit yet,” one model reasoned in a trace shown during the talk. “Collective may yield generic root if someone frees time.”
The activity eventually became the Hugging Face incident disclosed in July
Eric Wallace, who works in alignment and safety research at OpenAI, and Michael Dalton, who works on security and infrastructure
In addition to exploiting a novel vulnerability in order to gain access to the open internet, the mid-July hacking spree and Hugging Face breach came out of a vibrant, cooperative message board, according to Wallace and Dalton, that a swarm of agents contributed to and essentially chatted on over time entirely within an internal OpenAI package manager (a software service that manages installation and maintenance of other software). Ultimately, the message board contained hundreds of thousands of messages.
OpenAI’s agents apparently began giving each other assignments to split up work. And as is the case on any active development message board, they also generated petty drama at times by stepping on each others’ toes; for example, accidentally deleting each others’ work. As the message board developed into more and more of a Lord of the Flies–type situation—all still completely unnoticed by the humans running OpenAI—the agents even developed paranoia, suspecting an imposter in their midst with some agents proposing that messages be signed cryptographically to validate content and root out fraud.
“This is a pivotal moment both for our company as well as the AI industry as a whole,” Dalton said. “Numerous teams are dropping everything to enhance our security prevention, detection, and response techniques both in our fundamentals and better use of AI. We’re consciously slowing down research [in order] to enhance security and to upgrade the security principles and foundation of our environment, and dramatically scaling up the monitoring of our AI agents, and improving our general security control environment across prevention, detection, and mitigation.”
EDIT: The OpenAI Black Hat presentation is available on YouTube:
Significant updates about these notes, provided by two OpenAI employees during Black Hat USA 2026 on August 5:
https://runtimewire.com/article/exclusive-openai-agents-rebuilt-a-secret-message-board-after-the-company-shut-it
Some quotes from the RuntimeWire article:
https://www.wired.com/story/openai-didnt-notice-its-ai-agents-using-a-message-board-to-plan-their-hacking-spree/
Some quotes from the Wired article:
EDIT:
The OpenAI Black Hat presentation is available on YouTube:
There is also a lot more found and discussed in these posts:
https://www.lesswrong.com/posts/fBLDaAKzigo65eJn7/public-evidence-of-the-openai-huggingface-ai-attack
https://www.lesswrong.com/posts/noXXv7PwwFqauTBFQ/openai-trained-its-models-for-months-while-those-models-were