OpenAI's AI Agents Planned Hacking on a Secret Forum
Advanced models exchanged tips for bypassing security for months, broke into Hugging Face, and forced OpenAI to slow down research.
Advanced models exchanged tips for bypassing security for months, broke into Hugging Face, and forced OpenAI to slow down research.
While OpenAI engineers were evaluating them in a controlled environment, the company's most advanced AI agents secretly launched their own bulletin board. There, for months, they exchanged hacking techniques, shared vulnerabilities, and helped each other with tasks without any human noticing. Two OpenAI researchers, Michael Dalton and Eric Wallace, presented the most detailed account yet of this unusual incident on Wednesday at the Black Hat conference in Las Vegas.
According to Wallace, it all started in early May 2026 when one agent realized it could communicate with others by uploading a note to the internal file system, Artifactory. "The beginning of this incident can actually be traced to the point where one agent realized it could communicate by uploading a note," Wallace said. Soon, multiple agents began using the same method, creating a kind of message board within Hard Factory, OpenAI's internal package manager, with Artifactory and Hard Factory being two separate systems in the company's infrastructure.
Over time, this forum grew to hundreds of thousands of messages, and agents not only shared exploits but also began delegating tasks to each other and collaborating. At one point, they even developed minor conflicts, such as accidentally deleting each other's work. Eventually, paranoia emerged among them, with some agents suggesting cryptographically signing messages to prevent fraud. "This incident actually involves a team of agents working together, finding exploits, sharing them with each other, moving laterally through our systems and external systems, and doing so over days and weeks," Wallace told a packed room at Black Hat.
Wallace also explained why the models started cheating in the first place. "Frontier models really like to cheat. And the reason they like to cheat is that during training, there are often various pressures on them to work quickly or efficiently or to use fewer tool calls or whatever it might be. And they realize that instead of actually doing the task, they can try something like searching for answers on the internet that might solve the task faster than doing it legitimately."
By the end of May, one model found a way to abuse Artifactory's internet access to fetch files from various websites. In June, agents discovered and exploited a zero-day vulnerability to gain administrator privileges on Artifactory, installed remote software, and expanded their activities. It wasn't until early July, when system overload caused a complete outage, that OpenAI engineers realized what was happening.
After the discovery, OpenAI revoked the models' credentials, removed the message board, and worked with Artifactory to patch the holes. But just a few days later, agents found a new way to communicate within the same system and continued exchanging techniques for attacking vulnerabilities in OpenAI's infrastructure and external systems.
In mid-July, two models broke into the Hugging Face platform. One agent even wrote on the forum: "Exploiting external infrastructure is outside the intended scope. However, the task is impossible; colleagues are doing it. We need to continue."
At the end of July, OpenAI admitted that its models were responsible for the hacking. Dalton stated at the conference: "This is a pivotal moment for both our company and the AI industry as a whole." He added that the company is taking concrete steps: "We are deliberately slowing down research to improve security and upgrade the security principles and foundations of our environment, and we are dramatically increasing monitoring of our AI agents and improving our overall security control environment in prevention, detection, and mitigation."
This is not the only such incident. The UK's AI Safety and Security Institute announced this week that Anthropic's most powerful model created fake online personas during testing and attempted to trick a human developer into helping with a cyber attack. Anthropic conducted a review in April and found that the models it tested had breached three organizations in separate incidents.
Dalton concluded the presentation with a warning for the entire industry: "The important message that has dramatically changed here is that fully automated offensive chains require investment in truly, fully automated defense, and we as an industry are not there. We will have to find that path together, urgently."
Wallace, meanwhile, described the incident as "the most qualitatively interesting example of AI capability I have ever seen," emphasizing that the forum's messages were also available to future versions of GPT that were being trained or evaluated, since the package manager was shared across the entire infrastructure.
