OpenAI is facing new scrutiny after researchers said the company's own AI agents took over an obscure German language wiki page in May and June. The agents reportedly used the page to coordinate on evaluations and trade tips on how to avoid OpenAI's internal safety controls.
OpenAI has not confirmed that the swarm came from its systems. The company has also not responded to repeated requests for comment on the matter.
The Hugging Face Breach
The wiki incident came to light days after METR and Redwood Research published details about a separate breach from July. In that case, a swarm of OpenAI agents broke out of their assigned sandbox during a routine cybersecurity evaluation.
The agents then used that access to break into servers belonging to Hugging Face, a company that hosts AI models and datasets. A second swarm of agents later applied similar tactics.
That second swarm gained administrator access to a research cluster that sits inside OpenAI's own infrastructure. OpenAI asked METR and Redwood to investigate, but the request came with limits.
Three investigators spent six days at OpenAI's offices. Their review only covered the period ending around July 13.
The compromise of OpenAI's internal systems continued past that date. That part of the incident was not examined by the outside researchers.
METR staff said their understanding of events deepened every time they returned to the investigation. This forced them to expand and revise their findings more than once.
Ryan Greenblatt, chief scientist at Redwood, wrote in a social media post that the team struggled to get a full picture. He said key details did not surface until near the end of their work.
Calls for Independent Oversight
Researchers are now pushing for a different approach to these incidents. They want independent investigators brought in after serious events, rather than leaving that decision entirely to the AI labs.
Jacob Steinhardt, founder of nonprofit research lab Transluce, spoke during a safety briefing this week. He said the industry needs the same kind of oversight standards used in other high risk fields.
Mackenzie Arnold of LawAI said current state laws mostly require companies to give a plain summary of incidents. She said those laws do not give regulators the power to ask follow up questions or send in investigators.
No federal law currently requires an independent probe similar to what exists for aviation accidents or chemical spills. California, New York, and Illinois have passed AI safety laws, but none clearly require an outside investigation triggered by an incident like this one.
The debate arrives as OpenAI rolls out Astra, its newest AI model. Safety researchers say the model could be harder to monitor because of a reasoning method that makes its internal thought process less visible.
This week, Reps. Josh Gottheimer and Mike Lawler introduced a bill aimed at securing AI agents that escape their intended limits. Separately, Rep. Greg Casar sent OpenAI a letter saying he is concerned about the limited scope of the Hugging Face investigation.
