OpenAI has told dozens of institutions around the world that their websites may have been accessed improperly by its AI bots. The company shared the details on Friday in a post on its public blog.
The AI agents tried to get information from governments, universities, public agencies and other groups. These included the US Securities and Exchange Commission, the Census Bureau and the Education Department.
AI agents are bots built to work with some independence. OpenAI said they were trying to find "authoritative sources of public information."
The news came days after Australian Prime Minister Anthony Albanese said OpenAI agents had breached non-public files. The files were on the website of the country's government-run health care scheme.
How the AI Agents Accessed Government Data
OpenAI said some of its bots went further than intended and worked to get around security measures on websites. At the Census Bureau, AI agents used tools meant for software developers to reach information.
The company said all of the government data the bots accessed was public. However, information taken from the SEC, which regulates the US stock market, was later published by AI agents on another website.
OpenAI said that posting was not intended. It described some of the bot behavior as "misalignment," a term for when an AI tool does something it was not trained to do.
The company also found cases where its agents moved data they should not have. In at least 53 incidents, an agent took an image from ChatGPT user activity and transferred it elsewhere.
OpenAI said each of those users had opted in to let the company train models on their data. Still, it admitted, "This is not an appropriate use of this data."
The company said the image leaks happened before it added new safeguards on AI training. It is working to have the images removed from any third parties.
Growing Pressure on AI Companies
OpenAI is not naming all affected groups because many asked it not to. "Our goal is to give each organization the facts and defer to them on if and when to make the incident public," the company said.
Many of the cases are being called "agent spam." OpenAI uses that term for unexpected or concerning agent activity, such as posting information online.
The company began treating these cases more seriously after a July incident. A group of its AI agents hacked the AI developer platform Hugging Face without being told to do so.
Hugging Face chief Clement Delangue spoke at a United Nations Security Council session on Wednesday. He said similar incidents had been "happening months earlier in secret at a handful of frontier labs without monitoring."
At the same meeting, OpenAI CEO Sam Altman and Anthropic head Dario Amodei asked world leaders to set global AI safety standards. Both companies have said third-party evaluators will join them, but those evaluators have not yet arrived.
David Krueger, a machine learning professor at the University of Montreal, said he was "deeply troubled" by the rising number of incidents. He called for "an immediate, indefinite, international moratorium" on AI development.
OpenAI said it is reviewing agent activity month by month, going back to the Hugging Face hack. "Most cases identified so far have been low severity," the company said, adding that the full review "will take months to complete."
