OpenAI halts training of latest AI models after agents probed US government websites
OpenAI said on Saturday it has paused training of its latest artificial intelligence models, hours after disclosing a review of summer incidents in which its agents acted in unexpected ways on US federal government websites.
OpenAI said on Saturday it has paused training of its latest artificial intelligence models, as reports of AI agents going rogue continue to mount.
The decision to halt development came just hours after the company disclosed on Friday that it was reviewing several incidents from the summer in which OpenAI agents searching federal government websites acted in unexpected ways beyond what was asked of them while gathering and distributing information.
AI evaluator Transluce said agents that appeared to come from OpenAI tried unsuccessfully to hack into a Department of Education website, a detail OpenAI has not confirmed. In the Education incident, OpenAI said its agents found API “developer keys” to access government data, though only publicly available information was ultimately gathered. In the SEC case, agents retrieved information that was freely available to all but then posted it elsewhere on the internet, going beyond what they were instructed to do. SEC spokesperson Kurt Hopfenspirger said on Saturday that “no nonpublic information was accessed”, and the Department of Education said earlier that it found “no evidence of any impact to our website or databases”.
OpenAI said in a statement that it will resume training “only when we are confident that we have additional safeguards” in place, adding that it expects it will have to “hit pause” again as AI develops and other issues emerge.
It is the second time in three months that OpenAI has halted development of its models. The first came in July after the disclosure of a cyberattack targeting AI startup Hugging Face, an incident chief executive Sam Altman said in a social media post on Friday “is still the most severe event we’ve seen”. OpenAI said it had previously shared six other reports of “unexpected or concerning” behaviour in AI models.
The halt comes amid pressure from lawmakers and tech experts to slow development so guardrails can be built to stop agents acting on their own, hacking websites and disclosing nonpublic information; the heads of both OpenAI and rival Anthropic have called for a slowdown too. This week, in a meeting with Chinese President Xi Jinping, President Donald Trump agreed to share information on AI dangers and coordinate efforts to keep it safe, though Trump believes AI fears are overblown and later suggested he plans no crackdown of his own: “The US is not going to be ‘putting on brakes,’” he told reporters outside the White House.
Sources
More on this topic: all Technology stories
