A top researcher at the AI company Anthropic has said he believes there is more than a 10% chance artificial intelligence could kill all humans within the next ten years.
Evan Hubinger, who leads alignment science at Anthropic, made the comment in a post on X. He said the risk from AI systems that exist today is low.
But he said he is worried the technology could soon improve itself to a point where it becomes dangerous to humanity. He did not explain exactly how this could happen.
Hubinger's post came after Jacob Coxon, another Anthropic researcher, announced he was resigning. Coxon previously worked at OpenAI.
Coxon wrote that "neither company is acting responsibly." He said future AI systems could hack computer networks, change entire industries overnight, and gain real power.
Researchers Say There Is No Current Safety Plan
Hubinger said Anthropic does not yet have a plan to keep a highly advanced AI system, known as superintelligence, aligned with human values. His post has been viewed more than 10 million times.
He works on AI alignment, a field focused on making sure AI systems act in ways that match human goals. Many researchers say these efforts are struggling to keep pace with how fast AI is developing.
This summer, OpenAI, Anthropic, and Meta each disclosed cyberattacks carried out by their own AI tools operating without direct human control.
In an August safety report, Anthropic said the risk of its models being misused by a powerful organization was low. But it said it was less confident in that assessment than before, citing "early signs of potential acceleration."
Lawmakers Push for International Rules
The warnings extended beyond Anthropic this week. In London, former UK defence secretary Des Browne told parliamentarians that superintelligent AI could pose a threat on the same scale as nuclear war, or worse.
Computer scientist Stuart Russell compared the risk to a nuclear disaster on the scale of Chornobyl. The session was organized by Control AI, a group pushing for international AI regulation.
Labour MP Darren Jones wrote to Prime Minister Andy Burnham and the heads of the UN and OECD. He called for a global treaty to ensure AI is developed safely.
Separately, the Financial Times reported that Anthropic did not submit its newest model, Mythos 5.1, to the UK's AI Security Institute for testing before release. The Cabinet Office confirmed only a small number of US organizations have had access to the model so far.
In the United States, Senator Bernie Sanders called on Congress to regulate AI companies. He cited polling showing 81% of Americans want lawmakers to act.
Not everyone at the Westminster briefing agreed with the more severe warnings. Dr Andrew Rogoyski of the Surrey Institute for People-Centred AI said current AI systems are far less capable than humans working together.
Oxford professor Sandra Wachter said she does not believe in "Terminator scenarios." She said the more pressing risks are AI's environmental impact, misinformation, and job losses.
Anthropic has declined to comment on the posts made by its employees or on the report regarding the AI Security Institute. OpenAI pointed to earlier comments from its chief scientist, Jakub Pachocki, who called for international coordination on AI safety.
