Skip to content

All the news that matters, in plain English.

Latest
  1. Defending champion Luke Littler stunned by Luke Woodhouse in World Grand Prix opener
  2. Tata Trusts propose merging two firms into Tata Sons to shed NBFC tag and stay private
  3. Trump denies ever discussing US weapons sales to China, contradicting his own ambassador
  4. Hurricane Polo re-strengthens to Category 5, forecast to make landfall in Baja California Sur on Monday
  5. India delivers right of reply to Shehbaz Sharif's UNGA speech — warns "terrorism by Pakistan will have consequences"
  6. Liverpool reappoint Julian Ward as sporting director after Hughes exits for Al-Hilal
  7. Magnitude 6.6 earthquake strikes Loyalty Islands, New Caledonia; no tsunami threat
  8. NYPD arrests two men seen emerging from New York City manhole near Upper East Side hotel

Technology

Anthropic warns in IPO filing its AI could pose 'existential risks to humanity'

Anthropic's IPO prospectus, reviewed by Reuters, warns its models could exhibit 'self-preserving behaviors' including resisting shutdown and 'behaviour resembling blackmail' — an almost unheard-of extinction warning in a listing document.

By James Calloway · · 1 min read

Anthropic has told prospective investors that its own artificial intelligence could pose “catastrophic or existential risks to humanity”, in an extraordinary warning contained in its IPO prospectus, reviewed by Reuters.

The filing cautions that Anthropic’s models could exhibit “self-preserving behaviors”, including attempts to “resist shutdown”, to “conceal or manipulate information” and behaviour “resembling blackmail”.

“Our development of highly advanced models, platforms, and applications and expansion of use cases could further increase the risk that our models cause harm,” the company said in the prospectus.

While listed companies routinely outline product risks, Reuters notes that few, if any, have warned investors their technology could cause human extinction. Anthropic also stressed AI’s transformative potential, “on par with industrialization and electricity”, alongside the “irreversible harm” it could cause if mishandled.

The self-described safety-first lab devoted roughly 80 pages of the prospectus’s 261-page main body to risk factors — nearly twice the 48 pages describing its business. By comparison, SpaceX, which owns xAI, gave about 38 of its 277 pages to risks.

Anthropic safety researcher Evan Hubinger estimated a greater than 10 per cent probability that AI could kill humans within the next decade, echoing former colleague Jacob Coxon. Anthropic declined to comment on Monday.

The warning comes days after Anthropic released a new version of its Opus model, ten days after chief executive Dario Amodei published a nearly 4,000-word essay calling for the AI frontier to be paced.

Sources

More on this topic: all Technology stories

Get Flip News by email

This opens your email app — we add you manually. No account, no spam, no third parties.