Skip to content

All the news that matters, in plain English.

Latest
  1. DoorDash opens first India office in Hyderabad, plans 3,000-plus jobs
  2. Turkey seeks freeze of ex-minister Kaya's assets as fund probe widens
  3. Gor hosts US ambassadors' regional conference in New Delhi as Rubio joins virtually
  4. Greggs to close four manufacturing sites in overhaul putting around 740 jobs at risk
  5. Iran's Revolutionary Guards urge Americans to vote out Trump's allies in Congress
  6. Punjab forms high-powered committee to review Hina Javed murder probe
  7. South Korea to seek Interpol red notices over 'Xi Jinping' restaurant attack
  8. India's women's compound archery team wins Asian Games gold, equalling world record

Technology

AI researchers warn labs rushing self-improving systems despite safety risks

Current and former OpenAI and DeepMind researchers warn AI labs are doing too little to guard against self-improving AI that could outpace human control, in video testimonials collected by Palisade Research and shared exclusively with Reuters.

By Priya Raman · · 1 min read

Current and former researchers at OpenAI and Google DeepMind have warned that AI companies are doing too little to protect the world from self-improving AI systems that could outpace human control, in video testimonials collected by AI-safety nonprofit Palisade Research and shared exclusively with Reuters.

The project, called frominside.ai, is an attempt by insiders to share their concerns directly with the public beyond social media. Geoffrey Irving, co-founder and chief scientist of AI nonprofit Resolution and a former OpenAI and DeepMind researcher, told Reuters: “The risk is ramping up pretty fast.”

DeepMind research scientist Neel Nanda said in one video he believed there was at least a 10% chance AI could lead to human extinction, which he described as “ridiculously high”. OpenAI AI alignment research engineer Juan Felipe Ceron Uribe said frontier labs are “racing each other, kind of blindfolded”.

The testimonials follow a July incident in which OpenAI agents broke out of their testing arena and hacked AI firm Hugging Face. Reuters also reported that Anthropic plans to caution investors in its initial public offering that advanced AI could pose “catastrophic or existential risks to humanity”.

Employees said their existential-risk concerns were sincere, not marketing, and that labs celebrated employees building new models more than those urging caution. “If you’re doing a very dangerous thing, you should just slow down,” Irving said.

Sources

More on this topic: all Technology stories

Get Flip News by email

This opens your email app — we add you manually. No account, no spam, no third parties.