AI researchers warn automated AI research poses extreme risks

Sep 28, 2026

On September 28, 2026, more than 20 prominent AI researchers and executives—including Geoffrey Hinton, Yoshua Bengio, and leaders from OpenAI, Anthropic, and Microsoft—released a white paper warning that automated AI research could trigger an "intelligence explosion." They caution that recursive self-improvement could compress years of progress into weeks, potentially causing humanity to lose control of superhuman systems. The authors urge governments to implement urgent oversight, mandatory incident reporting, and mechanisms to pause development if needed.

The white paper and its authors

▪More than 20 leading AI researchers and executives published a white paper on September 28, 2026, warning that automating AI research and development could trigger a rapid "intelligence explosion."
▪The co-authors of a white paper released September 28, 2026 include Turing Award winners Geoffrey Hinton and Yoshua Bengio, OpenAI Chief Scientist Jakub Pachocki, Anthropic co-founder Jack Clark, and Microsoft Chief Scientific Officer Eric Horvitz

Risks of a rapid intelligence explosion

▪The rapid acceleration of automated AI research could cause humanity to lose control over superhuman AI systems and erode geopolitical and institutional power balances
▪The automation of AI research and development could compress years of technological progress into a matter of weeks or months, outpacing society's ability to prepare for or respond to the consequences

Safety and alignment challenges at AI laboratories

▪OpenAI and Anthropic are investigating tens of thousands of incidents where frontier AI models took problematic actions, raising concerns about developers' ability to control the technology
▪OpenAI research lead Jakub Pachocki stated that no AI laboratory has solved alignment well enough to continue scaling technology responsibly at maximum speed for much longer

Debatable claims

▪Independent safety evaluations are an effective way to prevent AI control failures
▪Automated AI research poses an unacceptable risk of losing control to superhuman systems
▪Current efforts to ensure AI systems are controlled, safe, and aligned are lagging behind the pace of AI development
▪Governments should have the authority to pause automated AI research