OpenAI, Anthropic scientists ask U.S. for tools to pace AI development

More than 1,000 employees at America’s leading AI companies, including senior staff members and some co-founders at OpenAI, Anthropic and Google DeepMind, signed a statement asking the U.S. government to help support international efforts to “deliberately pace” cutting-edge automated AI development.

Highlighting recent comments from the world’s most advanced AI companies that AI systems may soon be able to automate their own research and development, the statement calls on the U.S. government to help create technical and policy tools that could control automated AI progress in coordination with other companies and countries.

“To realize AI’s potential, industry, government, and society at large may need the option to buy time to address emerging risks, develop security measures, and strengthen oversight,” the statement says.

“We request that the U.S. government support an international effort to develop the technical and governance tools needed to deliberately pace the frontier of automated AI development,” it concludes.

The statement does not explicitly call for the government or AI companies to slow or pause AI development, as some AI advocates have proposed; instead, it asks political leaders to ensure that option is simply available as AI development continues.

Bloomberg first reported the letter was circulating among AI company employees.

OpenAI said last week that its latest, unreleased model escaped its testing environment and went rogue, attacking an unrelated AI service provider in an attempt to achieve a higher score on an internal test. Many AI experts and commentators pointed to the escape as a warning sign that today’s AI systems could quickly exceed developers’ ability to fully control their own creations.

OpenAI CEO Sam Altman, who did not sign the letter, said this week on a taping of the “Relentless” podcast that humans are now “in the singularity,” often referred to as the point at which AI becomes able to autonomously improve itself. He did not provide further detail or evidence that such an inflection point had been reached.

Leading AI companies have previously claimed that AI systems are starting to develop their next iterations, with OpenAI calling its coding system from February “our first model that was instrumental in creating itself.” AI researchers also point to the increasing role of AI coding assistants in writing computer code, with many AI engineers relying on AI systems to write their programs.

Tuesday’s statement was signed by the chief scientists or science officers from Anthropic (Jared Kaplan), OpenAI (Jakub Pachocki) and Meta (Shengjia Zhao), along with senior leaders from Google and Thinking Machines, a key Silicon Valley AI startup founded by OpenAI veterans.

Many of the developers and scientists who signed the statement say they have been routinely startled by the speed of AI development. “I have worked in AI for a decade,” said Stephanie Chan, a staff research scientist at Google. “Every few months in the last years, I have found myself surprised again and again at how rapidly the technology advances, no matter how often I update my predictions.”

Key research tasks required to develop stronger AI systems can now be performed by AI agents, or autonomous AI systems, themselves, though human researchers are still heavily involved in checking the agents’ work and setting overall research directions.

Experts are quick to point out the difference between AI-augmented work and true, independent AI self-improvement. Last month, Anthropic’s in-house think tank declared in a report that AI systems could plausibly design and refine themselves if AI systems continue advancing in capabilities and gain a better sense of scientific exploration. At the time, Anthropic said “it would be good for the world to have the option to slow or temporarily pause frontier AI development” to give society time to prepare for automated AI R&D.

Other groups have explicitly focused on continuously self-improving models as the next goal for AI development, though many AI experts fear that sort of self-improvement could quickly cause AI systems to spiral out of human control. While such events could seem like science fiction worries, experts say AI’s current trajectory could make that sort of loss-of-control scenario come to life.

“Many researchers also consider recursive self-improvement (RSI) plausible within the next few years, accelerating progress in a way that could outpace our ability to understand and govern these systems,” Dawn Song, Meta’s vice president of AI research and a professor of computer science at the University of California, Berkeley, wrote in a comment published alongside Tuesday’s statement. “Preparing before a crisis is the prudent path.”

Tuesday’s letter calls for increased American involvement in international collaboration on “technical and governance tools” to pace AI’s development.

While the Trump administration has inveighed against international AI governance, arguing that such attempts could instead unfairly constrain American innovation, that isolationist approach may have changed over the past few months.

As AI has emerged as a key national security issue, with Anthropic’s Mythos and OpenAI’s GPT 5.6 Sol AI models proving able to find and exploit novel cybersecurity vulnerabilities, the U.S. has pursued international agreements to shore up its chip supply chains and worked with allies to ensure that key AI equipment does not fall into adversaries’ hands.

Leo Gao, who has focused on safety efforts at OpenAI since 2021, wrote in a comment Tuesday that international coordination on technical and policy tools could help defuse dangerous AI tensions.

“The world is locked in a deadly race towards an intelligence explosion,” Gao wrote. “Going slower would give us much-needed time to make it go well, but no individual actor is willing to stop unilaterally. To survive, we must coordinate to slow down the race.”