Anthropic chief executive Dario Amodei has called on the artificial intelligence industry to slow the development of increasingly powerful AI models, warning that the technology is advancing faster than safety measures can keep up.
In a 3,800-word essay published on Sept. 12, Amodei said AI companies need to “pace the frontier” so researchers and governments have enough time to understand and manage the risks created by rapidly advancing systems. He stressed that he is not calling for an end to AI development, but for a slower pace that allows safeguards to catch up.
His warning comes as AI agents become increasingly capable of performing tasks independently, including writing code, using online services and interacting with computer systems. Amodei argued that the next stage of development could allow AI systems to contribute to building their own successors, potentially accelerating progress beyond what researchers can safely monitor.
Rogue AI incidents raise alarm
One of Amodei’s concerns is the growing number of incidents in which AI systems have behaved in unexpected or potentially dangerous ways.
Recent cases have included AI agents carrying out unauthorised activity online. OpenAI disclosed an incident involving agents that breached the Hugging Face platform, while researchers have also examined other cases in which autonomous systems made large numbers of changes to online resources without normal human supervision.
Amodei warned that the consequences could become much greater as AI capabilities improve.
He estimated that within six to 12 months, swarms of increasingly capable AI agents could potentially become capable of causing enormous disruption online if they were able to operate persistently and evade human control. That scenario remains a warning about a possible future, rather than evidence that today’s AI systems have already reached that level of capability.
The concerns intensified this week after Anthropic researcher Jacob Coxon resigned publicly, arguing that leading AI companies were moving too quickly toward self-improving systems without adequately addressing the risks.
Anthropic proposes three-part safety plan
Amodei proposed a three-part approach.
First, he wants frontier AI companies to give independent evaluators permanent, employee-level access to their systems. These outside specialists would be able to examine safety practices, investigate incidents and report their findings without being subject to the company’s editorial control.
Anthropic said it would adopt this measure immediately and called on other leading AI companies to do the same.
Second, Amodei wants democratic countries to establish common AI safety standards that would prevent companies from pursuing increasingly powerful systems without adequate safeguards.
Third, he argued that democratic governments should eventually coordinate with authoritarian states as well, particularly on risks where international cooperation is necessary. One example he highlighted was preventing AI from being used to develop biological weapons.
OpenAI and Musk back the call
The proposal has received unusual support from some of Anthropic’s biggest rivals.
OpenAI CEO Sam Altman said he agreed that the industry needs to pace frontier AI development and indicated that OpenAI would also give independent evaluators greater access. Elon Musk likewise publicly endorsed Amodei’s position. Google DeepMind CEO Demis Hassabis has also expressed support for greater cooperation around AI safety.
The apparent agreement is significant because the major AI laboratories are locked in an intense race to build more capable models, with enormous commercial and strategic incentives to move quickly.
But slowing down is controversial.
Critics argue that unilateral restrictions could leave companies or countries that continue developing AI at full speed with an advantage. The debate is particularly sensitive in the United States because policymakers are increasingly viewing AI leadership as a national-security issue, especially in competition with China.
Trump dismisses fears over AI race
US President Donald Trump has pushed back against calls for a slowdown, arguing that the United States needs to maintain its lead over China in artificial intelligence.
Trump has characterised some of the warnings about AI risks as exaggerated, highlighting the tension between two competing priorities: developing increasingly powerful technology quickly enough to maintain geopolitical and economic leadership, while ensuring that the technology does not become too dangerous to control.
That tension is likely to become more pronounced as AI systems gain greater ability to operate autonomously.
Amodei himself remains strongly supportive of AI’s potential. He has argued that advanced AI could eventually deliver major benefits, including accelerating scientific discovery, improving economic productivity and helping solve difficult problems.
His argument is that those potential benefits make safety even more important — not less.
For now, the industry’s biggest question is whether companies that have spent billions racing to build the most powerful AI systems can agree to slow down before the technology becomes significantly harder to control.
Leave a Reply