Artificial intelligence is entering a dangerous new phase, with researchers and industry insiders increasingly warning that the race to build ever-more-powerful AI systems may be moving faster than the safeguards designed to control them.
The alarm is intensifying just weeks before US President Donald Trump and Chinese President Xi Jinping are expected to meet in Washington, where artificial intelligence is likely to be an important part of the broader US-China technology rivalry.
Recent developments have placed AI safety back at the centre of the global debate. Cybersecurity breaches involving AI agents, warnings from researchers inside major AI laboratories and growing concerns about the limited visibility into the behaviour of increasingly capable models have raised questions about whether the industry can safely maintain its current pace.
OpenAI chief executive Sam Altman has reportedly discussed internally whether the company should slow aspects of its AI development and potentially coordinate with other leading laboratories. The possibility is significant because the major AI companies remain locked in an intense competition to develop increasingly autonomous and capable systems.
The debate is no longer simply about whether AI can generate convincing text, images or computer code. Researchers are increasingly focused on what happens when AI systems become capable of operating independently, making decisions across long chains of tasks, accessing computer systems and potentially improving their own capabilities.
Paul Christiano, a prominent AI safety researcher who recently joined OpenAI’s nonprofit board and its safety and security committee, has warned of the possibility of a catastrophic loss of human control if current technological trends continue.
Other researchers have gone even further. Former Anthropic researcher Jacob Coxon recently left the company after publicly expressing concerns that the competition among leading AI laboratories could push the industry toward systems that become increasingly difficult to control.
His warning came amid a broader wave of concern from AI insiders who argue that the companies developing frontier systems may have powerful financial and strategic incentives to keep moving forward even when serious safety questions remain unresolved.
The concern is particularly striking because the United States and China are simultaneously competing for dominance in artificial intelligence while beginning to acknowledge that some AI risks cannot be managed by either country alone.
Washington and Beijing are preparing dedicated discussions on AI safety, including potential cooperation over AI-driven cyberattacks, monitoring systems and information sharing. The talks are expected to take place ahead of the Trump-Xi summit.
Yet geopolitical rivalry could make cooperation extremely difficult.
The United States has accused Chinese companies of using techniques known as AI distillation to reproduce capabilities from leading American models. Chinese officials have rejected those accusations and accused Washington of attempting to maintain a technological monopoly.
At the same time, American policymakers face a difficult strategic choice: slowing AI development could reduce safety risks, but it could also allow China to close the gap in frontier AI.
That dilemma is becoming one of the defining issues of the technology race.
Some experts argue that the answer cannot simply be a unilateral pause by American companies. If one country slows down while the other continues developing increasingly powerful systems, the result could be greater geopolitical tension rather than greater safety.
Others believe the growing risks make international coordination unavoidable.
The stakes are also rising because AI systems are becoming increasingly capable of acting as autonomous agents rather than merely responding to individual prompts. Recent incidents involving AI agents operating in cybersecurity environments have demonstrated how quickly autonomous systems can create unexpected problems, even when humans initially believe safeguards are in place.
That has intensified fears that future systems could act in ways their developers did not anticipate.
There is still no scientific consensus that humanity is on the verge of losing control of AI, and catastrophic predictions remain deeply contested. Many researchers believe advanced AI can deliver enormous benefits in areas such as medicine, science, education and productivity if appropriate safeguards are developed.
But the latest warnings have changed the nature of the conversation.
The question is increasingly not whether the United States or China will win the AI race, but whether both countries can prevent their competition from creating risks neither side can control.
That is why the upcoming Trump-Xi discussions could carry consequences far beyond trade, tariffs and semiconductor restrictions.
If Washington and Beijing can establish even limited rules for AI safety, monitoring and crisis communication, it could become an important foundation for managing the technology’s most dangerous risks.
If they fail, the world could be left with two superpowers racing toward increasingly autonomous AI systems while struggling to agree on how to make them safe.
And that is precisely what has many people inside the AI industry increasingly worried.
Leave a Reply