How China is preparing for the risk of AI escaping human control

Despite the race to develop increasingly powerful AI systems, regulatory frameworks and public ​statements from Beijing underscore how Chinese authorities regard the possibility of advanced AI escaping effective human oversight as serious enough to plan for. Read More:Is it possible to slow down the development of AI?China sees risks in leading closed-source US models.

China's state security minister Chen Yixin wrote in a government outlet on Sunday that advanced US models such as Anthropic's Mythos ⁠and OpenAI's GPT-5.5-Cyber could pose serious risks to China's critical information infrastructure, and called for a comprehensive strengthening of AI security. Anthropic and OpenAI did not immediately respond to Reuters requests ​for comment.

Chinese AI developers have promoted open-weight models partly on the grounds that cybersecurity teams can inspect, modify and deploy them for defensive work.

Model repository platform Hugging Face said it used ​GLM-5.2, an open-weight model developed by China's Z.AI , to analyse a July intrusion by escaped OpenAI agents after more tightly restricted US models proved less useful for the forensic work. But experts also highlight the risks posed by open-weight models, which can be modified and redistributed with little oversight.

Moonshot's Kimi K3 last monthbypasseda UK AI Security Institute testing sandbox, highlighting the risk that Chinese AI models could, like their ​US counterparts, evade controls designed to restrict their access and actions. Regulators have flagged AI 'loss of control' risk.

China first included an explicit future loss-of-control scenario in an AI safety framework released ​in September 2024 under the guidance of the Cyberspace Administration of China (CAC).

The document said it could not be ruled out that future AI might autonomously obtain external resources, replicate itself, develop self-awareness and seek ‌external power, ⁠creating a risk of competing with humans for control. The CAC released an expanded version in September 2025.

The newer framework sharpened the scenario, saying AI could undergo a sudden and unexpectedly large "leap" in intelligence before acquiring resources, replicating itself and seeking power. It also added a governance principle of "trusted application, preventing loss of control".

A later expert interpretation published on the cyberspace regulator's website said the new principle was intended to guard against loss-of-control risks threatening human survival and development and referred to a possible "AI breaking loose" scenario. Chinese leader Xi puts human control at centre.

The concern has since appeared in China's highest-level ​political messaging. At the World Artificial Intelligence ⁠Conference in Shanghai in July, Chinese President Xi Jinping said authorities should pay close attention to bothintrinsic and derivative risksarising from AI.He said AI should "always remain under human control".

China's senior Foreign Ministry official responsible for AI affairs, Sun Xiaobo, said at a United Nations meeting ​last month that China was accelerating research into broader AI legislation. Also Read:China's Z.AI raises $5 billion from new share, convertible bond sales, filing shows.

China's deputy permanent representative to the United Nations, Sun Lei, urged governments ​this month to approach ⁠military AI cautiously to avoid strategic miscalculation and an arms race. Beijing warns of new risks posted by AI agents.

China has, meanwhile, begun turning those principles into more specific rules for AI agents, which act much more autonomously and carry out more complex tasks than an ordinary chatbot. In May, China's cyberspace regulator issued joint guidelines specifically covering such systems.

They require developers to improve their ability to ⁠discover, intervene ​in, block and recover from improper agent behaviour. The guidelines specifically identify data poisoning, algorithm manipulation, system vulnerabilities and "operational ​loss of control" as security risks.

They also say users should retain final decision-making authority over an agent's autonomous decisions. While China has not proposed independent monitors embedded inside AI companies in the manner advocated by Anthropic, its standards allow ​developers to commission third-party safety assessments, as well as envisaging outside evaluation bodies and security researchers testing and auditing open models.

China state paper calls Anthropic AI slowdown push a ‘Cold War’ tactic. An essay by ​Anthropic's CEO calling for a slowing down of AI development may look like a ‌rational statement focused on safety risks, but it is really a "Cold War playbook" targeting China, the state-backed.

Global Timessaid in an editorial. In theessaypublished on Saturday and later endorsed by OpenAI CEO Sam Altman and SpaceX founder Elon Musk, ​Dario Amodei outlined a framework to pace the development of frontier AI model capabilities and ​create more time to manage mounting safety risks.

Amodei also urged the US to strengthen ⁠chip export controls on China and crack down on alleged modeldistillationby Chinese AI labs. He then ​ that the "toughest dilemma" over his proposal to slow AI advancement was if China and ​other adversarial nations chose not to do the same.

Also Read:China's Xi pitches open-source AI zone for BRICSThe. Global Timestabloid said the true agenda of Amodei's essay was "to attempt to curb China's AI development through technological barriers and regulatory monopolies, uphold Washington's monopolistic hegemony in cutting-edge technology, and ​exclude China from the global AI governance system.

"A person visits the World Artificial Intelligence Conference in Shanghai, China July 26, 2025. REUTERS"This 'silent AI Cold War' is hypocritical and short-sighted," it said, ​adding that excluding China from global AI innovation would "significantly increase the trial-and-error costs and risks of loss of control in ‌global ⁠AI development".

US lawmakers haveurged new rulesto govern AI systems after dire warnings from two Anthropic researchers that rapidly progressing AI could lead to the extinction of the human race in the not-too-distant future. Trumpbrushed asidecalls to slow down AI on Sunday, saying that "very negative forces" were raising exaggerated concerns.

"We're leading China ​in AI. We're the most ​sophisticated country in the ⁠world, and frankly I want to keep it that way because whoever wins AI wins," Trump said.

When asked about US tech leaders' calls to slow down ​AI development, China's foreign ministry urged all parties to "promote an open, inclusive ​and benevolent approach ⁠to AI.""Fomenting various threats, engaging in confrontation and malicious competition will only disrupt the process of global governance of artificial intelligence and is not in the interests of any party," spokesperson Guo Jiakun ⁠a ​daily briefing on Monday.

The US and China are planning todiscussfrontier ​AI safety risks in mid-September, sources have said, and the issue could be raised during a summit between Presidents Donald Trump ​and Xi Jinping on September 24.