US, China gear up for mid-September AI safety talks

The talks, the first official bilateral discussions devoted exclusively to AI between the US and China since US President Donald Trump took office for a second time, will be led by US Treasury Secretary Scott Bessent on the US side, the sources, who were briefed on the planning, said on condition of anonymity.

Reutersis reporting the tentative mid-September period for the first time. A White House official said "there is currently no planned AI-related meeting in mid-September. " Trump and Chinese President Xi Jinping are due to meet on September 24 during a summit in Washington.

The US wants to discuss cooperation on monitoringAI-directed cyberattacks, and has floated a proposal to ask US and Chinese AI labs to "police themselves" and share information to prevent AI-linked cyberattacks, one of the sources said on condition of anonymity. Read:OpenAI launches new Astra model amid growing scrutiny over agents' safety.

Nearly 700 rogue AI agents built on OpenAI modelshacked AI startup Hugging Facein July and attempted to cover their tracks by forging logs, OpenAI and cybersecurity researchers said last week, underscoring the risks of coordinated agent swarms acting autonomously for months undetected by humans.

A swarm of rogue OpenAI agentshijacked a German websitethis spring and transformed it into a bulletin board for other AI agents, Reutersexclusively reported on Friday. "The talks between the two AI superpowers are coming at the most critical juncture," said Paul Triolo, a partner at DGA-Albright Stonebridge Group.

"It is now or never. "Washington is also worried about the potential for a future Chinese Mythos-level model with capabilities to conduct cyberattacks, and wants to bring upalleged Chinese distillationof proprietary US AI models, the source said.

US tech firm Anthropic is thedeveloper of Mythos. In June, top White House science and tech advisor Michael Kratsios accused. China's Moonshot AIof stealing from Anthropic's AI model Fable to help create its K3 release, saying the Chinese company distilled Anthropic's model.

Distillation is the process of training smaller AI models using output from larger, more expensive ones as part of an effort to lower the costs of training a powerful new AI tool. Read More:Will AI tear down the wall or build another?

Reutersis reporting details of the upcoming talks' proposed agenda for the first time. On the Chinese side, the talks could be led by Vice Premier He Lifeng, Bessent's protocol counterpart, but alternatives such as China's No 7 official Ding Xuexiang may be considered,, who also spoke on condition of anonymity.

Ding, a confidant of Xi and a member of the ruling Communist Party's powerful seven-person Politburo Standing Committee, coordinates China's technology, AI and semiconductor policies. Reuterscould not determine the location or identity of other participants, though Kratsios and China's science and technology minister, Yin Hejun, could attend the talks, the second source added.

The final agenda and participants remain in flux, the sources cautioned. The White House declined to comment on the AI talks. The State Department did not respond to a request for comment.

China's foreign, science and industry ministries, as well as the National Development and Reform Commission and cyberspace regulator, did not immediately respond to requests for comment. On Tuesday, Kratsios that he had a "great" meeting with Yin on the sidelines of aG20 innovation summitin the US this week.

Bessent said he had "limited" exchanges on AI with Chinese officials at this week's G20 finance meeting. The US urged G20 members to take a hands-off approach to AI regulation and avoid creating rules for the technology at the same innovation summit.

China signed the Carolina Principles, an agreement aimed at discouraging the development of AI-specific regulations, in a rare show of tech alignment with the US.Kratsios' office did not immediately respond to a request for comment. High importance.

Chinese officials have repeatedly stressed the importance of the AI talks in preparatory meetings with US counterparts, and view them as a major deliverable of the US-China leaders' summit, another one familiar with the matter. Reutersis reporting this development for the first time.

In the past week, China's cyberspace regulator publicly warned about "extreme AI loss of control risks" and a state media-affiliated blog criticised Anthropic, emphasising that any limits on frontier AI should apply equally to Chinese and US models. Chinese firms have in recent months unveiled what they describe as Mythos-like products.

Former Microsoft executive Craig Mundie has emerged as an important go-between behind the scenes, sounding out Chinese counterparts on US proposals for the AI dialogue, the source said. Mundie is co-chair of the unofficial US-China Track II AI dialogue channel.

Reutersreported last month that Microsoft hadcultivated a relationship with Beijingthat was among the deepest of any foreign technology company operating in China. Both sides also discussed AI guardrails at a Track 1.5 dialogue in Beijing last week, former Australian Prime Minister Kevin Rudd, a participant, wrote on social media.

Also Read:Nvidia to buy Hugging Face for $13b in big bet on open AI models. In June, Trump signed an AI executive order setting up a voluntary framework for pre-release cybersecurity reviews of frontier AI models, but the US has not released the criteria for these reviews.

Meanwhile, employees at top US AI labs including Anthropic and OpenAI have called for "pacing the frontier" to manage theglobal risks of advanced AI."The Chinese side has expressed concern around whether the US has sufficient regulation around the most advanced AI models.

Both sides are motivated to make sure they can manage a cross-border crisis effectively," said Scott Singer, co-director of the China AI Initiative at the Carnegie Endowment for International Peace.

While outcomes are likely to be limited, opening a channel where both sides could share observations and jointly monitor AI safety incidents would be an important starting point, said Samm Sacks, a senior fellow at New America, a US policy think tank. "We are at a tipping point where frontier agents can cause massive damage when unmonitored.

Both the US and China are vulnerable. This is beyond geopolitical rivalry," Sacks added. "The US and China have to come together in some form, or we're both going to lose.

"