Artificial intelligence could create a new and unusually dangerous problem for Washington and Beijing: a military system acting so quickly that human leaders have only minutes to determine whether they are witnessing a technical failure, an unauthorized operation or an intentional attack. That concern is at the center of new recommendations from U.S. and Chinese security experts who are calling for explicit limits on how autonomous artificial intelligence can be used around nuclear command systems and other strategically important infrastructure.
The proposals come from Melanie W. Sisson, a senior fellow at the Brookings Institution, and Tianjiao Jiang, an associate professor at Fudan University. Both participate in a U.S.-China Track II dialogue on AI and national security convened by Brookings and Tsinghua University since 2019. Their recommendations were published ahead of planned government-level AI discussions and an expected September 24 meeting in Washington between President Donald Trump and Chinese President Xi Jinping.
The central concern is not that an AI system would necessarily “decide” to start a war. It is that increasingly autonomous systems could create an incident that looks like an act of war before human officials have enough information to understand what happened.
Why AI creates a new nuclear risk
For decades, nuclear deterrence has depended heavily on human judgment. Leaders and military commanders interpret warnings, assess intelligence and decide whether an apparent attack is real.
AI can compress that decision-making timeline.
An autonomous system could detect a cyber intrusion, classify it as hostile and initiate a response far faster than human officials could investigate the event. If the targeted system belonged to another nuclear-armed country, the response itself could become evidence of an attack in the eyes of the other side.
That creates a dangerous feedback loop:
- An AI system detects unusual activity.
- The system interprets the activity as hostile.
- An automated response affects the other country’s military infrastructure.
- The targeted country detects the response.
- Officials must determine whether it was deliberate, accidental or unauthorized.
- A second AI system could potentially react before diplomats or military leaders establish what happened.
The experts’ concern is therefore less about science-fiction scenarios and more about compressed decision times, ambiguous signals and the difficulty of attributing cyber activity during a crisis. Reuters reported that the experts are specifically concerned that Washington or Beijing could have only minutes to determine whether an AI-related incident represented an accident or an intentional attack.
What are the proposed AI nuclear red lines?
One of the clearest recommendations is to establish boundaries around nuclear command, control and communications systems, commonly known as NC3.
Sisson argues that humans should retain exclusive authority to initiate AI-enabled cyberattacks against another country’s nuclear command systems or strategically important infrastructure.
The proposal would extend an existing U.S.-China principle: artificial intelligence should not replace humans in decisions to use nuclear weapons.
The distinction matters. An AI system might assist with surveillance, intelligence analysis or defensive cybersecurity. But the decision to launch a consequential offensive operation against another state’s strategic systems would remain a human decision.
Sisson and Jiang propose that the two countries develop explicit red lines that would prohibit AI from independently launching attacks against NC3 systems or independently deciding to use nuclear weapons. Jiang’s proposal also identifies critical infrastructure such as energy, financial services and health care as areas where additional boundaries could be considered.
This would not amount to a general agreement on how Washington and Beijing should regulate military AI. Instead, it would focus on a narrow question: which decisions are too consequential to delegate to an autonomous system?
What does “meaningful human control” actually mean?
The phrase “human control” sounds straightforward until two governments have to define it.
A system can technically have a human somewhere in the decision-making chain without that person exercising meaningful control over what happens.
For example, there is a significant difference between:
- A human approving an operation after reviewing the target and circumstances.
- A human supervising a system but being unable to intervene before it acts.
- A human receiving an automated recommendation and approving it almost automatically.
- A system independently selecting and engaging a target, with humans informed only afterward.
The U.S. and China could theoretically agree that “human control” is required while still maintaining very different understandings of what that requirement entails.
Jiang therefore proposes that Washington and Beijing develop a common definition of “meaningful human control.” He suggests that regular dialogue or a terminology working group could help ensure that both sides understand the safeguards attached to the term in the same way.
That may sound like a technical drafting exercise, but the language could have operational consequences. If one country interprets human control as an approval before every offensive action while the other considers periodic supervision sufficient, the two governments could believe they have reached an agreement when they have not.
Brookings has previously highlighted the importance of developing a common glossary for AI and national-security terminology precisely because different definitions can create misunderstandings between Washington and Beijing.
The U.S. and China already have a nuclear AI principle
The proposed safeguards build on an existing point of agreement between the two governments.
In November 2024, then-President Joe Biden and Xi Jinping affirmed that humans, rather than AI systems, should retain control over decisions involving the use of nuclear weapons. Brookings describes that understanding as a starting point for expanding discussions about AI-enabled military capabilities.
The new proposal would take the principle one step further.
Instead of focusing only on the final decision to use nuclear weapons, it would address cyber operations that could affect the systems responsible for nuclear command and control.
That distinction is increasingly important as AI becomes integrated into military operations beyond traditional weapons platforms. The Brookings-Tsinghua dialogue has examined AI’s role in areas including cyber operations, intelligence, command and decision-making.
Why an AI-specific U.S.-China hotline could matter
The second major proposal is a dedicated U.S.-China military communication channel for incidents involving autonomous AI.
The reasoning is simple: if AI can accelerate military activity, communication between governments also needs to operate quickly enough to interrupt an escalating chain of events.
A hotline would not prevent an AI system from malfunctioning. It could, however, provide officials with a mechanism to ask a critical question immediately:
Was that operation intentional?
That distinction could be decisive during a cyber incident involving military infrastructure.
The problem: Existing hotlines have not always worked
The proposal also confronts an uncomfortable precedent.
During the 2023 Chinese surveillance-balloon incident, then-U.S. Defense Secretary Lloyd Austin sought to contact his Chinese counterpart after the United States shot down the balloon. China declined the request at the time. Chinese officials said the circumstances did not provide the appropriate atmosphere for dialogue.
The episode demonstrated that having a communication mechanism is not the same as having a communication mechanism that officials will actually use during a crisis.
That distinction becomes even more important with autonomous military systems. A hotline cannot reduce escalation risk if political or military leaders are unwilling to answer it when tensions are highest.
The U.S. and China would therefore need to address not only the technical infrastructure of an AI hotline, but also questions such as who can activate it, who has authority to respond and what types of incidents require immediate contact.
Why the proposals are emerging now
The recommendations reflect a broader shift in the U.S.-China AI relationship.
For years, much of the competition between Washington and Beijing over artificial intelligence centered on computing power, semiconductor access, research and advanced AI models.
Military autonomy introduces a different category of risk.
The issue is no longer simply who develops more capable AI. It is also how those capabilities behave when connected to military networks, sensors, weapons and decision-making systems.
Reuters reported this week that AI competition is expected to be part of the broader agenda surrounding the upcoming Trump-Xi talks, alongside disputes involving advanced chips, technology access and AI governance.
At the same time, neither country has adopted the Brookings experts’ recommendations as a bilateral agreement. The proposals are intended to inform discussions rather than establish binding rules.
That distinction is important. The existence of a U.S.-China expert proposal does not mean Washington and Beijing have agreed to implement these safeguards.
What could go wrong if there are no shared rules?
The most serious problem may not be an AI system deliberately starting a war.
It could be an ordinary technical failure occurring during an already tense military confrontation.
Consider a hypothetical scenario:
- An AI cybersecurity system detects unusual traffic entering a military network.
- It classifies the activity as an attack.
- A preauthorized defensive mechanism blocks or disrupts systems associated with the suspected source.
- The other government detects the disruption and attributes it to hostile military action.
- Its own automated systems begin responding.
- Human leaders enter the process after the escalation has already begun.
Every individual decision might appear defensible from inside the system that made it. The danger emerges from the interaction between multiple automated systems and two governments that do not fully trust each other.
This is why the experts’ proposals emphasize attribution, human authority and communication rather than simply asking whether AI systems are “safe.”
What would meaningful safeguards need to cover?
A workable U.S.-China framework could involve several separate layers rather than one broad AI treaty.
1. Nuclear systems
The countries could establish explicit restrictions preventing autonomous AI from independently deciding to use nuclear weapons or launch cyberattacks against nuclear command systems.
2. Strategic infrastructure
The two governments could identify infrastructure where an autonomous attack could have consequences beyond the immediate target, including energy, financial and health systems.
3. Human authority
Rules could specify which military actions require an accountable human decision rather than automated authorization.
4. Shared terminology
Washington and Beijing could establish common definitions for terms such as “meaningful human control,” “autonomous system” and “AI-enabled cyberattack.”
5. Crisis communication
A dedicated mechanism could allow officials to rapidly determine whether an AI-related incident was intentional, accidental or unauthorized.
Together, these measures would target the point where technical failures can become strategic crises.
Are these proposals a U.S.-China AI agreement?
No.
The recommendations come from participants in a long-running expert dialogue and are intended to contribute to government-level discussions. They do not represent a treaty or an announced bilateral commitment by the U.S. or Chinese governments.
That limitation is significant because implementing the proposals would require governments to agree on definitions, procedures and verification mechanisms.
There is also an underlying strategic tension. Both countries are investing heavily in military applications of AI while simultaneously trying to reduce the risks associated with those capabilities.
That makes military AI governance fundamentally different from a conventional arms-control discussion. The technology is developing rapidly, and neither side wants safeguards to become a mechanism that leaves it technologically disadvantaged.
What happens next?
The immediate test will be whether the expert recommendations move from Track II discussions into official policy.
The September 24 Trump-Xi meeting provides a potential diplomatic setting for broader discussions, although it remains separate from the experts’ recommendations and should not be treated as evidence that either government has accepted them.
For policymakers, the core question is narrower than the broader debate over AI regulation:
Can Washington and Beijing agree that certain military decisions are too consequential to delegate to machines, even while competing aggressively over AI technology?
The answer could shape how the world’s two leading AI powers manage the most dangerous edge cases of military autonomy.
The fundamental issue is not whether machines will replace human leaders tomorrow. It is whether governments can ensure that, when a military AI system behaves unexpectedly, humans still have enough authority, time and information to prevent an error from becoming a crisis.
