Top artificial intelligence executives and leading AI researchers have warned the United Nations Security Council that rapidly advancing AI systems could create serious risks for global security and human society if development outpaces effective safeguards.
The high-level briefing in New York brought technology leaders and AI experts before the 15-member council during the annual UN General Assembly gathering. The discussion focused on how increasingly capable systems could operate with greater autonomy and why governments may need stronger international mechanisms to manage the technology.
Among those raising concerns was Anthropic CEO Dario Amodei, who warned that AI could pose a broad risk to humanity if it is developed or deployed poorly. OpenAI CEO Sam Altman similarly argued that decisions about AI’s future should not be left solely to technology companies and should instead involve democratic institutions.
The warnings reflect a growing debate over how governments should manage increasingly powerful AI systems without slowing beneficial applications of the technology.
Why did AI leaders address the UN Security Council?
The Security Council traditionally deals with threats to international peace and security, including armed conflict, terrorism and weapons proliferation.
AI is increasingly being discussed in the same international-security context because advanced systems can potentially be used across multiple domains, from cybersecurity and intelligence to military planning and autonomous weapons.
The UN briefing therefore represented an effort to bring AI risks into a forum designed for international security discussions.
What did Dario Amodei warn about?
Amodei, the chief executive of Anthropic, emphasized that the consequences of advanced AI depend heavily on how the technology is developed and controlled.
His warning was not that AI will inevitably cause catastrophe. Rather, it centered on the possibility that increasingly capable systems could create serious risks if appropriate safeguards fail to keep pace.
That distinction is important because AI researchers and technology companies remain divided over the likelihood, timing and severity of potential extreme outcomes.
Amodei’s comments add to Anthropic’s broader public emphasis on AI safety and the need for safeguards around increasingly capable models.
What did Sam Altman say about AI governance?
OpenAI CEO Sam Altman also warned about the possibility of humans losing control over important decisions as AI systems become more capable and autonomous.
A central point in his remarks was governance.
Altman argued that decisions about how powerful AI systems should be developed and deployed should involve democratic institutions rather than being determined exclusively by technology companies or researchers.
That issue is becoming increasingly important as governments attempt to determine who should establish AI safety standards, how those standards should be enforced and which uses of advanced AI should be restricted.
The debate is no longer limited to technical questions about model performance. It increasingly involves questions of accountability, international coordination and public oversight.
Could autonomous AI create security risks?
One concern discussed at the briefing was the possibility that AI systems could eventually perform increasingly complex tasks with limited human intervention.
Researchers have raised potential risks involving:
- Automated cyberattacks and defensive cybersecurity systems
- AI-assisted military decision-making
- Autonomous weapons
- Manipulation or misinformation at scale
- Systems interacting in unexpected ways
- AI being used to accelerate the development of other technologies
Yoshua Bengio, an AI pioneer and co-chair of the UN Independent International Scientific Panel on AI, warned the council that governments face risks that cannot necessarily be addressed by any one country acting alone.
The concern is partly about the speed and scale at which advanced AI could operate. A system capable of rapidly analyzing information or carrying out digital tasks could potentially affect events much faster than conventional regulatory processes can respond.
However, the likelihood of the most extreme scenarios remains contested among researchers. Predictions about when, or whether, AI could become capable of causing catastrophic harm should therefore be distinguished from risks that are already observable.
Why are autonomous weapons a major concern?
AI’s military applications have become one of the most contentious areas of the international debate.
UN Secretary-General António Guterres has repeatedly called for restrictions on lethal autonomous weapons systems and argued that machines should not be allowed to make life-and-death decisions without meaningful human control.
The concern extends beyond hypothetical future systems. AI can already assist with surveillance, targeting, intelligence analysis and military planning, raising questions about where human responsibility should remain in the chain of command.
A global agreement on autonomous weapons, however, remains difficult because countries have different security priorities and views about how much autonomy military systems should have.
Why are countries divided over AI regulation?
Although there is growing international discussion about AI safety, governments do not agree on how much regulation is appropriate or what form international oversight should take.
European policymakers have generally pushed for stronger regulatory frameworks and international cooperation. The United States has simultaneously sought to maintain its position in the global AI race, creating tension between safety measures and technological competitiveness.
President Donald Trump has criticized approaches he views as excessive international regulation and has emphasized the importance of maintaining US leadership in artificial intelligence.
This creates a difficult policy problem.
Governments want to reduce the possibility of dangerous AI development while also avoiding rules that could put their companies at a disadvantage compared with competitors in countries with fewer restrictions.
Can countries cooperate on AI safety?
One proposed solution is greater international coordination.
The argument is straightforward: AI systems can be developed in one country, operated through infrastructure in another and used by people anywhere in the world. That makes purely national regulation difficult to enforce against every potential risk.
International cooperation could involve:
- Shared technical safety standards
- Information-sharing mechanisms for major AI incidents
- Common testing requirements for advanced models
- Coordination between governments during serious AI-related emergencies
- Rules governing military applications
- Greater transparency around highly capable AI systems
Some major powers are already discussing mechanisms for communicating during high-level AI security incidents.
But international agreements face the same problem that has affected other emerging technologies: countries may be reluctant to reveal capabilities or accept restrictions that they believe could weaken their strategic position.
What makes AI regulation particularly difficult?
AI is unusual because the technology is advancing while governments are still developing a common understanding of its risks.
Traditional arms-control agreements generally address physical weapons with relatively identifiable capabilities. AI systems are software, can be updated rapidly and can potentially be reproduced and distributed at comparatively low cost.
That makes enforcement more complicated.
A government could impose restrictions on companies operating within its borders, but it may have less ability to control systems developed elsewhere. The technology’s commercial applications also overlap heavily with its potential security uses.
The same underlying AI capabilities that can improve productivity, scientific research or cybersecurity can potentially be adapted for harmful purposes.
Is AI an existential threat?
The term “existential threat” is increasingly used in discussions about advanced AI, but it should be treated carefully.
Some researchers believe highly advanced AI could eventually create risks severe enough to threaten humanity’s long-term survival. Others place greater emphasis on nearer-term problems such as misinformation, discrimination, cybersecurity vulnerabilities, labor disruption and concentration of technological power.
These are not mutually exclusive concerns.
The UN briefing reflects the fact that governments are now being asked to consider both immediate and longer-term risks at the same time.
The available evidence does not establish that an existential AI catastrophe is inevitable. Nor does it establish that such a scenario can be safely dismissed. Much of the disagreement concerns uncertain future capabilities and how humans will choose to deploy them.
What happens next for global AI governance?
The UN discussion highlights a central challenge: AI development is international, but regulation remains largely national.
Technology companies are building increasingly capable systems in a highly competitive market, while governments are attempting to establish safety standards without sacrificing innovation or strategic advantages.
That tension is unlikely to disappear soon.
For policymakers, the question is therefore not simply whether AI should be regulated. It is how safety requirements can be designed, enforced and coordinated across countries while preserving legitimate uses of the technology.
The Security Council briefing marks another step in moving that discussion beyond technology companies and research laboratories and into international diplomacy.
The biggest question now is whether governments can establish common rules before the technology advances beyond the regulatory systems designed to oversee it.
