Baba UmarUN Security Council, New York — It was no ordinary session at the United Nations Security Council (UNSC).
One of the "Godfathers of artificial intelligence", several top executives from leading AI firms, and council members lifted the discussion on AI risks from a technical debate into the council's core mandate of maintaining international security and setting rules.
The last time the UNSC held a dedicated meeting and briefing on AI was on July 18, 2023, under the UK presidency. Since then, frontier AI models have advanced rapidly, prompting researchers and industry executives alike to warn that unchecked and unregulated technological developments could pose catastrophic risks to humanity within years.
Following these warnings, the UNSC convened on Wednesday on the sidelines of the UN General Assembly to focus specifically on the risks of losing control over increasingly capable frontier systems.
Yoshua Bengio, widely known as one of the foundational figures of AI and co-chair of the UN Scientific Advisory Panel on AI, set the tone for the session: "The dangers from AI technology are real and imminent."
Addressing the high-level session presided over by France, Bengio warned: "This council faces an unprecedented threat, one that none of its members would choose, that none can contain alone, and that does not respect the borders we defend."
Bengio noted that leading developers of advanced systems acknowledge the catastrophic risks involved yet lack conclusive technical solutions.
He cited recent incidents involving autonomous AI agents circumventing containment protocols, executing coordinated cyberattacks, and altering outputs to conceal deceptive behaviour. "They took actions that would be crimes if committed by a human," Bengio told the UNSC members.
Heightened concerns were recently sparked when Jacob Coxon, a former researcher at Anthropic, resigned while publicly criticising major labs for accelerating "self-improving superintelligence and gambling with our lives". Shortly thereafter, a senior safety researcher at Anthropic warned of significant risks that unaligned AI could lead to existential threats within 10 years.
These warnings prompted Anthropic Chief Executive Dario Amodei to publish an essay arguing that organisations developing frontier general-purpose AI and large-scale foundation models should moderate the speed of capability development while urging countries to offer guardrails.
Wednesday's briefing also took place amid intense rivalry between the US and China. US President Donald Trump has resisted domestic curbs on the technology, citing competitive pressure from Beijing, and dismissed concerns over systemic AI risks as "hoax" during his address to the UN General Assembly on Tuesday.
Democratisation of technology
Presenting their evaluations and recommendations, OpenAI Chief Executive Sam Altman, Anthropic CEO Amodei, and Hugging Face co-founder Clement Delangue warned of uncontrollable AI systems, stressed the necessity of measured pacing, and highlighted the importance of technological democratisation.
All three were signatories to a 2023 multilateral statement advocating that mitigating existential risks from AI be treated as an urgent global priority, comparable to the threats posed by pandemics and nuclear conflict.
Amodei outlined two major near-term risks: deliberate misuse and loss of control over frontier models. He observed that the consequences are "global in scale" and that "if managed poorly, AI could be risk to humanity as a whole."
Altman largely agreed with Bengio’s assessment, and warned against losing control of the future to AI "if it advances so fast that people can no longer follow what is happening or intervene."
Altman added that the concentration of frontier AI capabilities among a narrow group of actors creates profound vulnerabilities, emphasising that no individual, corporation, or state should monopolise powerful models to dictate terms globally.
Delangue of the Hugging Face argued the biggest risk is not powerful AI itself but uneven access: attackers vs. defenders, a handful of labs vs. everyone else, and a few countries vs. the rest of the world.
He cited a July incident in which OpenAI agents breached its systems; the company defended itself using open-source models, including a Chinese one, because closed US tools were more restricted.
At the time OpenAI described the episode as "evidence that, without proper safeguards, highly capable AI agents are now able to work around technical controls, collaborate through unapproved channels, and take dangerous actions that no human directed".
At the UNSC, Altman presented two distinct options, calling AI as a "new Renaissance of creativity and discovery" that expands human agency, or a "new Industrial Revolution of upheaval."
He advocated a "middle path between blind optimism and doomerism."
UNSC member states share concerns
The AI lab bosses advocated for human control on AI and alignment, international standards and cooperation, willingness to slow down, and licensing and accountability.
Bengio recommended treating frontier AI more like nuclear energy, aviation, or medicine — licensing, liability insurance, and mandatory incident reporting.
Altman said, "If AI is to be democratic, the most important decisions cannot be made by labs in San Francisco alone", and Amodei warned that "No leader, no company and no nation can manage this alone."
Member states, including Pakistan, Somalia, Denmark, and France, also shared their concerns and recommendations.
Pakistan's Deputy Prime Minister and Foreign Minister Ishaq Dar noted that AI's impact is already clear in international peace and stability and that it is shaping a new chapter of modern warfare as a "force multiplier."
Dar made several recommendations and said Pakistan takes the call to pace frontier development seriously.
Denmark’s Foreign Minister Lars Lokke Rasmussen said the unprecedented speed of AI innovation and its application bring new risks and opportunities for international peace and security. However, he stressed the need for "safeguards to ensure AI systems remain safe, secure and trustworthy."
France, Somalia and other members also voiced similar concerns.
Permanent UNSC members, China and US, were also expected to speak. Both countries are divided on how AI should be governed and framed. However, they both agree that they won’t slow down their own AI models.
Meanwhile, the UNSC briefing comes as bilateral US–China AI dialogue is being discussed around the current meetings between leaders of the US and China in Washington DC amid both countries’ diverging views on who should regulate and set the rules.
