Social Scientists have warned about this danger for long; now the people building some of the world’s most powerful artificial intelligence systems are asking governments to do the same thing: put stronger controls around the technology they are creating.
Speaking before the United Nations Security Council on Wednesday, OpenAI CEO Sam Altman and Anthropic CEO Dario Amodei warned that AI’s rapid development needs safeguards to keep it from becoming too powerful to control.
International news agencies reported the executives calling for “countries to work together on rules and protections for AI”.
These warnings come as governments and technology companies grapple with how to manage systems that are becoming increasingly capable and widely used.
‘We could lose control of the future’
Altman described two very different possibilities for AI.
“AI can either be more like a new renaissance of creativity and discovery, or more like a new industrial revolution of upheaval and disarray,” he said, according to AP.
The OpenAI chief also warned that mankind could lose control of the technology's direction. Associated Press reported him as saying: “We could lose control of the future to AI.”
At the same time, Altman noted AI's potential benefits, including giving people greater power over their own lives and helping advance science and other areas of human development.
Amodei, whose company develops the Claude AI system, gave a similarly severe warning.
“If managed poorly, I even believe AI could be a risk to humanity as a whole,” he told the Security Council, according to AP.
A technology debate that is moving beyond Silicon Valley
The discussion at the UN shows a growing
shift: AI is no longer only a conversation among technology companies and researchers.
Governments are now debating how to establish safeguards, standards, and accountability while allowing the technology to develop.
The two executives also warned about the concentration of AI power. According to AP, they argued that the technology’s capabilities should not become concentrated in a single company or country.
According to Media reports, British Foreign Secretary Ed Miliband told the Security Council that the warnings from technology leaders showed why AI governance would not simply be left to the companies developing the systems.
At the same meeting, however, U.S. White House science adviser Michael Kratsios argued against creating a new global governance structure to constrain AI development. The differing positions underline the unresolved debate over how much regulation is appropriate and who should set the rules.
Why this matters for children
The debate at the center of this discussion is children growing up with AI,, and it is particularly relevant because an increasing number of children are using AI for homework and beyond. AI is fast becoming a study companion, a friend and even counsellor for many children.
As these systems become more capable, privacy, misinformation, dependence on technology, and the boundaries between human and computer interaction are likely to become increasingly important.
The UN discussion therefore raises a question that goes beyond technology: what safeguards should be in place before today's children grow up in a context where AI is embedded in almost every part of everyday life?
The message from the technology leaders was not that AI should simply be stopped. It was that its development needs safeguards, international cooperation, and mechanisms to ensure humans remain in control.
With inputs from Associated Press.