King Charles has warned AI executives of the “existential dangers” posed by the technology falling into the wrong hands.
He made the comments at a summit he had convened at Dumfries House in Ayrshire to discuss how the tech can be used to benefit society.
Those gathered at the meeting also included Kanishka Narayan, the UK’s AI Minister, the Pope’s advisor, and participants from AI giants such as Nvidia, OpenAI and Anthropic.
It comes amid a series of recent warnings from people inside the industry about the rapidly evolving capabilities of advanced or “frontier” AI models and the regulation needed to keep them in check.
“Those who have created these technologies are now increasingly warning that AI risks developing darker capacities – perhaps even to take life,” the King told the executives.
He said there seemed to be an “urgency” in “adequately considering the existential dangers of such technologies falling into the wrong hands, and being used in potentially catastrophic ways”.
“The task before you is not merely to advance technology, but to ensure that it remains firmly in the service of humanity, community and the natural world,” he told the group.
The summit has been called to consider whether a “shared set of principles” can be developed to “help guide the future application” of AI.
Jensen Huang, chief executive of Nvidia, was among top industry leaders attending.
Nvidia is a central player in AI infrastructure, supplying chips to leading AI model developers including OpenAI and Meta.
It is also the world’s most valuable company, with a stock market value of around $5.15tn.
Huang told the room he believed safety was “paramount”, adding that companies should hold back a product and “keep engineering” if it was not safe enough.
The Nvidia boss also highlighted the value of open models – systems which give users direct access to the software – as a way to “ensure that people and countries are not left behind”.
He said these systems must also be protected so “researchers, universities, start-ups and countries” can develop applications “none of us would have imagined alone”.
“Let us lead with responsible optimism, build AI safely and securely, open it to more people, expand our ambition and solve the problems that matter to the world,” Huang added.
