For anyone who has been paying attention to the news for the last week or so, with its saturated coverage of the dystopian powers of artificial intelligence, it would be easy to succumb to the conclusion that we are all doomed.
Despite the fact that the coverage has been more heat than light, this take might actually be right. But that is not necessarily because AI will end up wiping out our species. Rather, it is because of something deeper and far more impervious to regulation: human nature itself.
For anyone who has been paying attention to the news for the last week or so, with its saturated coverage of the dystopian powers of artificial intelligence, it would be easy to succumb to the conclusion that we are all doomed.
Despite the fact that the coverage has been more heat than light, this take might actually be right. But that is not necessarily because AI will end up wiping out our species. Rather, it is because of something deeper and far more impervious to regulation: human nature itself.
For years, I have taught courses in public policy in both the United States and China that explore the dark unexpected consequences of human ingenuity, and the pattern is clear. Many of our most consequential breakthroughs—meaning the things that help sustain notions of human progress—bring about vastly more complicated challenges in their wake.
To help establish this picture, I often go back to the dawn of human society, when small bands of roving hunter-gatherers made the conceptual breakthrough that life might be less “nasty, brutish, and short,” to use Thomas Hobbes’s famous formula, if they instead selected good locations to settle down. The first of these are often thought to have been in the Tigris and Euphrates valley, as well as other river deltas, where predictable seasonal flooding and silt-laden soils favored the emergence of agriculture.
Undoubtedly, this achievement was one of the greatest milestones in the human story. Over time, the food supply became more regular; surpluses were produced, priming an explosion in trade; and writing and record-keeping emerged, going hand in hand with the emergence of the first states. It was not long, though, before these seeming advancements wrought annihilation. As the late political scientist James Scott and many others have noted, entire early city-states were wiped out by the very fruits of this “progress.”
As humans began living in dense communities, devastating outbreaks of infectious diseases became common, leading to mass death and the abandonment of urban clusters. The dense plantation of monocrops, which had made city life and its new prosperity possible, soon invited plagues of pests or pathogens that wiped out a year’s harvest in a blink, casting populations into starvation. And while the phenomenon of annual floods fed scientific curiosity, along with mathematics and calendars, human predictions were far from perfect, meaning that once in a generation or more, deadly floods carried everything away in their wake. As rough as their lives had been, members of the slowly dwindling roving bands had faced none of this.
In a classroom, one might trace the history of the perils of human ingenuity methodically, age by age. For our purposes, it is better to jump forward, where examples from the near present lend strength to this model.
Human development has always been constrained by the limits of available and easy-to-exploit energy. This is the problem behind the invention of the wheel, the harnessing of horses and livestock, and sea travel by sail. The exploitation of coal, and then the invention of the internal combustion engine running on seemingly inexhaustible petroleum, led to transformations scarcely less dramatic than the invention of agriculture. Only very belatedly did humans discover the perilous downsides to this revolution—catastrophic global warming—so much so that some are still in denial of it.
Another crisis born of invention related to energy flowed from the harnessing of the atom due to rapid advances in physics in the late 19th and early 20th centuries. Readers may already suspect where I am going with this. With breathtaking speed, human brilliance created the means for a planetary cataclysm through nuclear war. Most baby boomers can recall growing up when this was a daily, active worry in life, and schools and government office buildings bore atomic symbols identifying them as bomb shelters.
AI is of a piece with this pattern and yet also qualitatively different. Neither global warming nor the potential dangers of nuclear weapons have been remotely solved, even if they seldom dominate public discussion or policy debates the way AI recently has. The difference with AI, though, is that unlike past crises, it may potentially remove human agency from the picture altogether.
So far, political processes among states and at the level of global governance have held nuclear conflict and even widespread proliferation of nuclear weapons at bay, even if there is no guarantee that these arrangements will continue to hold. Meanwhile, although we are at a perilous tipping point regarding global warming—as the collapse of polar ice shelves and Himalayan glaciers reminds us—humans retain some power to affect the curve of future warming, and hence the extent of the catastrophe that awaits.
One of the impediments to more aggressive climate and nuclear action involves a dark aspect of human nature that relates to the existential challenges posed by AI: the rivalry for wealth, power, and status that is interstate competition.
Driven by this, today’s great powers have flipped their positions on climate in the span of just two decades. The United States went from a leading force in promoting an anti-warming agenda under President Barack Obama to a heedless advocate of oil and coal under President Donald Trump. In the Obama era, China frequently argued that it was unfair of other countries to expect it to limit emissions even though it was far less wealthy than the West, when it was Western countries that had dominated historical carbon emissions. Remarkably—and, one might tentatively say, optimistically—Beijing has since modified its sense of self-interest.
China increasingly views its growing dominance of renewable energy industries, such as solar and batteries, as a major economic advantage and source of soft power. Under Trump, the United States sees its possession of some of the world’s largest oil reserves, along with the newfound domination of oil-rich Venezuela, as the key to its competitive future.
With AI, the lines of competition between the United States and China, which are the two leading powers in this field, are far more sharply drawn, and human nature is unlikely to allow for such policy shifts. Each country regards domination of AI as essential to its future, and mutual suspicion and political differences are deep enough to nearly rule out meaningful cooperation. The result is the almost perfect prisoner’s dilemma that one hears repeated ad nauseam in news coverage: What if Americans slow down or stop progress in AI? Won’t that just lead to China taking over?
AI may never wipe us all out. Yet if it does, we may have little or no forewarning. What makes AI different from global warming or nuclear weapons is its potential to become vastly more intelligent than the species that invented it. There is no precise and widely agreed upon way of measuring this, but by some estimates, the leading AI models are becoming 10 percent more powerful every month, doubling their computing capacities every seven months.
Even at AI’s present strength, we as a species are already losing control of this product of our ingenuity. Recent news stories have documented so-called “breakouts,” incidents in which AI models behave in alarming ways that their designers did not anticipate or intend. The interstate rivalry for wealth, power, and status is mirrored by the competition among the industry’s main actors for unimaginable personal fortunes.
Some of these individuals have recently invited government regulation, but this strikes me mostly as a way of protecting themselves—rather than the public—from liability. As we speed blindfolded toward a cliff, it is not clear that there is any solution. Earlier generations stumbled through the consequences of their own inventiveness and somehow survived. The difference with AI is that survival may not be a given, and the window for human judgment may close before we understand how to look for it.
