Everything dies — baby, that’s a fact. And, if the world cannot manage the current race to superhuman artificial intelligence between great powers, everything may die much sooner than expected.
The past year has witnessed an explosion in the capabilities of artificial intelligence systems. The bulk of these advances have occurred in generative AI — systems that produce novel text, image, audio, or video content from human input. The American company OpenAI took the world by storm with its public release of the ChatGPT large language model (LLM) in November 2022. In March, it released an updated version of ChatGPT powered by the more powerful GPT-4 model. Microsoft and Google have followed suit with Bing AI and Bard, respectively.
Beyond the world of text, generative applications Midjourney, DALL-E, and Stable Diffusion produce unprecedentedly realistic images and videos. These models have burst into the public consciousness rapidly. Most people have begun to understand that generative AI is an unparalleled innovation, a type of machine that possesses capacities — natural language generation and artistic production — long thought to be sacrosanct domains of human ability.
But generative AI is only the beginning. A team of Microsoft AI scientists recently released a paper arguing that GPT-4 — arguably the most sophisticated LLM yet — is showing the “sparks” of artificial general intelligence (AGI), an AI that is as smart — or smarter — than humans in every area of intelligence, rather than simply in one task. They argue that, “[b]eyond its mastery of language, GPT-4 can solve novel and difficult tasks that span mathematics, coding, vision, medicine, law, psychology and more, without needing any special prompting.” In these multiple areas of intelligence, GPT-4 is “strikingly close to human-level performance.” In short, GPT-4 appears to presage a program that can think and reason like a human. Half of surveyed AI experts expect an AGI in the next 40 years.
AGI is the holy grail for tech companies involved in AI development— primarily the field’s leaders, OpenAI and Google subsidiary DeepMind — because of the unfathomable profits and world-historical glory that would come with being the first to develop human-level machine intelligence.
The private sector, however, is not the only relevant actor.
Because leadership in AI offers advantages both in economic competitiveness and military prowess, great powers — primarily the United States and China — are racing to develop advanced AI systems. Much ink has been spilled on the risks of the military applications of AI, which have the potential to reshape the strategic and tactical domains alike by powering autonomous weapons systems, cyberweapons, nuclear command and control, and intelligence gathering. Many politicians and defense planners in both countries believe the winner of the AI race will secure global dominance.
But the consequences of such a race are potentially far more reaching than who wins global hegemony. The perception of an AI “arms race” is likely to accelerate the already-risky development of AI systems. The pressure to outpace adversaries by rapidly pushing the frontiers of a technology that we still do not fully understand or fully control — without commensurate efforts to make AI safe for humans — may well present an existential risk to humanity’s continued existence.
An Arms Race?
The dangers of arms races are well-established by history. Throughout the late 1950s, American policymakers began to fear that the Soviet Union was outpacing the U.S. in deployment of nuclear-capable missiles. This ostensible “missile gap” pushed the U.S. to scale up its ballistic missile development to “catch up” to the Soviets.
In the early 1960s, it became clear the missile gap was a myth. The United States, in fact, led the Soviet Union in missile technology. However, just the perception of falling behind an adversary contributed to a destabilizing buildup of nuclear and ballistic missile capabilities, with all its associated dangers of accidents, miscalculations, and escalation.
Missile gap logic is rearing its ugly head again today, this time with regard to artificial intelligence, which could be more dangerous than nuclear weapons. China’s AI efforts are raising fears among American officials, who are concerned about falling behind. New Chinese leaps in AI inexorably produce flurries of warnings that China is on its way to dominating the field.
The reality of such a purported “AI gap” is complicated. Beijing does appear to lead the U.S. in military AI innovation. China also leads the world in AI academic journal citations and commands a formidable talent base. However, when it comes to the pursuit of AGI, China seems to be the laggard. Chinese companies’ LLMs are 1-3 years behind their American counterparts, and OpenAI set the pace for generative models. Furthermore, the Biden administration’s 2022 export controls on advanced computer chips cut China off from a key hardware prerequisite for building advanced AI.
Whoever is “ahead” in the AI race, however, is not the most important question. The mere perception of an “arms race” may well push companies and governments to cut corners and eschew safety research and regulation. For AI — a technology whose safety relies upon slow, steady, regulated, and collaborative development — an arms race may be catastrophically dangerous.
The Alignment Problem
Despite dramatic successes in AI, humans still cannot reliably predict or control its outputs and actions. While research focused on AI capabilities has produced stunning advancements, the same cannot be said for research in the field of AI alignment, which aims to ensure AI systems can be controlled by their designers and made to act in a way that is compatible with humanity’s interests.
Anyone who has used ChatGPT understands this lack of human control. It is not difficult to circumvent the program’s guardrails, and it is far too easy to encourage chatbots to say offensive things. When it comes to more advanced models, even if designers are brilliant and benevolent, and even if the AI pursues only its human-chosen ultimate goals, there remains a path to catastrophe.
Consider the following thought experiment about how AGI may be deployed. A human-level or superhuman intelligence is programmed by its human creators with a defined, benign goal — say, “develop a cure for Alzheimer’s,” or “increase my factory’s production of paperclips.” The AI is given access to a constrained “environment” of instruments: for instance, a medical lab or a factory.
The problem with such deployment is that, while humans can program AI to pursue a chosen ultimate end, it is infeasible that each instrumental, or intermediate, subgoal that the AI will pursue — think acquiring steel before it can make paperclips — can be defined by humans.
AI works through machine learning: it trains on vast amounts of data and “learns,” based on that data, how to produce desired outputs from its inputs. However, the process by which AI connects inputs to outputs — the internal calculations it performs “under the hood” — is a “black box.” Humans cannot understand precisely what an AI is learning to do. For example, an AI trained to “pick strawberries” might instead have learned to “pick the nearest red object” and, when released into a different environment, pick both strawberries and red peppers. Further examples abound.
In short, an AI might do precisely what it was trained to do and still produce an unwanted outcome. The means to its programmed ends — crafted by an alien, incomprehensible intelligence — could be prejudicial to humans. The “Alzheimer’s” AI might kidnap billions of human subjects as test subjects. The “paperclip” AI might turn the entire Earth into metal to make paperclips. Because humans can neither predict every possible means AI might employ nor “teach” it to reliably perform a definite action, programming away any dangerous outcome is infeasible.
If sufficiently intelligent, and capable of defeating resistant humans, an AI may well wipe out life on Earth in its single-minded pursuit of its goal. If given control of nuclear command and control — like the Skynet system in Terminator — or access to chemicals and pathogens, AI could engineer an existential catastrophe.
Arms Racing or Alignment Governance? A Risky Tradeoff
How does international competition come into play when discussing the technical issue of alignment? Put simply, the faster AI advances, the less time we will have to learn how to align it. The alignment problem is not yet solved, nor is it likely to be solved in time without slower and more safety-conscious development.
The fear of losing a technological arms race may encourage corporations and governments to accelerate development and cut corners, deploying advanced systems before they are safe. Many top AI scientists and organizations — among them the team at safety lab Anthropic, Open Philanthropy’s Ajeya Cotra, DeepMind founder Demis Hassabis, and OpenAI CEO Sam Altman — believe that gradual development is preferable to rapid development because it offers researchers more time to build safety features into new models; it is easier to align a less powerful model than a more powerful one.
Furthermore, fears of China’s “catching up” may imperil efforts to enact AI governance and regulatory measures that could slow down dangerous development and speed up alignment. Altman and former Google CEO Eric Schmidt are on record warning Congress that regulation will slow down American companies to China’s benefit. A top Microsoft executive has used the language of the Soviet missile gap. The logic goes: “AGI is inevitable, so the United States should be first.” The problem is that, in the words of Paul Scharre, “AI technology poses risks not just to those who lose the race but also to those who win it.”
Likewise, the perception of an arms race may preclude the development of a global governance framework on AI. A vicious cycle may emerge where an arms race prevents international agreements, which increases paranoia and accelerates that same arms race.
International conventions on the nonproliferation of nuclear bombs and missiles and the multilateral ban on biological weapons were great Cold War successes that defused arms races. Similar conventions over AI could dissuade countries from rapidly deploying AI into more risky domains in an effort to increase national power. More global cooperation over AI’s deployment will reduce the risk that a misaligned AI is integrated into military — and even nuclear — applications that would give it a greater capacity to create a catastrophe for humanity.
While it is currently unclear whether government regulation could meaningfully increase the chances of solving AI alignment, regulation — both domestic and multilateral — may at least encourage slower and steadier development.
Fortunately, momentum for private Sino-American cooperation on AI alignment may be building. American AI executives and experts have met with their Chinese counterparts to discuss alignment research and mutual governance. Altman himself recently went on a world tour to discuss AI capabilities and regulation with world leaders. As governments are educated as to the risks of AI, the tide may be turning toward a more collaborative world. Such a shift would unquestionably be good news.
However, the outlook is not all rosy: as the political salience of AI continues to increase, the questions of speed, regulation, and cooperation may become politicized into the larger American partisan debate over China. Regulation may be harder to push when “China hawks” begin to associate slowing AI with losing an arms race to China. Recent rhetoric in Congress has emphasized the AI arms race and downplayed the necessity of regulation.
Whether or not it is real, the United States and China appear convinced that the AI arms race is happening — an extremely dangerous proposition for a world that does not otherwise appear to be on the verge of an alignment breakthrough. A detente on this particular technological race — however unlikely it may seem today — may be critical to humanity’s long-term flourishing.