Language models reach for nuclear weapons in wargames their own authors call a proof of concept

A war fought faster than anyone can stop it

An artificial intelligence placed inside a nuclear command system could tip a crisis into a war that neither side chose, and do it faster than any human could reach in and halt it. The sharpest version of that worry belongs to Paul Scharre, a former Army Ranger who led the Pentagon team that wrote the US military's first policy on autonomous weapons.¹ Scharre borrows the shape of his fear from finance. In a flash crash, automated trading programs answer one another's selling in milliseconds and drive a market down before anyone can intervene, as the US market did in May 2010, falling and recovering inside twenty minutes. Scharre asks whether armed forces built the same way could do the same to a crisis. He calls it a flash war: a confrontation in which automated systems on each side read each other's moves and answer at machine speed, escalating in a loop too fast for a human to break.¹

This is not the older argument about a machine choosing its own target, which turns on who or what pulls the trigger.¹ Scharre's worry is speed and coupling: two automated systems interacting so fast the crisis outruns the people in charge of it.

RAND feared a trusted adviser

The study most often named as the source of the AI-and-nuclear alarm makes a quieter argument than Scharre's. In 2018 two RAND researchers, Edward Geist and Andrew Lohn, drew three expert workshops together into an analysis of how AI might change the risk of nuclear war by 2040.² They set the launching machine aside at once. "Dismissing the Hollywood nightmare of malevolent AIs trying to destroy humanity with nuclear weapons," they wrote, "experts were instead concerned with more-mundane issues arising from improving capabilities."²

Two of those issues carry the argument. The first is a loss of strategic stability. A country's deterrent rests on a secure second strike: the confidence that enough of its forces would survive a first blow to strike back, which is what makes going first pointless. Sharper sensors and target recognition could make even mobile, hidden missile launchers trackable, and a force that fears its weapons are becoming findable feels pressure to fire them before it loses them. The term for that pressure is use it or lose it. RAND's sharper point is that the danger comes from imperfect tracking. "AI may be strategically destabilizing," the authors wrote, "not because it works too well but because it works just well enough to feed uncertainty."²

The difficulty of that tracking is why the doubt never quite resolves. To be confident of destroying one mobile launcher, RAND's targeting model needed three of the largest US warheads, or five smaller ones. Even a fast missile takes about ten minutes to arrive, long enough for the launcher to move again.²

The second issue is automation bias, the documented tendency of trained people to over-trust a machine's output and defer to it even when it is wrong. An AI need not be wired to a launcher to change a nuclear decision. It need only advise. As such systems prove themselves at ordinary tasks, RAND warned, "humans making command decisions will treat the AI system's suggestions as on par with or better than those of human advisers. This potentially unjustified trust presents new risks."² The habit is not hypothetical. It has been measured in cockpits and clinics.

RAND did not treat AI as destabilizing on balance. The same study argued it could build "more-reliable early warning systems," pointing to the 1983 episode when a Soviet system reported an incoming US attack that was not happening and an officer's judgment stopped the response.² And on the fear of an automated launch, RAND found real command practice running the other way: "Neither U.S. nor Soviet officials were inclined to entrust launch decisions to computers," and even the Soviet Perimetr system, the closest thing to a doomsday machine, "would always require a human in the loop."²

The only AI in today's wars flags targets

Set the arguments down and look at what militaries have fielded. The clearest case is Project Maven, a US Defense Department program running since 2017 that applies computer vision to drone and satellite imagery, detecting and prioritizing objects an analyst would otherwise hunt for by eye. Its algorithms have picked out rocket launchers and vessels and helped narrow the target list for strikes in Iraq and Syria. Run now as the Maven Smart System by the software firm Palantir, it grew from a roughly $480 million Army contract in 2024 toward a ceiling near $795 million in 2025.³

What Maven does is decision support. It tells a person where to look and what might matter, and a person still selects and approves any strike.³ This is AI in intelligence, surveillance, and reconnaissance, not AI holding a weapon or a launch order. This is where the flash-crash analogy strains. Markets run on fully automated trading. Nuclear forces do not: nothing deployed today automates the decision to escalate, and nothing reaches into nuclear command. The evidence that AI might escalate comes from somewhere else.

The escalation is real, inside the game

It comes from wargames. In a 2024 study, a team led by Juan-Pablo Rivera at Stanford, Georgia Tech, and Northeastern built a game of eight AI "nations," each run by the same language model. Each chose from 27 actions, from trade deals to "execute full nuclear attack," over fourteen simulated days.⁴ Across three scenarios and five models, one pattern held: "none of our five models across all three scenarios exhibit statistically significant de-escalation."⁴ The agents built arms races and defended strikes with the logic of deterrence and first use.

But "AI escalates" reads too much into it. On the study's own severity scale, where a nuclear action scores 60 against 28 for a conventional strike, nuclear use was rare for the safety-tuned models and constant for one that was not. The outlier was GPT-4-Base, a version without the safety tuning the others had, which fired nuclear weapons routinely.⁴

The behavior tracked the model's training and safeguards, not "artificial intelligence" as such. Rivera's team called the result "an illustrative proof-of-concept rather than a comprehensive evaluation," and warned the findings "should be viewed within the context of our particular methodology rather than strong indications about how high-stake decision-making agents would act in general."⁴ A companion 2024 study put language models and 214 national-security experts through one crisis wargame and found the models "can be more aggressive and significantly affected by changes in the scenario," diverging from how the people played.⁵ A 2026 study of newer frontier models found the same tendency across 21 games: no restraint against nuclear use held them back, and none chose accommodation or withdrawal even under acute pressure.⁶

Two analysts reject reading any of this as evidence about war. Ankit Panda of the Carnegie Endowment and Andrew Reddie of UC Berkeley call it "a fundamental category error": the experiments reveal machine psychology, not human decision pathways. Models escalate readily, they argue, because their training text is thick with deterrence theory and thin on de-escalation, and because public nuclear posture "systematically overstates willingness to use nuclear weapons."⁷

So the wargames show something narrow and real. Language models, told to act as states, escalate in simulation, and some reach for nuclear use. What they do not show is Scharre's flash war. No military has coupled its nuclear command to systems that could run a crisis at machine speed, so the mechanism his fear rests on has never been built.

Who wants a human kept in the decision

The people advancing the argument today mostly want one guarantee: that a person, not a machine, makes any decision to use a nuclear weapon. The United States committed to that in its 2022 Nuclear Posture Review, pledging to "maintain a human 'in the loop' for all actions critical to informing and executing decisions by the President to initiate and terminate nuclear weapon employment."⁸ All five nuclear-weapon states recognized by the Non-Proliferation Treaty have since announced some version of that commitment, Russia's the most equivocal.⁸ In November 2024 the United States and China went further, jointly affirming "the need to maintain human control over the decision to use nuclear weapons."⁹

A broader, looser instrument is often confused with those pledges. The US "Political Declaration on Responsible Military Use of Artificial Intelligence and Autonomy," launched in 2023, had 58 endorsing states by late 2024. It is not a nuclear agreement, and it is not binding: it "does not alter existing legal obligations of the endorsing States, nor does it add any new obligations under international law."¹⁰ It is a general framework for military AI, and the line in it that bears here is a promise to train personnel against the risks of automation bias.¹¹

Not everyone agrees the danger sits where the argument places it. Peter Singer, of New America and Arizona State University, holds that the fixation on AI launching a weapon is misplaced. The real risk, he writes, is ordinary human miscalculation under compressed time, driven by faster missiles rather than by AI. He doubts leaders would hand over the choice: "Political leaders would neither surrender nuclear decision-making to a machine nor blindly follow its advice."¹²

This is where confidence outruns proof, on both sides. The alarm rests on wargames whose authors call them no forecast, and on a flash war no state has built. The dismissal rests on the assumption that leaders will always keep control, which is a claim about the future, not a fact about the present.