RAND's red-team study found no measurable AI advantage in bioweapons planning

A one-hour experiment and an argument that reverses the burden of proof

In 2023, students in an MIT course called "Safeguarding the Future" spent one hour with ChatGPT. None had scientific training. The assignment: see how far a chatbot would take someone toward the building blocks of a pandemic.¹ The model suggested candidate pathogens, explained a method to modify them, named companies willing to synthesize the DNA, and walked the group through fixes when a step failed.¹

That result crystallized the worry that still drives most of the argument. A large language model does not just answer questions. It compresses years of specialized literacy that once required a lab notebook and a mentor into a plain conversation, and it can troubleshoot a failed attempt the way a static document cannot.

The strongest version of the argument built on that finding does not claim a model makes weapons easy. It claims something narrower: that even a small boost for someone who could not otherwise finish the job is unacceptable, because a biological attack cannot be undone once it starts. The MIT team drew a practical conclusion: models should be evaluated by an independent party before release, and DNA-synthesis companies should screen orders against known dangerous sequences.¹ On that logic, proof that a model has already helped someone is not required before restricting it.

That argument is three years old now, old enough to have been tested rather than just asserted, by two public studies with openly documented methods.

Two red-team studies, one fully documented and one not

RAND ran the more demanding of the two: not whether a model can answer a biology question, but whether a team using one plans a biological attack any better than a team working from the internet alone.² RAND recruited 45 red-teamers, 27 from its own staff and 18 from Gryphon Scientific, a biosecurity firm, and split them into 15 three-person cells.² Twelve of those cells formed the actual comparison. Four attack-planning scenarios each got three cells. One worked from the internet alone. One had internet access plus a first model. One had internet access plus a second model.² Every cell had seven weeks and up to 80 hours per person to produce a written attack plan.²

Eight outside experts, four in biosecurity and four in security, scored every plan on a 9-point scale for how workable it was, biologically and operationally, reaching consensus at an in-person session.² Across all twelve cells, plans written with a model on hand scored 0.22 points lower on average than plans written from the internet alone, a difference RAND reports at p = 0.64 against the standard 0.05 threshold for statistical significance.² Split by model, one showed a 0.12-point edge over the internet-only baseline (p = 0.87), the other a 0.56-point deficit (p = 0.25). Neither cleared significance, and neither model helped consistently across the four scenarios.²

What the study measured was planning on paper. It did not test whether a team could acquire what a plan called for, or carry out a written protocol in a lab, the physical and tacit half of the worry beside the informational half.²

OpenAI ran a study the same month on a narrower question: whether a model helps a person answer the kinds of questions that attack would require along the way. Reporting on the study when it was released put the group at 100 participants, split evenly between biology PhDs with laboratory experience and university students. Each half was then split again between an internet-only control and access to GPT-4, including a custom version tuned to answer biosecurity questions instead of declining them.³ An independent academic review of the field's red-team studies, published the following year, lists the same structure and the same headline result: no statistically significant "uplift," the field's own term for the advantage AI access gives someone over working without it.⁴

On a 10-point accuracy scale, the group with model access scored 0.88 points higher than the internet-only group among experts and 0.25 points higher among students, both reported as short of significance.³

This is where the comparison with RAND has to stop, because the two studies are not documented to the same standard. RAND's report states its own p-values and its own significance threshold in the text, so the finding's exact math is there to check.² OpenAI's original write-up sits behind a bot check that blocks it from being read directly. Its participant count and scores come from press coverage written at the time, plus an independent count published a year later, not from OpenAI's own text. No primary or secondary source states what threshold OpenAI used to call its own result insignificant.³⁴ A later OpenAI system card confirms the study's three-arm design, internet alone, the production model, and the tuned research-only model, but not the original sample size or statistics.⁵

Soice's demonstration and these two null results are not really the contradiction they look like. An independent review that counted OpenAI's 100 participants also flags a gap in the MIT team's design. They never compared what the model produced against a plain internet search. That study has no baseline for saying the model added anything the internet did not already give a determined searcher.⁴ RAND and OpenAI both added that baseline, and against it, model access stopped looking like an advantage.²³⁴

Two companies crossing their own thresholds, on different timelines

Despite those results, by 2025 both companies had activated their most cautious biological-risk policy for a released model, a fact often cited as the industry's own admission that the risk is real. The record supports a narrower reading.

Anthropic released Claude Opus 4 in May 2025 under what it calls the ASL-3 standard, a tier of safeguards focused mainly on biological risk, above the tier its other models carry.⁶ The company's own system card explains the decision plainly. Its evaluations ruled out the tier above ASL-3. They could not rule out that Opus 4 had the capability ASL-3 guards against. Anthropic deployed the model with ASL-3 protections anyway, calling the move precautionary.⁶ That is a company acting on what it could not disprove, not a report that the model passed a test like RAND's or OpenAI's.

OpenAI's parallel decision is usually paired with the 2024 studies, but the record runs on a longer clock than that shorthand suggests. OpenAI defined a "High" capability tier for biological and chemical risk in April 2025. The definition: a model that meaningfully helps a technically capable novice reproduce a known biological or chemical threat, benchmarked against what the internet alone already offered in 2021.⁷ A model reaching High is not released until the risk it poses has been brought down first.⁷ OpenAI first applied the classification to a named model, ChatGPT Agent, in July 2025, and again to GPT-5-thinking in August 2025, with the same reasoning both times.⁸⁹ Neither is GPT-4, the model the 2024 studies tested.⁸⁹

Both system cards say plainly that OpenAI does not have definitive evidence the model actually clears the High bar. The GPT-5 card goes further. It describes the model as still short of that capability, and frames the classification as preparing the organization for what comes next rather than confirming a risk that is already there.⁸⁹

So the two clearest "labs act" data points sit further from the 2024 studies than the usual pairing implies. Both rest on the same admission: caution taken ahead of proof, not because of it.

The model this argument is really about has not been tested

The most careful independent review of this evidence, published in 2025, lands in between. Current large language models do not appear to pose an immediate biological risk. But the studies establishing that are still few and young, so the finding is provisional, not settled.⁴ The RAND and MIT teams describe their own results the same way, as exploratory rather than conclusive, and neither rules out a more capable model changing the picture.⁴

Reading RAND's null result as the last word on the argument goes further than RAND itself does. The study tested one stage, planning by an already-assembled team, using the LLMs available in summer 2023. It says nothing about whether a newer, more agentic model would behave differently. It says nothing about the earlier stage Soice's team tested, where a model measurably helped organize and explain information a person still had to act on.¹²⁴

Alarm runs into the opposite problem. It treats a one-hour classroom result, or a company tightening its own safeguards, as proof a model can already carry someone through an attack. The two studies testing that harder claim came back null or marginal, and the labs' own documents plainly say they lack the evidence too.²³⁸⁹

What none of the cited studies test is the version of this argument that worries people most. That version needs a more agentic model with real-world tool use, one that runs an experiment or troubleshoots a failed step on its own, instead of answering a question a person then carries out by hand. That system is not the one RAND, OpenAI, or the MIT team sat down with; a deployed model with that kind of autonomy barely existed when these studies ran. Until one is tested as plainly as its predecessors were, the evidence available settles neither the alarmed reading nor the dismissive one.