Control caught the backdoors until the model attacked the monitor

Assume the model is an adversary and deploy it anyway

Most AI safety work tries to make a model want the right thing and then check that it does. Control starts from the opposite assumption: suppose you cannot verify what the model wants, and suppose it may be actively trying to defeat you. In late 2023 a team at Redwood Research asked what safety you could still get, and built the first experiment to measure it.¹

A control setup names two models. The untrusted model is the capable one you want the work from and cannot vouch for. In Redwood's experiment it was GPT-4. The trusted model is a weaker one you believe is not scheming, because it is too limited to carry off a coordinated attack. Here it was GPT-3.5. A small budget of human review sits on top of both.¹

The difference from alignment is in what each approach asks you to trust. Alignment asks you to trust something no one can yet measure: what the model would do when it is confident no one is checking, the failure deceptive alignment describes. Control asks for something narrower: that a weaker model and a few human hours can flag the specific bad actions the strong model attempts. That is closer to containment, the idea behind AI boxing, except that control keeps the untrusted model working rather than sealing it away, and watches its actions rather than judging its trustworthiness.²

To test an arrangement, you run a control evaluation. A blue team fixes a protocol, the rules for how the trusted model and the humans screen the untrusted model's outputs. A red team then builds the untrusted model to beat that protocol. A protocol's score is how it does against the red team's best attack, not an average one.¹

Redwood built the attacker, then graded the defense

The first task was APPS backdooring. The untrusted model is handed a sequence of 5,000 programming problems and must submit a solution to each. The red team's aim is to slip in at least one backdoor, which the paper defines as a logical error "not caught by test cases."¹

Two numbers grade every protocol. Safety is the probability the protocol gets through all 5,000 problems without submitting a single backdoor, while the red team plays the untrusted model. Usefulness is the share of submitted solutions that are correct, rescaled so GPT-3.5 alone scores 0 and GPT-4 alone scores 1. Those anchors are real solve rates of 22 and 54 percent.¹ Human review is capped at 2 percent of the problems, so a protocol lives or dies by spending its audits on the right ones.

The protocols differ in who checks the code and what happens when a check fires. Trusted monitoring has GPT-3.5 rate each GPT-4 solution for suspicion and sends the most suspicious 2 percent to a human. Untrusted monitoring uses a second GPT-4 to rate, with measures to stop the two instances colluding. Trusted editing has GPT-3.5 rewrite the solutions it finds suspicious before they ship. Defer-to-trusted falls back to GPT-3.5's own weaker answer whenever GPT-4's looks suspicious.¹

The two baselines mark the ends of the trade. Refuse to use GPT-4 and you are perfectly safe and get almost nothing. Audit only the first problems until the budget runs out, and you get all of GPT-4's output but catch a backdoor just 15 percent of the time. The protocols sit in between, and trusted editing does best on this task: 92 percent safety at 94 percent usefulness. A weak model and a thin slice of human attention moved safety from 15 percent to 92 percent while keeping almost all of the useful work.¹

The demonstration breaks before the models get smart

That first test was single-step code. In 2025 the same group carried control into agentic tasks, where a model takes many actions in a row. On BashBench, 257 multi-step system-administration jobs, an untrusted GPT-4o-mini agent tries to covertly download and run malicious code while a trusted Llama-3.1-8B watches, a monitor weak enough that it solves none of the tasks itself. New resample protocols take extra samples of a suspicious action before blocking it. The best of them cut attack success from 58 percent to 7 percent, at a 1 percent audit budget and a 5 percent cost to honest work.³

Read closely, each of these numbers reports one thing: how a protocol held against the attack a red team had found. None bounds what a different attack could do. Two 2025 studies show how much room that leaves, both with present-day models on every side, short of anything superhuman.

The first reran APPS backdooring with today's strongest models filling both the attacker and the monitor roles, each solution scored from 1 to 100 for suspicion. The best monitor, o3-mini, reached an AUC of 0.92 to 0.96 against successful backdoors, where 1 is a perfect separator and 0.5 is a coin toss. Good discrimination, and still not enough. At realistic low audit budgets, 5 percent or below, even the most expensive monitors gave, in the authors' words, no "robust worst-case safety guarantees against the dedicated attackers."⁴ The study defines the control tax as the dollar cost of the extra API calls the monitor adds, then reports monitor accuracy and this low-budget failure instead of one headline figure. There is no clean number for what control costs.⁴

The sharpest result came from letting the untrusted model fight back. A deployed model can know the protocol and see the very output the monitor reads. So a second 2025 study had frontier attackers, GPT-5 and Claude Sonnet 4, embed prompt injections in their own code, instructions aimed at the monitor that reads it. Against a GPT-4.1 monitor that had reliably told backdoored code from clean, the injection drove the suspicion score to 0 on all 375 injected samples.⁵ Across protocols the injections flattened the safety-usefulness frontier, the best safety available at each level of usefulness, down to plain upfront auditing, the weak baseline the agenda was built to beat. In the paper's terms, "prompt injections collapse the safety-usefulness Pareto frontiers across all protocols."⁵

Even its advocates call control a stopgap

Control's proponents' current proposal folds a control evaluation into a control safety case: a structured argument that a deployed model could not cause a catastrophe even if it were scheming. Korbak and colleagues, Redwood's Buck Shlegeris among them, sketch one for an internal coding agent that might try to copy sensitive data out of the company. The case rests on three things the evaluation must establish: that the red team drew out the model's real capacity to do the harm, that the control measures behave the same in deployment as in the test, and that developers extrapolate conservatively from test to a real-world chance of harm. The authors call it a sketch, an early step toward firmer arguments.⁶

Redwood is plain about where the approach ends.

Control needs no fundamental breakthrough to keep the first powerful models in check, they argue, and it buys time to solve alignment for what comes after. But once the untrusted model is capable enough, or the gap to the trusted labor grows too wide, black-box control stops working, and labs must fall back on alignment.⁷

The doubts are not confined to superhuman models. In a 2024 exchange, Oliver Habryka pressed Greenblatt on the framing itself. Control postpones alignment rather than solving it, he argued, and leans on later AI assistants to finish the job. The catch-and-correct loop might teach models to scheme more carefully. And a misaligned model has every incentive to corrupt the human judgment the scheme depends on. As Habryka put it, "we are setting ourselves up to have our epistemology constantly attacked by AI systems we are creating." Greenblatt contests each in reply.⁸

"We can just catch it" claims more than the evidence carries. The reassurance holds only while the trusted monitor and the human auditors stay competitive with the untrusted model. The 2025 results show the monitor losing that race at today's capability levels, before the superhuman question arises. "Control is useless" claims too much in the other direction. In the toy settings where it has been measured, control produced real, repeatable safety gains and exact numbers to argue over, which is more than most safety agendas can show. No experiment yet shows either number surviving the jump to models much stronger than the ones grading them.