Rich Sutton's essay has four examples and zero citations

Sutton read seventy years of AI history down to one page

"The Bitter Lesson" is about 1,100 words, posted on 13 March 2019 on a personal website with no institutional letterhead, signed only "Rich Sutton" and a date.¹ It has no data table, no chart, and no citation of its own. The argument rests entirely on Sutton's own reading of AI's history. He co-authored the field's standard reinforcement-learning textbook, and in March 2019 held two research posts at once: professor at the University of Alberta, and distinguished research scientist at DeepMind. Five years later he and his co-author, Andrew Barto, won the 2024 Turing Award for the foundations of reinforcement learning.²

The essay's claim arrives in its first sentence, stated as an established finding rather than a proposal. "The biggest lesson that can be read from 70 years of AI research is that general methods that leverage computation are ultimately the most effective, and by a large margin. The ultimate reason for this is Moore's law, or rather its generalization of continued exponentially falling cost per unit of computation."¹ The mechanism is simple. Computer time keeps getting cheaper. Hand-built domain knowledge does not. A method built to use more compute eventually beats one that depends on a person's understanding of the problem, however sophisticated that understanding was at the start.

Sutton's clearest worked example is computer chess. In 1997 IBM's Deep Blue beat the reigning world champion, Garry Kasparov, in a six-game rematch, 3.5 games to 2.5, evaluating 200 million chess positions per second across 32 processors.³ Chess researchers of the time had built their systems on human understanding of the game's special structure. Sutton describes them losing to "massive, deep search" instead: "these researchers wanted methods based on human input to win and were disappointed when they did not."¹

Sutton draws four steps from that pattern, common to all four examples. Researchers build knowledge into a system, which "helps in the short term" and satisfies the researcher. "In the long run it plateaus and even inhibits further progress," until a general method wins instead by search and learning.¹

Chess and Go carried more human design than the essay says

Sutton names Deep Blue as evidence that search alone beat the field's prior approach. IBM's own account describes something more particular. The team that built the 1997 version "improved the databases dealing with chess endgames, created a more powerful evaluation function for chess positions, hired additional chess grandmasters to advise the team." IBM's C. J. Tan put it this way: "Garry prepared to play against a computer. But we programmed it to play like a grandmaster."³ The 200 million positions a second did the work Sutton credits. What they were scored against was tuned by people the essay never mentions.

Sutton's essay treats Go as one case, not two: "enormous initial efforts went into avoiding search by taking advantage of human knowledge... all those efforts proved irrelevant, or worse, once search was applied effectively at scale."¹ But the program that first beat a human champion this way, AlphaGo, was not knowledge-free. Its 2016 Nature paper says its networks "are trained by a novel combination of supervised learning from human expert games, and reinforcement learning from games of self-play."⁴ Before it ever played itself, it studied thousands of human games. Playing that way, AlphaGo won 99.8% of its games against other Go programs and beat the human European champion, Fan Hui, five games to none.⁴

The essay's stronger case exists, under the same name as the weaker one. AlphaGo Zero, published before Sutton's essay, dropped the human game data entirely and started from random play. After three days training against itself, it beat the previously published AlphaGo 100 games to 0. After forty days it outperformed a further version known as "Master."⁵ That is the result Sutton's description of Go actually fits: search and self-play, no human data at all. His essay names it "Go" without telling which of two systems, one built on human data and one without it, the sentence is about.

Kaplan and Chinchilla measured what Sutton only observed

Sutton's argument is an induction: a pattern read off four historical cases by one person, no denominator, no held-out test, no citation trail. Two papers, both filed under the name scaling laws, did something different. They ran experiments and fit curves to the results.

Kaplan et al.'s 2020 paper measured how a language model's loss changes with model size, training data, and compute. The relationship followed a power law, "with some trends spanning more than seven orders of magnitude."⁶ Two years later, Hoffmann et al.'s Chinchilla paper tested how a fixed compute budget should split between model size and training data. It trained over 400 language models, from 70 million to more than 16 billion parameters, on 5 billion to 500 billion tokens.⁷

The two papers also corrected each other, which an essay with no data of its own cannot do. Kaplan's headline recommendation was to train "very large models on a relatively modest amount of data."⁶ Hoffmann's team, with 400 more trained models to check that recommendation against, found current large models were "significantly undertrained." For compute-optimal training, model size and token count should grow together, not stay fixed.⁷ Their own compute-matched test model, Chinchilla, had 70 billion parameters and four times the training data of the 280-billion-parameter Gopher. It beat Gopher, GPT-3, and two other larger models across a range of benchmarks, including a score of 67.5% on the 57-subject exam MMLU, more than 7 points above Gopher.⁷

That is what measured evidence about compute looks like: hundreds of trained models, curves spanning orders of magnitude, an earlier recommendation checked and revised by more experiments. Sutton's examples are real, but his essay predates this kind of test and offers nothing comparable in its place. It is a pattern noticed, not a pattern measured.

The transformer and RLHF are the human design that stayed

The essay's citation record shows how far its four examples now travel. Wikipedia's page on the essay counts "hundreds of formal citations according to Google Scholar," including a 2022 benchmark paper that argues against spending research effort on problems "likely to be solved by scale alone."⁸

Those citations describe systems built on an architecture Sutton's essay never mentions. The Transformer, introduced by Vaswani and colleagues at Google in 2017, replaced recurrent and convolutional networks with attention, a specific, hand-designed structural choice. It beat the previous best machine-translation results: 28.4 BLEU on one 2014 English-German benchmark, more than 2 points above the prior best including model ensembles.⁹ The Transformer is the substrate every model in Kaplan's and Chinchilla's curves runs on. Citing Sutton to say architecture stopped mattering runs into that paper's own premise: someone still had to design it.

Scale alone did not produce the systems users actually noticed, either. OpenAI's 2022 InstructGPT paper fine-tuned GPT-3 on demonstrations and rankings written by human labelers. In human evaluations, outputs from the resulting 1.3-billion-parameter model were preferred to outputs from the 175-billion-parameter base model, "despite having 100x fewer parameters."¹⁰ A smaller model, shaped by curated human feedback, beat a larger one on the task users judged it on. That is not the outcome the essay's opening claim predicts for a model with a hundred times fewer parameters.

Rodney Brooks, a robotics researcher who has built systems on hand-designed structure, published a response six days later. His target is not the chess or speech examples but the architecture claim: "the very essence of CNNs is that the front end of the network is designed by humans to manage translational invariance." More broadly: "for most machine learning problems today a human is needed to design a specific network architecture for the learning to proceed well."¹¹ His example: a stop sign with a few pieces of tape on it, misread by a trained self-driving CNN as a 45 mph speed limit sign. "No human makes that error because they know that stop signs are red, and speed limit signs are white. The CNN doesn't know that."¹¹ Brooks does not contest Sutton's narrower claim, that hand-coded heuristics lose to search and learning. He targets what happens when that claim gets stretched to cover architecture and curated training signal too, exactly the stretch the essay's own citation record shows happening.