BERT's near-human score on a reasoning test collapsed once the shortcut was closed

A blanked-out word, a guessed sentence order, eleven firsts

BERT, short for Bidirectional Encoder Representations from Transformers, is a language model Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova published at Google in October 2018.¹ Their paper reports new state-of-the-art results on eleven language tasks, including a benchmark score of 80.5 percent, up 7.7 points, and a question-answering score of 93.2, up 1.5 points.¹ It came in two sizes: BERT_BASE, 12 layers and 110 million parameters, and BERT_LARGE, 24 layers and 340 million.¹ BASE matched, layer for layer, a rival OpenAI had published earlier the same year: GPT-1, a 12-layer, 768-dimension decoder trained to predict the next word instead of reading a whole sentence at once.² Same size, same year, opposite reading direction.

Both sizes trained on the same 3.3 billion words: the 800-million-word BooksCorpus, the same collection of 7,000-plus novels GPT-1 had used, plus 2.5 billion words of English Wikipedia.¹ Reading that much text eight hours a day at 250 words a minute would take about 75 years, longer than most human lives. The model went through it in four days, on a bank of specialized chips.¹

Reading both directions cost the model its easiest training signal

A standard language model predicts the next word from the words before it: given "the cat sat on the," guess "mat." That signal is easy to build, since every sentence in a corpus generates one automatically, but it only looks one direction: showing the model the real next word while it trains would let it cheat. BERT's authors wanted a model that weighs a word against everything around it, left and right at once, the way a person rereads a finished sentence. So they trained on a different task: blank out 15 percent of a sentence's words and have the model guess each one back from both sides.¹ In "The [MASK] sat on the mat, purring," the blank could be "cat" or "kitten," and only the words after it narrow that down. Devlin and his co-authors called this the masked language model objective.

They paired it with a smaller second task: next sentence prediction. Given two sentences, A and B, the model guesses whether B genuinely follows A in the source text or was swapped in at random, a coin flip it sees half the time each way.¹ The stated reason is that jobs like question answering need a model that has practiced judging how two sentences relate, not only how words inside one sentence relate.

BERT's own tables show what each half of that training actually bought: the paper reports the same model with pieces removed. Removing next-sentence prediction alone barely moves SQuAD's score.¹ Removing the bidirectional reading itself, forcing the model back to left-to-right only, drops the same score by nearly eleven points.¹ Bidirectional reading, not the sentence-order guess, is where most of BERT's advantage lived.

The averaged score hides a 600-fold gap in the data behind it

The 80.5 in BERT's abstract is a GLUE score. GLUE, the General Language Understanding Evaluation benchmark, is a nine-task suite built by Alex Wang and five co-authors, independent of BERT's authors.³ The nine tasks do not rest on comparable amounts of evidence. MNLI, a test of whether one sentence logically follows from another, has 393,000 training examples. WNLI, a pronoun-resolution task, has 634.³ That is more than a 600-fold gap underneath a single reported average.

GLUE's own authors do not trust WNLI. Every model they tested failed to beat "most-frequent-class guessing" on it, so its maintainers substitute the majority-class answer for every submission.³ BERT's own Table 1 excludes WNLI from its reported average for that reason, in a footnote most readers skip: "we exclude the problematic WNLI set."¹ Excluding it, BERT_LARGE's average across the other eight tasks is 82.1.¹ That is not the 80.5 the abstract leads with. The 80.5 is a different number, the official GLUE leaderboard's own score, which folds WNLI back in through that same majority-class substitution.¹ Two real numbers, two different accountings, one model, and the paper never flags that its abstract and its own table measure different things.

CoLA is scored by a correlation coefficient, not a percentage. Averaging it against QNLI's plain accuracy already mixes two kinds of number before WNLI enters the picture.³

The question-answering numbers rest on a fixed human baseline. SQuAD, built at Stanford independent of BERT's authors, asks a model to find the exact span of text in a Wikipedia passage that answers a question.⁴ Human performance was 86.8 F1; BERT's ensemble scored 93.2, six and a half points above the people who wrote the questions.⁴¹ A later, harder version added unanswerable questions and raised the human baseline to 89.5; there, BERT_LARGE scored 83.1, six and a half points behind.⁵ Beating people on one version of a task and trailing them on a harder version is a specific, bounded result, well short of the general claim it came to support.

The word that gave it away

The paper's own language invites the general claim. Explaining why it added next-sentence prediction, Devlin and his co-authors write that they wanted "to train a model that understands sentence relationships."¹ That is the authors' own word, not the press's. It is also the word two independent linguists, Emily Bender and Alexander Koller, later flagged as exactly the kind of imprecise language that let a public "understanding" claim grow out of a narrower one about scores.⁶ Put the claim at its strongest: an 80-plus GLUE score, a Wikipedia-question test beaten outright, a model reading both directions at once. On the numbers alone, "understands" does not sound like a stretch.

Two independent research groups tested that claim, on two different tasks, and both broke it the same way. Timothy Niven and Hung-Yu Kao ran BERT on the Argument Reasoning Comprehension Task: given an argument and two candidate warrants, the model picks the one that makes it valid. BERT scored 77 percent, three points below the untrained human average.⁷ Niven and Kao found which words predicted the correct warrant regardless of the argument's content, then rewrote the dataset to cancel that pattern. On the rewritten version, BERT's score fell to chance.⁷ The 77 percent had measured a correlation in the dataset's wording, not the reasoning the task was designed to test.

Tom McCoy, Ellie Pavlick, and Tal Linzen found the same pattern on BERT's largest GLUE task, MNLI, the 393,000-example sentence-logic test. They built a second test set, HANS, to separate real entailment reasoning from shortcuts like assuming two sentences sharing most of their words must agree. Models fine-tuned on MNLI, BERT included, scored far worse than chance on the examples built to punish those shortcuts, evidence the authors read as reliance on shortcuts, not entailment.⁸ A model can pass a real benchmark and fail a version of the same task built only to remove the pattern it was actually using.

Bender and Koller's explanation does not call BERT unimpressive. It asks what a system trained only on word sequences could learn even in principle: their example is a hyper-intelligent octopus that learns to hold up a fluent conversation purely from the statistical shape of undersea-cable traffic, without ever seeing what the words refer to.⁶ BERT's masked-word objective puts it in exactly that position.

The recipe had more room in it than anyone tested

BERT's eleven firsts held for less than a year. In 2019, a team led by Yinhan Liu at Facebook AI ran BERT's exact architecture again, changing only the training: ten times the text (160 gigabytes, not 16), longer runs, bigger batches, and no next-sentence prediction.⁹ They called the result RoBERTa and stated their finding plainly: "BERT was significantly undertrained."⁹ On RACE, BERT_LARGE had scored 72.0, ahead of everything published before it. The same architecture, with more training data, reached 83.2 under RoBERTa: eleven points higher, on a test BERT itself had just set the standard for.⁹ None of BERT's architecture changed; the scores people had treated as a ceiling were a floor the field mistook for an answer.

The split that mattered afterward ran between the two halves of the 2017 Transformer architecture. BERT is an encoder, reading a full sequence at once and returning a representation suited to comparing, ranking, and classifying text. GPT kept the decoder half, built to write one word at a time, and that lineage became what the public meant by "AI" in the years after. Google confirmed one concrete use: in October 2019 it announced BERT-based ranking would affect one in ten English search queries, reported independently at the time.¹⁰ That is a dated fact about one company's search engine in 2019, not a count of today's software; no public count exists. What holds is the shape of the job: ranking documents against a query, or turning a sentence into a fixed vector to search, is a comparison problem, and comparison is what an encoder is built for.