Rename the people in a word problem and the model does worse

A word problem the model solves until you change it

Give a current language model a grade-school word problem and it usually gets the answer right. The problems in GSM8K, the standard set of 8,500 of them, are the kind a careful ten-year-old can do: a few sentences of setup, a short chain of arithmetic, one number at the end.¹ This is not the arithmetic itself going wrong. An earlier lesson covered that, the model fumbling a large multiplication it has rarely seen. Here the arithmetic is small and the model can do it. What changes is only the wording around it.

The cleanest version of the change is to add a sentence. Shi and colleagues at Google took 100 GSM8K problems a model already solved and added one irrelevant sentence to each, the kind of aside a human reader skips without noticing. On the strongest setup they tested, per-problem accuracy fell from 95.0% to 72.4%.² Same problems, same numbers to add up. The only new thing in the prompt was a sentence that changed no quantity the question asked about, and the score dropped more than twenty points.

The rest of this lesson works backward from that drop to what produces it. The question it answers is why a change a person would ignore moves a machine that looks like it is reasoning.

A single score hides a spread of outcomes

How big is the effect? The honest answer needs two numbers rather than one. Scored problem by problem, the model on the added-sentence set was still right 72.4% of the time. But ask the stricter question, of the problems the model could solve, how many did it solve under every kind of irrelevant sentence, and the figure is 6.0%. No more than 18% of them survived all of them.² A model that is usually right can be almost never robust, and which number you quote decides which story you tell.

The second measurement makes the spread visible. Mirzadeh and colleagues at Apple regenerated GSM8K problems with different names and different numbers, left the structure untouched, and ran about 25 models over many such versions.³ A model does not have one GSM8K score. It has a distribution. The gap between Gemma2-9B's worst and best version was more than 12 points, and Phi-3.5-mini's about 15.³ Changing only the names moved the score a little; changing only the numbers moved it more. The plain GSM8K figure most papers report sits near the middle of the changed-names spread but not the changed-numbers one, so for the harder change the widely-cited number is closer to a best case than a typical one.

What the model is holding when it reads the problem

To see why a skipped sentence changes the answer, look at what the model actually has in front of it. The problem does not arrive as a problem. It arrives as a sequence of tokens, the pieces a model's text is cut into before anything else happens, and the model produces its answer by predicting the next token from all the earlier ones at once, through the attention mechanism this course has already taken apart.

There is no second copy of the problem anywhere in the model. No list of the quantities, no note of which sentence states the question and which is background, no separate workspace where the arithmetic is set up and solved. Everything the model has to go on is those tokens and what its weights do with them. Work on how transformers handle multi-step problems finds them matching a question to computation patterns seen often in training, rather than running a general procedure over it.⁴ Accuracy falls as a problem needs a path that was rare in the training data. A person reading the word problem builds a small picture of the situation and reasons over that, dropping the exact wording once the meaning is out. The model never leaves the wording. The wording is all it has.

Nothing tells the model which words to ignore

Because the wording is all it has, every token in the prompt is a candidate influence on the answer, and there is no separate step that sorts the tokens the question needs from the ones that are only scenery. Change the names and the match to any particular remembered problem gets a little weaker. Change the numbers and it gets weaker still, which is why numbers move the score more than names do: the digits carry more of the work of picking out a familiar pattern.³ A related lesson showed the same underlying fact from the formatting side, a colon in place of a space swinging the score. Here the change is not formatting but content, a different name or an added fact, and the logic is untouched.

Add a sentence and the new tokens do not sit quietly to one side. If the sentence is on-topic, mentioning the same objects the problem is about, its tokens look like the kind that usually carry a quantity the answer needs, and the model folds them in. Mirzadeh's team built exactly this test, called GSM-NoOp, adding one true clause to each problem that changes nothing the question asks.³

The effect reaches past math. Nezhurina and colleagues posed a trivial puzzle, Alice has some brothers and some sisters, so how many sisters does one of her brothers have, and varied only the two counts.⁵ For a single model the rate of correct answers could swing from near 1 down to near 0 depending on which small numbers were used, though every version was the same one-step problem. In mid-2024 the strongest models they tested, GPT-4o and Claude 3 Opus among them, answered it correctly only a little more than a third of the time. Wu and colleagues found the same shape across eleven different tasks. Give a task its familiar default form and models do well. Keep the task identical but shift a default assumption, such as doing the arithmetic in a base other than ten, and accuracy degrades substantially and consistently.⁶

Where this leaves the reasoning question

Put the results together and one thing is settled. A model's answer to a word problem is conditioned on the whole surface of the prompt, and the model has no reliable filter separating the tokens the question needs from the ones it does not. Rename it, renumber it, or pad it with a true aside, and the output can move. That much is not in dispute among the people who build these systems. The mechanism-level work explains why: a model tends to match a problem to patterns it saw often in training rather than run a general procedure. A change that makes the surface less familiar can degrade the result.⁴

What that means for the word reasoning is where it gets contested, and the contest is worth following because the strong reading is easy to overstate. Mirzadeh's team reported that GSM-NoOp's single added clause dropped accuracy by up to 65% across state-of-the-art models, and concluded they found no evidence of formal reasoning.³ That figure became the headline. Coverage carried it as proof that the models cannot reason and cannot be a foundation for reliable agents.⁷

Two things complicate the strong reading. The first is in the paper's own numbers: the drop is uneven, and it is smallest for the models trained hardest to reason step by step.

o1-preview, tuned to work step by step, lost about 15 points where the general-purpose GPT-4o lost more than 30 and a small open model lost most of its accuracy.³ Whatever the added clause tests, more reasoning-oriented training buys real resistance to it.

The second complication is that the added clauses may not be as irrelevant as the test assumed. A 2026 re-analysis had auditors classify all 945 GSM-NoOp clauses and judged only 117 of them, about one in eight, as unambiguously beside the point; the rest could plausibly bear on the answer.⁸ Re-running on the full set reproduced the original steep drop, but on the 117 clearly irrelevant clauses the accuracy loss for current frontier models was between zero and two points, indistinguishable from no effect at all.⁸ If that holds, much of what GSM-NoOp measured was how models handle genuinely ambiguous wording, not whether they can ignore a truly irrelevant fact.

The settled behavior stands on the cleaner changes, the added-sentence problems, the pure name and number swaps, the Alice puzzle, none of which turn on whether a clause was secretly relevant. The single most-quoted number behind the strong claim, the 65% drop, rests on the one test where that objection bites. A reader can hold both halves at once. Surface form provably moves these models, and the leap from there to they cannot reason at all runs ahead of what the clean tests have shown.