A transformer can only compute more by writing more

Tell it to show its work and the score jumps

Ask a fluent model a multi-step arithmetic word problem and it often answers wrong. Add four words to the request, think step by step, and the same model on the same problem gets it right far more often. On the GSM8K grade-school math set, PaLM 540B solved 17.9% of problems when asked for the answer directly and 56.9% when the prompt first showed it a few worked solutions.¹ The newer reasoning models make the same move on their own. Trained by reinforcement learning to write out its reasoning first, DeepSeek-R1-Zero climbed from 15.6% to 71.0% on the 2024 AIME competition as it learned to spend more words thinking.²

The finding that intermediate steps raise accuracy is old news by now, named and measured in the chain-of-thought paper. What it left open is what the extra words actually buy.

A transformer spends the same effort on every token

The natural explanation is the human one. Told to slow down and reason it through, the model reasons harder, the way a person does when they stop guessing and concentrate. That is not what changes. A transformer runs the same fixed stack of layers for every token it produces, and the amount of computation it can do before it has to commit to the next word is set by that stack. It does not grow when the question gets harder.

This is not a hunch about how the models behave; it is a proved limit on what the architecture can do. Merrill, Sabharwal and Smith showed that a transformer producing a single output sits inside the complexity class TC0, the set of problems a circuit of fixed depth can solve.³ Fixed depth means a fixed number of computational steps, no matter how long or how hard the input. So one token's worth of computation is a hard ceiling. A problem whose solution needs a longer chain of dependent steps than that ceiling allows cannot be finished inside a single token, however large or fluent the model is.

Watch a product appear one line at a time

Multiplication shows the ceiling directly. Ask a model for 837 × 649 in one shot and it usually misses. Dziri and colleagues measured single-shot three-digit-by-three-digit multiplication at about 55% for GPT-3.5 and 59% for GPT-4, and the accuracy falls toward zero as the digits grow.⁴ The product is a short chain of dependent steps, and the chain is longer than one token can hold. Give the model room to write the partial products, and each line becomes a step it can finish and then read back.

The written lines are where the computation happens. Each is one bounded step of the kind a single token cannot hold, and once written, each becomes ordinary input the model reads back on its next pass, the same way any generated token does, at the per-token cost of reading the whole context again. Serial reasoning of length T needs on the order of T written tokens to carry it.

Complexity theory proves this is real power and not a trick of prompting. Feng and colleagues showed that a fixed-depth transformer forced to answer immediately cannot solve elementary arithmetic and equation-solving at any feasible size, while a constant-size transformer that is allowed to write a step-by-step derivation can.⁵ Merrill and Sabharwal turned the number of written steps into a dial. With a linear number of steps a transformer can recognize things it provably cannot answer in one pass. With a polynomial number it reaches every problem solvable in polynomial time.⁶ Li and colleagues state it plainest. Chain-of-thought gives the model the serial computation it structurally lacks, and T written steps buy the power of a circuit of size T.⁷ A transformer asked for an immediate answer, as one of these researchers put it, is "quite weak". The written steps are where the strength goes.⁸

Where writing the steps stops helping

A fair question at this point is whether the content of the steps matters or only their number. In theory, and in models built for it, the tokens can be pure computation regardless of what they say. Pfau, Merrill and Bowman trained transformers from scratch to solve hard algorithmic tasks using meaningless filler tokens, strings of dots standing in for a chain of thought. The benefit of the extra tokens can be computation independent of what they say.⁹ But that took dense, purpose-built supervision, and the trick does not appear in off-the-shelf models. Tested on a deployed model, the result flips: Lanham and colleagues replaced a real model's chain of thought with filler dots and saw no accuracy gain at all.¹⁰ The free computation is a fact about the architecture and about models drilled to use it. A working model appears to need steps that carry real content, not padding.

Nor does writing the steps help on every kind of problem. Sprague and colleagues, reviewing more than a hundred papers and running fresh evaluations, found the gains concentrated on math and symbolic tasks and small elsewhere. On the MMLU knowledge test, answering directly matched step-by-step accuracy except where the question involved an equals sign.¹¹ And even where it works the compute buys no free generalization: Dziri's scratchpad restored multiplication accuracy only within the range of problem sizes the model had seen, and failed on larger ones.⁴ The extra computation is real. It does not, on its own, make the model reliable outside the cases it was trained on.

The printed steps might not be the computation

Everything so far shows that written tokens can carry the computation. It does not show that the chain a given model prints is the computation that produced its answer. That question is open, and the evidence that it is open is direct. Turpin and colleagues planted a bias in the prompt, reordering the multiple-choice examples so the answer was always "(A)". The model's answer swung to follow the bias, while the reasoning it wrote never mentioned the bias and argued fluently for the new answer instead. Across thirteen tasks the planted cue cost as much as 36% accuracy, with the printed steps rationalizing the answer rather than reporting how it was reached.¹² Lanham and colleagues found that how much the printed chain actually drives the answer varies by task, and that on most tasks they studied it gets less faithful as models grow from 13 billion to 175 billion parameters.¹⁰

Those measurements were taken on older, smaller models and on prompts with a specific planted bias, not on the long reasoning traces that today's frontier models print. So the case that matters most is genuinely unsettled. Settling it would take a way to read the computation a network actually ran off its internal state and check the printed steps against it, and no such check exists yet. The theorists mark the same limit on their own results: a proof that a transformer with T steps can express a computation does not show that a trained model learns or uses that computation.⁸ Until that check exists, a printed chain of thought is evidence that the model spent computation, not proof of which computation it spent.