A model can spell strawberry and still miscount its letters

The miss anyone can check in two seconds

The word strawberry has three r's: s-t-r-a-w-b-e-r-r-y. Anyone can verify it in the time it takes to read the word. A chatbot often cannot. Asked how many r's the word contains, GPT-4o and Claude both answered two, a miss TechCrunch reported in August 2024.¹ The letters are all right on the screen, ten of them, and the same system that had just written a working paragraph of code could not add up one of them.

The reason is not that the model is sloppy, and it is not that counting to three is hard. It is that by the time the word reaches the part of the model that answers, it is no longer a string of letters at all. Something rewrote it first.

What arrives is str, aw, berry

Before a language model reads a single word, a separate program called the tokenizer cuts the text into tokens, the units the model actually processes. A token is usually a whole word or a fragment of one, drawn from a fixed list the model was built with. Run strawberry through the tokenizer that GPT-4 and GPT-3.5 use, and it comes out in three tokens: str, aw, and berry.²

Read the last column and the counting problem is plain. To answer three, the model has to know that str holds one r, that aw holds none, and that berry holds two, and then add them. But a token is a single entry in that fixed list. It does not arrive with its letters attached, the way the middle column here spells them out for a human reader. The model gets 496, 675, 15717, three arbitrary numbers, and whatever it has learned to associate with each.

It is usually worse than three pieces. In running text the word almost always has a space in front of it, and every one of these tokenizers treats a space-led word as a candidate to keep whole. With the leading space, strawberry becomes a single token in each of them.² One number now stands in for the entire ten-letter word, and the letters are packed even further out of reach.

How byte-pair encoding builds a token

The list the tokenizer draws from is not written by hand. It is built by an algorithm called byte-pair encoding, which grows a vocabulary by merging. It scans a large body of text, finds the pair of adjacent symbols that occurs most often, replaces every instance of that pair with one new symbol, and repeats until it has as many symbols as it wants.³ Philip Gage published the method in 1994, and not for reading text: it was a way to compress data, packing frequent byte pairs into fewer bytes.³ Sennrich, Haddow and Birch borrowed it for translation in 2016, using the same frequency merges to split rare words into reusable subword pieces, and GPT-2 later named their paper as its own reference.⁴

Production tokenizers run this over raw bytes. GPT-2's starts from a base of 256 possible byte values and merges upward to a full vocabulary of 50,257 tokens.⁵ Because the merges follow frequency, common sequences like berry or ing become single tokens while rarer strings stay in fragments. A token exists because its letters turned up together often in the training text, never because those letters line up into something a speller would recognize. The unit the model computes over was chosen for how often it appears, not for what it is made of.

This is not one vendor's quirk. SentencePiece, the other tokenizer family many open models use, builds a subword vocabulary the same way,⁶ and subword tokenization that begins from byte-pair encoding is the dominant approach across modern language models.⁷ What differs between tokenizers is mostly which strings become whole tokens, which is also why the same sentence can cost very different token counts in different languages.

The model can spell what it cannot count

The tokenizer hiding the letters is the settled part of the explanation. What happens after is not, and this is where the tidy one-liner goes wrong. If the letters were simply gone, a model would also be unable to spell its words out. It can. A benchmark called CUTE tested this kind of character work across models from 7 billion to 132 billion parameters, and found that the same systems that fail to insert, delete or swap a letter can usually spell a word out correctly, one letter at a time.⁸

Two further findings from the same work rule out the easy reading. The first: the failure does not come from the word arriving as one opaque token. When the researchers threw out every example where the tokenizer had split a word into pieces, accuracy barely moved, by at most about 3.5 percent and usually within half a percent.⁸ A word delivered in several tokens is nearly as hard as one delivered whole. The second cuts the other way. A separate study trained a small probe to read a token's internal representation and predict which characters it contains, and the probe worked: the character identities are encoded in the token's vector, present but indirect.⁹ The letters are not erased. They are recoverable in principle, and the model does not reliably reach for them when it counts.

The coarseness of the tokens is still a genuine driver, not a coincidence. When CUTE fed models random strings that happen to break into roughly one character per token, close to the finest split possible, performance on the character tasks held up or improved.⁸ Finer pieces, easier counting. So the honest shape of the answer has two parts. Settled: byte-pair encoding tokens are frequency-merged word-pieces, not letters, so a token hands the model no direct index of the characters inside it. Open, and open even to the people who build these systems: how much of that buried character information a model recovers, and why it can spell a word yet miscount its letters.

The fix has to reach the letters themselves

Once the failure is drawn this way, each route out of it addresses a different part of the shape. The cheapest works around the model. Since spelling a token out is something models do well, a system that first writes s-t-r-a-w-b-e-r-r-y and then counts that plain string has a far easier task than counting the tokens directly.⁸ This is what a step-by-step answer, or a call to a small counting tool, buys. It converts a character question the tokens obscure into a spelling question they do not. That is a large part of why later and more deliberate models handle strawberry when a fast single-shot answer still stumbles.

The deeper fix changes the tokenizer. Token-free models like ByT5 drop the subword vocabulary and read raw bytes, roughly one per character, which removes the hidden-character problem at its source and does measurably better on tasks that turn on spelling.¹⁰ It is not free. Working a byte at a time makes the input sequence about five times longer and costs proportionally more compute,¹⁰ which is why most production models kept byte-pair encoding and patched around its blind spot rather than paying to remove it.

The same blind spot surfaces elsewhere, and it helps to keep the cases apart. When an image generator draws a garbled word on a sign, the cause is the same, a whole-word token with its spelling blurred, reaching a different step of the system. A model misadding two long numbers is a different failure entirely, one about predicting the next digit rather than about tokens hiding characters. The strawberry miss belongs only to the first family: any task that needs the model to operate on characters it never receives as characters.