A transformer never reads the letters in strawberry

Why the model cannot count the r's

Ask a language model how many r's are in strawberry and it may miss, and the reason is not spelling. Run the word through the tokenizer that GPT-3.5 and GPT-4 actually use and it comes back as three pieces, str, aw, berry, none of them a single letter.¹ The model never receives the characters. It receives three integers, one per piece, looked up in a fixed table. The r's are buried inside str and berry, so a question that looks like spelling is a question about how the word was cut.

That cut is the first step of the pipeline, before any of the machinery the rest of this course covers. Every string of text is turned into a sequence of integer tokens, and the model reads only those tokens. The step has a name, tokenization, and one dominant method, byte-pair encoding.

The count matters beyond puzzles. A model's context window is measured in tokens, not words or characters: GPT-2 raised its window from 512 to 1024 tokens, and the same unit meters usage today.² A tokenizer that splits a text into more pieces makes it cost more and run out of room sooner, for the same sentence.

Counting pairs until a suffix appears

Start with four words and how often each occurs in some collection of text. Write every word as its separate characters followed by a marker for the end of the word, printed here as </w>, which keeps the tokenizer from merging the end of one word into the start of the next.³ The count beside each word weights every pair inside it.

Now count every adjacent pair, adding the word's frequency each time the pair appears. Pairs that would straddle two words are never counted.³ The pair (e, s) sits inside newest (6) and widest (3), so it scores 9. The full tally:

Three pairs tie at the top: (e, s), (s, t), and (t, </w>), all 9. The tie is not an accident of the numbers chosen. Those three pairs occur only inside the ending e s t </w>, which both newest and widest carry and nothing else does, so each pair always equals the frequency of newest plus widest, whatever frequencies are used. The honest reading is that the procedure has found the -est suffix, and the next three steps collapse it into one symbol. Merge the top pair, recount, merge again; when the top scorers all live in the same ending, they fall in sequence.

This is the whole of byte-pair encoding. Count adjacent pairs, replace the most frequent pair everywhere with a new combined symbol, and repeat until the vocabulary reaches a target size. The number of merges is the method's one setting.³ The idea is older than language models. Philip Gage described it in 1994 as a way to compress a file, replacing the commonest pair of bytes with a byte the file never used.⁴ On the string aaabdaaabac the commonest pair aa is swapped for one new symbol, then ab, and the string shrinks at each pass.⁵ Applied to characters instead of bytes, the same counting builds a tokenizer.

Splitting a word the table never saw

Breaking the tie by taking the first pair to appear in a left-to-right scan, the merges run in this order. Each row adds one symbol to the vocabulary.

Five learned symbols now sit above the plain characters: es, est, est</w>, lo, and low. This ordered list is the merge table, and applying it is a fixed procedure with no choices left in it. Take a word the four training words never contained, lowest, write it out as l o w e s t </w>, and run the five merges in order. Step 1 joins e s into es. Step 2 gives est. Step 3 gives est</w>. Step 4 joins l o. Step 5 gives low. What remains is low and est</w>: lowest becomes [low][est</w>], two tokens, a stem and a suffix, neither a whole word nor a single letter.³

The split is deterministic. The same word and the same table always give the same tokens, in the same order, because the merges are applied greedily by rank with nothing to decide. That determinism is also the method's known limit: it commits to one segmentation and cannot weigh alternatives, the gap the probabilistic unigram tokenizer was later built to fill.⁶ A word with a stray letter shows the greed plainly. slowest becomes [s][low][est</w>]: the leading s never earned a merge, so it is stranded as its own token.

This toy is not a caricature of the real thing. The production tokenizer, trained on far more text, cuts newest into exactly [new][est] and independently found the same -est suffix the four words discovered.¹ Common whole words such as lower and lowest are frequent enough to have earned a single merged token each, so they arrive whole; the rarer a string, the more it fragments toward its pieces.

What the corpus fixed into the table

Two different things have been quietly separated, and holding them apart is the point. Applying the table is a deterministic function of the string. Building the table was a one-time count over a training corpus, and once built, the table is frozen. The corpus is what decided that est and low deserve their own symbols and a rarer ending does not. Change the corpus and a different set of merges is learned; the tokenizer encodes the statistics of the text it was built on.⁶

Modern tokenizers count bytes rather than characters. A tokenizer built over the roughly 130,000 Unicode code points would start from a base of that many symbols before learning a single merge; working over raw bytes needs a base of only 256, one per possible byte value.² Because all 256 bytes exist as tokens, every possible string is representable and nothing is ever unencodable. GPT-2 built a vocabulary of 50,257 this way, the 256 bytes plus 50,000 learned merges plus one end marker.² The tokenizer is a full system in its own right, with its own training run and a rule that lets it rebuild the exact original text from the tokens, whitespace included.⁷

Where a frozen table splits oddly

A fixed table cut for one corpus makes choices that look strange elsewhere, and each is checkable. Whitespace decides edges: the model treats a leading space as part of the following token, so strawberry alone is three tokens, while  strawberry with a leading space is one, the same letters cut a different way.¹ The space was folded into the merges deliberately, because most words in running text follow a space, and it keeps common words from fragmenting.²

Digits show the same mechanics. The string 1234567890 splits into [123][456][789][0], four tokens chunked in groups that ignore place value, and a comma re-cuts the groups: 1,234,567 becomes five tokens, [1][,][234][,][567].¹ A model doing arithmetic is not reading digits in columns; it is reading arbitrary numeric chunks, which is why careful column arithmetic is hard for it. Rarity fragments too. A long, uncommon word such as supercalifragilistic breaks into six or seven tokens,¹ since no part of it was frequent enough to earn a large merged symbol, the way common subwords do.⁸ Rarity, not alphabet, is what drives this: a common Japanese greeting encodes to a single token, because it appeared often enough in the corpus, while a genuinely rare string of any script falls back toward its bytes.¹

The layer that runs before the model

Tokenization is the representation the whole pipeline stands on, and it is settled before the model does anything. Embeddings, attention, and training all operate on the token sequence this step produces; they never see the letters, only the integers.² A short word and a small table are enough to see the entire mechanism: count pairs, merge the most frequent, repeat, then apply the frozen list to any word, seen or unseen, and read off the pieces. The model inherits those pieces as its atoms.

So a large share of behavior that looks like reasoning is really the shape of the split. The r's in strawberry are unspellable to the model because they are welded inside str and berry. Arithmetic wobbles because digits arrive in chunks that ignore place value. Rare languages cost more and read worse because their text was scarce when the table was counted, so it shatters into bytes.⁶ None of that is the model failing to think. It is a table, learned once off some corpus, deciding what the model is allowed to see.