A generated image gets its letters wrong before the picture is ever drawn

The word on the sign is almost right

You ask an image generator for a storefront with a sign, and it gives you one. The sign has letters on it. They lean the right way, they sit in a plausible font, and they spell nothing. One reads CAFEE, another BQKERY, a third is a few confident strokes that resolve into no word at all. Every generator does this, and the people who build them say so plainly: Stable Diffusion's own model card lists, among its known limits, that the model cannot render legible text.¹

The easy explanation is that the model is simply not good at spelling, the way it is not good at hands. That explanation is not wrong so much as it stops one step too early. It treats the garbled word as a drawing problem, something a steadier hand would fix. The letters come out wrong for a reason that has little to do with drawing, and you can find it by walking backward from the sign to the moment the word entered the system.

The part that draws never sees a letter

Start with the part that makes the pixels. In a diffusion model, an image begins as a field of random static, and the model removes that noise a little at a time until a picture is left, the loop an earlier lesson walked through in full. Nothing in that loop knows the alphabet. To this part of the system, the strokes of a letter are texture, the same kind of thing as the grain of wood or the edge of a cloud. It has seen many images with writing in them and learned what letter-shaped texture tends to look like. That is why the output reads as writing at a glance and falls apart up close.

This part does not even receive your word as letters. The prompt reaches it through a separate step. In latent diffusion, the text is first turned into a code by a text encoder. That code steers the denoising through a mechanism called cross-attention: each patch of the forming image reads the encoded prompt to decide what it should become.² The drawing step is following instructions it did not write. If the instruction for the sign does not carry the word's spelling, the part holding the brush has nothing to spell it from. The letters are not decided here. So go back one step, to the thing that wrote the code.

The encoder turns your word into a chunk

The code comes from a text encoder, a network built to turn a run of text into a sequence of vectors that stand for its meaning. Turning a word into a vector is a move this course has met before. The catch is the very first thing the encoder does to your prompt, before any meaning is worked out: it breaks the text into tokens. A token is a chunk of characters treated as one unit and looked up as a single number in a fixed table. A common word like bakery is often a single token. A longer one like elephants splits into two, elephant and s. Either way, what moves forward is the number for the chunk, and the individual letters inside a chunk are no longer represented on their own.³ This is the same reason a model can recognize the word strawberry and still miscount its r's.

So by the time the prompt reaches the part that draws, the word has become a token or two standing for the whole chunk, and any spelling has to be recovered from that. The builders of DALL·E 3 say this about their own model, in as many words.

Stable Diffusion reaches the drawing step through a different encoder, the text side of a model called CLIP. The shape of the problem is the same: it reads a lower-cased vocabulary of 49,152 subword chunks, not one of them a single letter.⁵

It is tempting to say the letters are simply gone at this point. That goes too far. A large enough model can rebuild spelling from those chunks by brute statistics. Asked to spell common English words, a 540-billion-parameter language model gets them right more than 99% of the time. A frozen mid-size encoder of the kind these generators actually use manages about 66%.³ The letters are not erased. They are compressed into a form the encoder can recover only at a size most image generators will never pay for. That is the bottom of the chain. Below the point where the word became a chunk, nothing changes the answer, because the part that draws can only redraw what the chunk carried up to it. That the failure begins here, at the encoder, is settled: it is the account the people who build these systems give of their own results.

Fixing the letters means fixing the encoder

If the letters go wrong at the encoder, that is also where they get fixed, and two different changes have been tried. The first is size. Google's Imagen used a frozen text encoder called T5-XXL, with 4.6 billion parameters, far larger than CLIP's. Its builders reported a result that points straight at the encoder. Making it bigger improved how faithfully the image matched the prompt more than making the image-drawing model bigger did. Imagen's rendered text also came out clearer than DALL·E 2's.⁶ A second released model, DeepFloyd IF, is built on the same large frozen T5 encoder,⁷ which the widely used diffusers software library describes the same way from the outside.⁸ But T5 still reads subword chunks. So this is the brute-scale lever from a moment ago, pushed further: a larger character-blind encoder that recovers a little more of the spelling. Size helped first.

The second change is type, and it helps more for less. A team at Google ran the clean version of the experiment. Hold the image model, the training data, and the training budget fixed. Then swap only the encoder, from character-blind T5 to a character-aware one called ByT5 that reads the text as raw bytes, which is to say as its actual characters. Rendered spelling rose by more than 25 points on common words and more than 30 on rare ones.³

Two details show that the character-reading, not the size, is doing the work. The winning encoder kept its lead even against a character-blind rival trained more than six times as long, and a version of it 43% smaller than T5-XXL still won.³ The gain comes from reading letters, not from a bigger model.

The fix is not free. An encoder that reads only characters got worse at prompts with no text in them, so the practical recipe is to bolt a small character-aware encoder onto the ordinary one, a 4.8% increase in size, rather than replace it.³ Character-awareness buys spelling. It is not a general upgrade to everything the encoder does.

So look again at the sign

Now the garbled sign comes apart into two failures that used to look like one. The first is which letters. A character-blind encoder is guessing the spelling from a word's meaning, and its mistakes give that away. It swaps in a homophone, adds a stray letter, and in one test it "corrected" irregular verbs so that fought came out as fighted in about one try in nine.³ Those are the errors of something that knows what a word means and not how it is spelled, and a character-aware encoder mostly does not make them. Whether the model knows the letters is the encoder's business, and on that the story is settled.

The second failure is shaping and placing the letters once they are known. Even with a character-aware encoder, generated text still drops a letter, merges two, repeats one, or bends a stroke into a shape that is no letter at all. These slips turn up in every model tested, the character-aware ones included, and the researchers put them outside the encoder, in the image model that lays down pixels.³ This is the open part of the chain. No one has a tidy account of why the drawing step mangles a glyph it was told correctly, and no published figure says how long a word the best current systems can spell before they start to fail.

So the next time a generated storefront says BQKERY, you can read the failure off the sign. A word bent into almost-letters is the drawing step struggling to shape what it was handed. A confident, wrong, real-looking word is the older failure, the encoder that read the prompt in whole-word chunks and never had the spelling to pass on. Most of what improved these systems in the last few years fixed the second kind. The first kind belongs to the drawing step, and it is still open.