An image generator never counts the fingers it draws

The assumption behind a six-fingered hand

An image generator asked to draw a person routinely gets the hands wrong. Often enough that it became the standard tell of a fake picture, one hand comes back with six fingers, or with two fingers fused into one, or with a thumb where no thumb belongs. The failure is common enough that researchers have built tools whose only job is to find these hands in generated images and redraw them.¹

The natural reaction is that the model should know better. A hand has five fingers, the model has seen millions of hands, so somewhere inside it there ought to be a fact, a hand has five fingers, that it keeps failing to apply. That assumption is worth saying out loud, because everything about how these pictures are made cuts against it. The model holds no such fact. Seeing why means following the finished picture back to the process that drew it, one step at a time.

Nothing in the process counts fingers

An image generator of this kind is a diffusion model. It begins with a field of random noise and removes a little of it at each step, nudging the pixels toward a picture that looks plausible under the prompt, until an image emerges from what started as static. The earlier lesson on image generation walks through that denoising step by step, so take it as given here.

Training rewards one thing: reconstructing a picture from a noised version of itself.² The model is shown images with varying amounts of noise added and learns to predict the noise, so that at generation time it can subtract its way from static back to a clean image. Stable Diffusion, the system most of the hand studies test against, runs that same denoising in a compressed space and steers it with the text prompt.³ The objective it is trained on has terms for pixels and noise. It has no term for a hand, or for a finger, or for the number five. No step in the training target counts parts or checks anatomy.

This is a claim about the objective as written, not an experiment anyone ran. A large model trained on enough pictures can still pick up an implicit, statistical sense of what hands tend to look like, and that rough sense is exactly what the later fixes try to sharpen. What is absent is any explicit step that verifies a hand has five fingers. The model settles on a patch that looks locally like real skin and an image that hangs together as a whole, and a plausible-looking hand with six fingers satisfies both. As the AI-and-arts researcher Amelia Winger-Bearskin put it, the system is only ever "looking at how hands are represented in the images that it has been trained on," not learning what a hand is.⁴

Few pixels, many joints

If no image gets its fingers counted, why do hands fail so much more often than faces or chairs? There are two reasons, and they are different reasons, though they are easy to run together.

The first is what hands look like in the training pictures. According to Stability AI, the maker of Stable Diffusion, human images in these datasets "display hands less visibly than they do faces," and hands "tend to be much smaller in the source images, as they are relatively rarely visible in large form."⁴ The model sees far fewer clear, large examples of hands than of faces, so it has less to learn from. This is the explanation that reached general readers, and a reference-work account of why AI art "screws up" hands gives the same training-data reason.⁵ It is true, and it is only the first of the two reasons.

The second reason holds even where hands are plentiful. A hand is a small, intricate object that can take an enormous range of shapes. HanDiffuser's authors put it that hands "often take up a small part of the image, but are highly articulate," with fingers that "bend to various degrees relatively independently."⁶ A kinematic model of a single hand counts sixteen joints and twenty-seven degrees of freedom. The figure comes from ElKoura and Singh's 2003 study of hand animation, and is the count HandRefiner cites when it explains the difficulty.⁷ Add the angle a hand is seen from, what it is holding, and which fingers hide behind others, and the same five-fingered object turns up across the training set as a near-endless variety of pixel arrangements. FoundHand's authors give both halves in one line: hands "are not well-sampled in these datasets" because "they either occupy very few pixels, or complex articulations are not captured."⁸

So the model's internal sense of a hand is spread thin across all those arrangements. When it denoises a new one, it lands on something hand-shaped and plausible, and with the target that diffuse, plausible often means the wrong number of fingers. How often is not something anyone in this literature has measured cleanly. When HanDiffuser's researchers asked people to rate Stable Diffusion's hands, only 27 percent were rated good or better for plausibility.⁶ That is a judgment by human raters on one model, not a count of missing fingers, but it puts a number on how routine the failure is.

The gap keeps closing

Two things are worth holding apart here: how well the cause is understood, and whether the failure is fixed. They have different answers.

The cause is on firm ground. Three research groups, working independently, arrive at the same two reasons: hands are small in the frame and hugely variable in pose.⁸ The model's own maker gives the matching account of its training data.⁴ That much is settled.

The frequency is not settled, and neither is whether to call the problem solved. No study in this set measures how often generated hands come out wrong. The failure is attested by how obvious it is and by the fact that repairing it became its own research topic. And it is receding, so it is not a permanent limit of the method. The fixes run in two directions. One adds explicit structure. HandRefiner redraws the hand region after the fact, using a three-dimensional hand mesh as a guide.⁷ HanDiffuser feeds the generator explicit hand-pose parameters, so the model no longer has to infer a plausible hand from nothing.⁶ The other adds data: FoundHand trains on a dedicated set of ten million hand images labeled with keypoints, attacking the scarcity that started the trouble.⁸ Each reports real gains. That the failure gives way to more data and better guidance is the strongest sign that it was always a matter of coverage, not something the method cannot do.