Give GPT-4 the same options in a new order and it changes its answer

One reorder, a different answer

Pezeshkpour and Hruschka gave GPT-4 a commonsense question, then posed the same question with its answer choices rearranged. The model's pick changed from one option to another.¹ Nothing in the question or the options was different. Only their order was, and that was enough to change the answer.

The effect is not a single odd case. Across five benchmarks, zero-shot, Pezeshkpour and Hruschka measured the gap between the best ordering and the worst at roughly 13 to 75 accuracy points. Even GPT-4, above 90% on some of those tasks, still moved by 13.1.¹ Zheng and colleagues came at it from the other side. They moved each question's correct answer to a fixed slot and watched the score follow. On zero-shot MMLU, llama-30B answered 53.1% right on its default ordering, 68.2% when the answer was always at A, and 41.2% when it was always at D.² A third group reproduced the behavior independently and analyzed it the same two ways, by option order and by the answer tokens.³

So the behavior is settled and measured, across several models and groups. What causes it is the harder question, and there the people who measured it do not agree. Three parts of the system each have a hand in the answer: where an option sits in the sequence, the pull the letters exert on their own, and how the answer is read off the model.

Where a choice sits in the list

The question and its options reach the model as one sequence of tokens, read from front to back, with no divider between the question and the choices. A transformer has no built-in sense of where a token sits. The position of each one is supplied to it, and it does not weigh every position the same. Liu and colleagues showed this cleanly on a different task, finding a needed fact buried somewhere in a long input. Accuracy was highest when the fact sat at the very start or the very end of the input and sagged in the middle. They named the two edges primacy and recency, after the U-shaped curve the scores traced.⁴ In multi-document question answering, moving the answer-bearing document from the best position to the worst cost gpt-3.5-turbo more than 20 points.⁴

Pezeshkpour and Hruschka argue that this same weighting of position is what drives the option-order effect. Their Conjecture 4.1 puts it as two forces acting together: the model's uncertainty about which of its top choices is correct, and a positional bias that leads it to favor options by where they appear.¹ Their checks point the same way. On the questions most sensitive to reordering, the model agreed that more than one choice could be highly probable over 94% of the time. That is the uncertainty the conjecture needs. Freezing only its top two or three options in place, and shuffling the rest, barely changed accuracy, which reads as the model responding to position rather than content. And placing the two strongest options first and last amplified the bias, while setting them next to each other reduced it.¹

Liu and colleagues own the primacy and recency mechanism, but they measured it on long-context retrieval, not on a short list of answer choices. Pezeshkpour and Hruschka carry it over to options as a conjecture, not a settled decomposition. The position effect is real. That it is the main cause of the option-order swing is a claim, and it has a rival.

The letters were tokens before they were labels

A, B, C and D are not neutral slots. They are tokens the model saw countless times in training, and it carries a prior over them before it reads a single option. Zhao and colleagues found the general form of this on a different task, few-shot classification. They called it common-token bias: the model's pull toward tokens frequent in its pretraining data, the way an unsure model reaches for "United States" over "Saint Lucia."⁵ The option letters are among those frequent tokens, and the model leans on them for reasons that owe nothing to what they label.

Zheng and colleagues carry the idea to the A/B/C/D case and call it token bias. Their evidence is a pair of ablations. Shuffling the option contents while leaving the letters in place did not obviously change the bias, which argues against position as the main driver. Removing the letters entirely cut their bias metric by 6.4 on average, at a cost of 2.1 accuracy points, which argues for the letters.² That metric is the spread across how often the model chooses each letter. With no bias it would land on each about equally often. A wide spread means it is choosing a letter, not an answer. Estimating the letter prior from a small sample and subtracting it back removed much of the spread, and even nudged accuracy up. That is what you would expect if the letters were part of the problem.²

Zheng and colleagues state plainly that selection bias "arises less from LLMs' position bias," and name the token prior as the more salient cause. They cite Pezeshkpour and Hruschka's account as the common view they argue against.²

What counts as the model's answer

There is a step before any of this that the model never sees: how the evaluator decides to read its answer. Robinson and colleagues lay out two ways. The cloze way shows the question alone and scores the text of each answer against it. The natural way shows the lettered options and asks the model to emit a symbol, the letter "A." That second way needs the model to bind each option's content to its letter, a skill they call multiple-choice symbol binding that "varies greatly by model."⁶ The letter-scoring regime, where a prior on A/B/C/D can bite, is one way to pose a multiple-choice question, not the only one.

Zheng and colleagues measured the token bias through the first token it would emit, its probability on "A" against "B" against "C" against "D." Two other groups checked that probe against the answer the model actually writes out, and found they often disagree. Wang and colleagues report the first-token pick and the written answer clashing 56.8% of the time on Gemma-7b-Instruct and 51.4% on Llama2-7b-Chat, though only 10.2% on Mistral-7b-Instruct. Where the clash runs above half, the written answer holds up better under reordering than even the debiased first-token score.⁷ A second group put the overall mismatch above 60%, worst on models most heavily fine-tuned on conversation and safety.⁸ So some of the measured token bias belongs to the ruler, not the model: read the words the model writes instead of its first-token odds, and part of the effect goes away.

The Hugging Face team scored the same MMLU questions three ways and got three different numbers, far enough apart to reorder the leaderboard.

None of this overturns Zheng and colleagues' ablations, which stay valid inside their one scoring method. It only qualifies how much of the swing is the model choosing a letter and how much is the ruler used to read it. That a scoring choice alone can swing a score is its own lesson. The choice sits upstream of everything else, a third thing that can move the number.