Answer order alone swung a language model's verdict by 80 points

One model grades another model's homework

"Judged by GPT-4" means a language model read the same answers a person would have and returned a score or a pick, instead of a person doing it. That method, LLM-as-a-judge, comes from a 2023 paper by Lianmin Zheng and coauthors, who built a 160-question test called MT-Bench and set a judge model loose on it three ways.¹ Single-answer grading hands the judge one response and a rubric, and it scores that response alone, the way a single grader scores an essay. Pairwise comparison hands it two responses and asks which is better, or whether they tie. Reference-guided grading adds the correct answer to check against, which helps on questions that have one, math foremost, and removes one thing the other two modes leave the judge to guess at.¹

Zheng and coauthors checked whether any of this was trustworthy by comparing the judge's calls to a panel of human raters scoring the same questions. The judge, GPT-4, agreed with the humans more than 80 percent of the time, about the rate at which the humans agreed with each other.¹ The paper's other reference point is Chatbot Arena, the crowd-sourced site where real people cast the same kind of vote instead of a model.¹ This desk has already measured that platform on its own terms: the same task, judged by a different kind of reader.

An 80 percent match with humans is real, and it is also the best case: the same paper that reports it is the one that found where the number breaks.

The answer that wins by going first, or second

Feed a judge the same two answers twice, swap only which one comes first, and watch the verdict change without a word of the answers changing. Wang and coauthors ran exactly that test using ChatGPT as the judge, comparing an answer from Vicuna-13B against an answer from ChatGPT's own family. Vicuna's win rate was 2.5 percent when ChatGPT saw its answer first, and 82.5 percent when it saw the identical answer second.²

That is position bias: the verdict depends on where in the prompt an answer sits, not only on what it says. Across a larger set of comparisons, the same paper measured how often a verdict flipped when the order flipped, a "conflict rate": GPT-4 flipped 46.3 percent of the time, ChatGPT flipped 82.5 percent of the time.² Zheng and coauthors ran a milder version of the same check and reported it as a consistency rate rather than a flip rate: GPT-4 kept the same call in 65 percent of reordered pairs, Claude-v1 in just 23.8 percent.¹ Different test, same shape: whichever judge is doing the scoring, part of its verdict is coming from where an answer happened to land.

Because the flaw is a positional artifact of the prompt, the fix lives in the prompt: run every comparison in both orders, and count a win only if it survives the swap. Wang and coauthors built exactly that check into a calibration procedure, pairing it with a rule that asks the judge for its reasoning before its score, and recovered 9.8 points of accuracy for GPT-4 and 14.3 points for ChatGPT.² A verdict that has not been run both ways has not paid that cost, and the gap it reports could still be sitting inside an 80-point swing.

The answer that wins by saying more

Zheng and coauthors also tried feeding the judge padding: an answer built from repetitive, list-heavy filler with nothing new in it, to see whether length alone would carry it. GPT-4 fell for it only 8.7 percent of the time. Claude-v1 and GPT-3.5 fell for it 91.3 percent of the time, essentially every time the trick was tried.¹

That is verbosity bias: treating length as a stand-in for quality. It shows up even without an adversarial trick. AlpacaEval's own team measured its default evaluator and found it picked the longer of two answers about 68 percent of the time, regardless of which one was actually better.³ A judge that is not being tricked at all can still be rewarding bulk by default.

The win probability is itself a function of length, not only of quality. Dubois and coauthors built length directly into the scoring: a regression estimates what the preference would have been if both answers were the same length, and reports that adjusted figure as length-controlled AlpacaEval.⁴ Running it moved the metric's correlation with Chatbot Arena's human votes from 0.94 to 0.98, and cut how far a provider could move its own win rate just by writing longer, from a 25-point swing down to 10.⁴ A win rate that has not been length-controlled is carrying an unknown share of its gap in extra words, and AlpacaEval's own baseline says that share can run as high as 18 points above a coin flip.³

A model favors what it recognizes as its own

Zheng and coauthors also checked whether a judge favors its own family's answers over a rival's on otherwise similar questions. GPT-4 rated its own answers the winner about 10 percentage points more often than an outside judge would; Claude-v1 favored its own family's answers by about 25 points. But GPT-3.5 showed no such preference at all. The authors call their own numbers unsettled: "due to limited data and small differences, our study cannot determine whether the models exhibit a self-enhancement bias."¹

Panickssery, Bowman, and Feng put a firmer basis under the same idea by tracing the mechanism directly. GPT-4, shown a set of answers that included its own, could pick out its own writing with no special training 73.5 percent of the time. When the researchers fine-tuned GPT-3.5 and Llama 2 on just 500 examples until each could recognize its own output more than 90 percent of the time, each model's preference for its own answers grew right along with its recognition accuracy.⁵ The better a model gets at telling its own writing apart from everyone else's, the more it prefers it.

The same pattern shows up outside chat answers. Liu and coauthors' G-Eval, a GPT-4-based judge built for grading text summaries, consistently rated machine-written summaries above human-written ones, even on pairs where separate human raters preferred the human version.⁶ Overall, G-Eval's scores tracked human judgment reasonably well on summarization, a Spearman correlation of 0.514, well above older metrics like ROUGE, and that aggregate number is exactly what hides the specific pairs where its verdict and the human's verdict diverged.⁶

Self-preference is not caused by anything in the prompt, so it cannot be fixed by reordering answers or trimming words. What the recognition mechanism suggests instead: a judge from a different lab than either system it is scoring has nothing of its own to recognize and prefer, so it is not exposed to the same shortcut.⁵

How much a verdict is worth before you trust it

None of these three biases is a rare failure mode, and a verdict from a single run carries a fourth problem on top of them: it may not even repeat itself. Yagubyan ran the identical pairwise comparison twice across 29 tasks and found the verdict flipped 13.6 percent of the time on average, with more than a quarter of the tasks flipping over 20 percent of the time. Two judges from the same company, OpenAI's GPT-4o-mini and GPT-4.1-mini, scoring the same pairs agreed only 76 percent of the time.⁷

That 76 percent is not the same kind of measurement as Zheng and coauthors' finding that GPT-4 agreed with human raters more than 80 percent of the time.¹ Yagubyan tested two judges from one company against each other, on a different set of tasks; Zheng and coauthors tested one judge against people, on MT-Bench and Chatbot Arena.⁷ Different pair, different tasks, different year: not a clean before-and-after. What the two figures still show side by side is a direction, agreement running weaker judge-to-judge than judge-to-human even before any comparison crosses providers. A survey of the field's own literature reaches a similar verdict from outside any single lab, concluding that keeping these judges reliable "remains a significant challenge that requires careful design and standardization."⁸

Put the three biases and the instability together and a usable check falls out. Before trusting a claim that model X beat model Y, judged by some model Z, ask three questions the report usually does not answer. Was the comparison run in both answer orders, with a win counted only if it held both times? If not, the gap could be sitting inside a swing as large as the 80 points position bias has been measured to move a verdict.² Was the win rate computed with length controlled? If not, as much as 18 points of it could be nothing but extra words.³ Is the judge from the same lab as either model? The size of that risk is less settled than the other two. The one paper that put a number on it measured a 10-to-25-point spread, then called its own measurement inconclusive. The underlying mechanism, a judge preferring writing it recognizes as its own, is not in doubt.¹ A reported gap that survives all three checks, or one large enough to swallow whichever of them was skipped, is telling you something about the models. One that does not is telling you about the judge.