A faithful summary of a false passage passes the hallucination test

The percentage read as a lie detector

Hallucination percentages rarely reach the public as bare numbers. They arrive already read as scorecards for how often a model lies. Most of those percentages start at a leaderboard the search company Vectara has published since 2023, which scores each model and sorts the field from most faithful to least.¹ When a New York Times report first carried the launch numbers to a general audience, it said chatbots "invent information at least 3 percent of the time, and as high as 27 percent." OpenAI's model came in lowest and Google's PaLM-Chat highest.² A trade headline went further and ranked the models outright: "ChatGPT is Best, Palm-Chat Needs to Sober Up."³

Read as a ranking of honesty, the number is doing something it was never built for. Vectara's leaderboard measures one specific skill: how faithfully a model summarizes a document it is handed. It never asks the model a question and checks the answer against the world. The percentage quoted as "how often it invents things" is a summary-faithfulness score, and a summary-faithfulness score is not a truthfulness score.¹

From a pile of documents to one percentage

The rate is assembled the same way every time, in a fixed order.

1. Collect the documents

   Vectara keeps a private set of more than 7,700 articles, ranging from about 50 words to 24,000, drawn from news, medicine, law, science, and other fields. The set is kept private on purpose, so no model can train on the exact texts it will be scored on.¹

2. Hand each one to the model, under a strict rule

   Every model gets the same instruction: "Summarize using only the information in the given passage. Do not infer. Do not use your internal knowledge." The task is deliberately narrow, so that a correct answer lives entirely inside the text the model was given.¹

3. Score each summary with a second model

   A purpose-built classifier called HHEM reads the summary against its source passage and outputs a number from 0 to 1 for how well the passage supports the summary. HHEM is not a chatbot asked for an opinion; it is a small model fine-tuned for this one judgment.⁴

4. Subtract to get the rate

   The factual-consistency rate is the share of summaries HHEM judged supported. The hallucination rate is what is left over: 100 minus that consistency rate.¹

This design is careful where it counts. Because the correct summary sits entirely inside the supplied passage, a scorer can check the work automatically, with no human deciding what is true in the wider world. That is exactly why the task is summarizing and not open questions. GPT-4o's August 2024 build, for one, posts a 9.6 percent hallucination rate: HHEM judged 90.4 percent of its summaries fully supported by their sources.¹

Faithful to the passage, not to the world

Follow one document through the pipeline and the seam opens. Suppose a passage states, wrongly, that a drug was approved in 2019 when it was in fact approved in 2021. A model reads that passage and writes: "The drug was approved in 2019." The summary is accurate to its source in every word. HHEM checks the summary against that passage, finds the claim fully supported, and scores it as no hallucination.¹⁴ The model has just repeated a false fact and earned a clean mark for it.

A reader who hears "hallucination rate" expects the opposite, that a wrong date counts against the model. It does not. The rate rewards fidelity to the source, and a source can be wrong. A summary can be perfectly faithful to a passage that is entirely false.

The builders are not hiding this. It is stated on the same page as the number.¹ The gap opens later, when the percentage is lifted off that page and quoted as a measure of how often a model tells the truth.

The judge that misses most of them

Even inside its own task, the number is less exact than its decimal place suggests, because the classifier producing it is imperfect. HHEM's makers publish its accuracy on public test sets. On the RAGTruth summarization set, HHEM catches only about a third of the hallucinations it is asked to find: a recall of 31.86 percent, meaning roughly two of every three slip past it. Its precision there is high, near 76 percent, so the ones it does flag are usually real, but most of what it should catch, it misses.⁴

Independent testing is harsher on the hard cases. FaithBench gathered 660 summaries where the best automated detectors disagreed, then measured how well each detector did. HHEM landed at about 55.7 percent balanced accuracy and a zero-shot GPT-4o at about 56.3 percent, both a hair above a coin flip. As the paper puts it, "even the best hallucination detection models have near 50% accuracies on FaithBench."⁵

A grader this uneven cannot separate models sitting a point or two apart. Yet the leaderboard reports them to a tenth of a percent, and its top is a tight cluster: GPT-4o at 9.6 percent, Claude Opus 4 at 12.0, Sonnet 4 at 10.3, Gemini 2.5 Pro at 7.0.¹ Those leaders sit closer to one another than the classifier's own error. The answer rate hides a second confound. Phi-4 posts a lower 3.7 percent, but summarized only 80.7 percent of the documents, against GPT-4o's 93.8 percent. Skipping the roughly one in five it found hardest removes the cases most likely to be scored as hallucinated.¹

None of this makes HHEM a poor tool. A fast, consistent classifier is a sensible choice for a task this narrow. It is far steadier than a general chatbot pressed into service as a judge, where the order two answers appear in can swing the verdict. HHEM does its one job. The trouble starts when its output is read as more than that one job's score.

Three benchmarks, three different questions

Step outside Vectara's leaderboard and the word "hallucination" stops naming one quantity. Three widely cited benchmarks each define it differently, so their rates cannot be stacked into a single ranking. TruthfulQA asks 817 open questions written to bait common human misconceptions, with no document to be faithful to; its best model was truthful on 58 percent of them against 94 percent for people.⁶ RAGTruth annotates about 18,000 responses word by word across three separate tasks, counting any span the retrieved text does not support.⁷ FaithBench sorts summary hallucinations into four kinds, telling a harmless inference apart from a claim that contradicts the source outright.⁵ Putting Vectara's 9.6 percent next to a TruthfulQA figure compares a summary-fidelity score with a misconception score, two numbers that share a word and nothing else.

The gap bites hardest when the same models face open questions instead of summaries. OpenAI's own system card scores its reasoning models on PersonQA, a set of factual questions about people with no passage to lean on. There, o3 fabricated on 33 percent of questions and o4-mini on 48 percent, and the newer o4-mini, stronger on many tasks, hallucinated more than the older o1 at 16 percent.⁸ Trade coverage read the result as a surprising reliability regression.⁹

A single-digit summarization score and a one-in-three fabrication rate belong to the same class of model, measured on different tasks. The low board number does not predict the open-use one. Why a model fabricates at all, confidently filling a blank rather than admitting it does not know, is a mechanism of its own. The number supposed to track that behavior was measured on a task where the behavior barely shows.

The cost is not a line item anyone has totaled. It is a buyer choosing a model for a customer-facing job on a score that only ever measured summarizing. It is a policymaker citing "a 3 percent hallucination rate" as if it covered open questions. It is a public told a model "invents information 3 percent of the time" when the figure never touched the questions people actually ask.² The number is real and carefully made. The reliability it seems to promise is borrowed.