Google's AI restated a real Reddit glue joke as fact

From the I/O stage to glue in the sauce

On May 14, 2024, at its I/O developer conference, Google began putting an AI-written answer at the very top of Search. Elizabeth Reid, the vice president who runs Search, announced that AI Overviews would "begin rolling out to everyone in the U.S.," reaching hundreds of millions of people that week, with a target of more than a billion by the end of the year.¹ An AI Overview is a short summary the system writes above the ordinary list of blue links. It does not answer from memory. It runs the user's query, pulls a handful of web pages that match, and composes its answer from what those pages say. That design has a name: retrieval-augmented generation, where a model fetches text and writes its reply from what it fetched.

Within about two weeks, people began sharing the answers they got. Asked why the cheese would not stick to a homemade pizza, the AI Overview offered, among ordinary tips, "adding about ⅛ cup of non-toxic glue to the sauce to give it more tackiness."² The Daily Dot's Marlon Ettinger traced the line, on May 23, 2024, to an eleven-year-old Reddit comment by a user posting as u/fucksmith, who had written it as a joke.² The original comment could not be pulled up on its own page, so its wording and age rest on that reporting and on Google's own later acknowledgment of the example, not on the thread itself. The next day, NBC News confirmed the pizza answer and added a caution that would matter: some of the more alarming screenshots then circulating "seem to be fake."³

A rock a day

A second answer spread alongside the first. To the question "How many rocks should I eat?" the Overview relayed that geologists recommend eating at least one small rock a day.⁴ The advice was real in one narrow sense: a real page said it. The page was a 2021 article from The Onion, the satirical newspaper, headed "Geologists Recommend Eating At Least One Small Rock Per Day," crediting the recommendation to a fictional Berkeley geologist and advising readers to hide loose rocks "inside different foods, like peanut butter or ice cream."⁵

Reid would later explain this answer differently from the pizza one. Before the screenshots went viral, she wrote, "practically no one asked Google that question," and the only matching content was the satire, which "happened to be republished on a geological software provider's website."⁴ So the system did not reach past better sources to dig up The Onion. There were almost no other sources to reach past. Google has a term for that condition.

About last week

On May 30, 2024, Reid published a postmortem titled "AI Overviews: About last week." She granted that "some odd, inaccurate or unhelpful AI Overviews certainly did show up."⁴ She also drew a firm line around what had actually happened. A large batch of the scariest screenshots then spreading, telling people it is safe to leave a dog in a hot car, that smoking during pregnancy is fine, and giving dangerous answers about depression, were fabrications. "Those AI Overviews never appeared," she wrote, and she urged anyone who saw them to run the search and check.⁴ The glue and the rocks were real. Much of what went viral around them was not.

On scale, Google put a number forward in its own defense: it had found a content-policy violation on "less than one in every 7 million unique queries" on which an Overview appeared.⁴ Set against a feature just switched on for hundreds of millions of Americans,¹ even a rate that low is a large count of bad answers, and the disagreement is really about which figure to hold in view, the rare rate or the enormous surface it runs on. Google made more than a dozen fixes: better detection of nonsensical queries, new limits on drawing from satire and from user-generated forum posts, and added care on health questions.⁶

How a real joke became an answer

So how does an eleven-year-old joke end up stated as fact at the top of the most-used page on the internet?

Three pieces the course has already covered combine here. The first is retrieval itself: the system matched the query about cheese sliding off pizza to a comment thread that was, word for word, about cheese sliding off pizza. As a text match, u/fucksmith's comment was excellent. The second is that a language model draws no line between an instruction and a fact; the glue tip arrived as a run of words like any other, with nothing in it marked as a joke. The third is what happens at the output: the model's task is to write one fluent, confident answer, and nothing in that step checks whether the source it is repeating deserves to be believed. Retrieval finds the closest text. It has no reading of who wrote the page, or why.

This is a different failure from the ones this desk has covered before. When Air Canada's chatbot invented a refund policy or a lawyer's ChatGPT brief cited six court cases that did not exist, the system made something up. Here it made nothing up. The joke was real, the satire was real, and the AI repeated both accurately. The gap is not invention. It is that retrieval carries no model of whether a real source can be trusted.

Google's own account pushes back on that reading.

That account is partly right, and the rocks answer is where it fits best. A data void is exactly "not having a lot of great information available," and whether the system fully grasped that The Onion was joking is a fair question. Reid separately noted that in some cases the system had "misinterpret[ed] language on webpages."⁸

But the glue answer defeats the misinterpretation reading. The Overview did not misread u/fucksmith. It reproduced the joke exactly: the same ⅛ cup, the same non-toxic glue, the same tackiness.² Nothing was lost or garbled on the way from the source to the answer. The comprehension was perfect. What was missing was any sense that the perfectly understood source was a troll. That is the harder failure to fix, because you cannot patch it by teaching the model to read pages better. It already read the page.

Where the gap is now

That gap did not close when Google patched these particular queries. It is built into any system that answers by retrieving web text and restating it, which now takes in AI Overviews itself and the AI-answer features in the search engines and assistants the reader opens daily. Each can tell you, accurately, what some page on the internet says. None of them can tell you whether that page deserved to be believed. The fixes Google shipped were mostly a hand-written list of sources to distrust: satire, forum posts, nonsense queries.⁴ A list like that is not the same as a system that knows, on its own, when a strong text match is a bad source.