The AI Reader.
When AI Breaks

Bard's launch demo credited the James Webb telescope with a discovery ESO made in 2004

Google published the claim in its own promotional post two days before Bard's Paris launch, when the real first photograph of a planet beyond our solar system was already eighteen years old.

Why this matters

On February 6, 2023, Google announced Bard, its answer to ChatGPT, and picked one example to show what it could do: explaining a NASA space telescope to a nine-year-old. The example was wrong, in a specific and checkable way, and it went out under the company's own name two days before a launch event in Paris. Alphabet's stock fell that same week, and the two facts get told together so often that the second sounds like proof of the first. By the end you will know exactly what Bard's demo got wrong, and why a language model can write a false sentence as fluently as a true one. You will also know where to look for the same failure in the AI answers you read today.

Background optional reading
01
A model builds a fake citation the same way it builds a true one: why a model states a false fact as fluently as a true one.
02
Post-training makes a model less able to tell when it's wrong: why the confident tone a model uses is not a reading of how often it turns out to be right.

The example Google chose to show off Bard

Sundar Pichai, chief executive of Google and Alphabet, announced Bard on February 6, 2023, in a post titled “An important next step on our AI journey.”1 He described it as “an experimental conversational AI service, powered by LaMDA,” opened first to “trusted testers” ahead of a wider public release “in the coming weeks.”1

The post offered one worked example of what Bard could do: an animated GIF showing Bard “helping you to explain new discoveries from NASA's James Webb Space Telescope to a 9-year-old.”1 The same GIF ran on Google's Twitter account that day.2 Among the bulleted facts Bard's answer gave the imagined nine-year-old was this sentence, exactly as it appeared:

The answer Bard gave
“JWST took the very first pictures of a planet outside of our own solar system.”2 Bard, in Google's Feb. 6, 2023 promotional GIF

It is a plain, checkable sentence, and it is false. By February 8, two days after Google posted it, the error was public: Engadget reported the line and noted that the first image of a planet outside the solar system belongs to the European Southern Observatory's Very Large Telescope, from 2004.2

What a telescope actually photographed in 2004

Astronomers had already published, twice, exactly when the first exoplanet was imaged and by which instrument, and neither date is 2023.

  1. April 2004

    ESO detects a candidate exoplanet3

    Using the NACO adaptive-optics instrument on Yepun, one of the Very Large Telescope's four 8.2-meter units at Paranal in Chile, the European Southern Observatory detected a faint, very red point of light beside the brown dwarf 2M1207. ESO called it a giant planet candidate companion and said confirmation was still pending: “If the candidate companion of 2M1207 is really a planet, this would be the first time that a gravitationally bound exoplanet has been imaged around a star or a brown dwarf.”3

  2. 30 April 2005

    ESO confirms the first exoplanet image4

    A year of follow-up observations showed the object was bound to 2M1207 and about five Jupiter masses, a planet rather than a background star. ESO's own words: “Our new images show convincingly that this really is a planet, the first planet that has ever been imaged outside of our solar system.”4

  3. 1 September 2022

    JWST takes its own first exoplanet image5

    NASA's Webb team announced the telescope's first direct image of a world beyond the solar system, the gas giant HIP 65426 b. The same release said so plainly. “While this is not the first direct image of an exoplanet taken from space — the Hubble Space Telescope has captured direct exoplanet images previously — HIP 65426 b points the way forward for Webb's exoplanet exploration.”5

So Bard's demo did not just misdate an obscure fact. It handed JWST a record ESO had held, and had said it held, for eighteen years, and it did so five months after NASA had described the telescope's own much narrower first as explicitly not a first.

A stock drop the reporting won't pin on the demo

Google did not pull the demo or delay the launch. One day after Microsoft unveiled ChatGPT built into Bing, Google went ahead on February 8 with the Paris event where it planned to show Bard more widely. Reuters reported that analysts said the event lacked details on how Google would answer Microsoft's ChatGPT challenge.6 That same Wednesday, Alphabet's shares fell.

−8%Alphabet, to a $99.05 close, Feb. 8 2023
~$100Bmarket value lost that day
−9%intraday low, same session

Reuters put the closing move at 8 percent, or $8.59 a share, to $99.05, and more than $100 billion in market value.6 NPR reported the shares had slid as much as 9 percent intraday, and that Microsoft's stock rose 3 percent the same week on its own ChatGPT news.7 No source ties the fall to Bard's error alone. It arrived in the same two days as a Google event analysts said lacked details on how it would answer Microsoft's ChatGPT challenge, and a rival's competing launch. The reporting on the drop places all three side by side rather than picking one cause.6

Asked about the error, a Google spokesperson tied it to process rather than to any one person: “This highlights the importance of a rigorous testing process, something that we're kicking off this week with our Trusted Tester programme.”6

Why the sentence came out so sure of itself

Nothing about the JWST line was a lookup that failed. A language model like the one behind Bard writes by predicting one likely word after another from patterns in its training text, and it has no built-in check on whether the finished sentence is true. The false claim about JWST was written the same way the correct bullets in that same promotional GIF were written: word by word, from what usually follows, in a launch post Google itself had published two days earlier.1

It also did not read as a guess. Every bullet in Bard's answer, right ones and the wrong one alike, was delivered in the same flat, declarative voice.2 That is not a coincidence of style. A model's stated confidence is not a reading of how often it is actually right, so a false sentence about a space telescope comes out sounding exactly as certain as a true one about the same telescope. Fluent and confident is what the writing does by default. Correct is not part of the specification.

None of this needed the demo to be careless in some unusual way. Bard's underlying technology writes fluent, confident text by default, whether or not the specific claim is true. Google aimed that technology at exactly the kind of question where a fluent wrong answer is cheapest to produce and easiest to miss before publishing: something specific and checkable, explained to a complete novice. This was not a rare malfunction. It is the ordinary behavior of the technology, and Google's own launch ad is where it happened to surface.

The failure moved from a demo to a search bar

The mechanism did not stay inside one promotional GIF. In May 2024, Google rolled AI Overviews out across Search in the United States, putting an AI-written summary above the results for many queries. Elizabeth Reid, the company's VP of Search, wrote afterward: “But some odd, inaccurate or unhelpful AI Overviews certainly did show up.”8 The same post also states that AI Overviews “generally don't ‘hallucinate’” the way other language-model products do, because the feature is built to check its answers against retrieved web pages.8 That claim is Google's own account of its own design, not an outside finding, and it sits next to the same post's admission that wrong answers reached real users at Search's scale.

The problem is not that AI answers are usually wrong. It is that fluency was never the signal to check. Bard's launch demo and an AI Overview both write in the same finished, declarative voice, whether the underlying claim is right or invented. The only way to tell the difference is the one thing the writing itself does not do: check the claim against a source that actually settles it.

The takeaway

Bard's demo did not glitch. It wrote a false sentence about JWST the same way it wrote every true sentence around it in that GIF: fluently, from patterns, with nothing inside the system stopping to check the claim against ESO's own published record. Alphabet's stock did fall that week. The reporting on the fall will not let you credit the ad alone. A Paris event analysts said lacked details on how Google would answer Microsoft's ChatGPT challenge, and a competitor's launch the day before, share the same two days, and no source separates their effects. What you can carry away is narrower and more durable than a stock-price story: a launch demo, like any AI-written answer, is not a test of whether it is true. The confidence in the sentence is not evidence for the sentence, whether it is printed in a promotional GIF or sitting at the top of a search result you did not think to double-check.

Go deeper optional reading
01
Yes, it is the Image of an Exoplanet: ESO's own 2005 announcement of the discovery Bard's demo misattributed to JWST.
02
AI Overviews: About last week: Google's own account of the wrong answers AI Overviews produced after its 2024 Search rollout, and how it responded.

Sources

  1. Google · Sundar Pichai, “An important next step on our AI journey” (Feb. 6, 2023)
  2. Engadget · Will Shanklin, “Google's Bard chatbot confidently spouts misinformation in Twitter debut” (Feb. 8, 2023)
  3. ESO · “Is This Speck of Light an Exoplanet?” (Sept. 10, 2004)
  4. ESO · “Yes, it is the Image of an Exoplanet” (April 30, 2005)
  5. NASA · “NASA's Webb Takes Its First-Ever Direct Image of Distant World” (Sept. 1, 2022)
  6. Reuters, via Al Jazeera · “Google shares tank 8% as AI chatbot Bard flubs answer in ad” (Feb. 8, 2023)
  7. NPR · Emily Olson, “Google's AI chatbot, Bard, sparks a $100 billion loss in Alphabet shares” (Feb. 9, 2023)
  8. Google · Elizabeth Reid, “AI Overviews: About last week” (May 30, 2024)