Meta's Galactica invented citations for real researchers, and its public demo lasted about two days

Meta trained a language model on 48 million papers

Galactica was a language model that Meta's Papers with Code team trained only on scientific writing, with Ross Taylor as its first author. The corpus ran to 106 billion tokens, drawn from 48 million papers, textbooks, and lecture notes, plus millions of chemical compounds and protein sequences.¹ The largest version held 120 billion parameters. The team released the model family openly and presented it as a new interface for science, a single tool that could draft a literature review, work through a formula, or answer a technical question.¹

On 15 November 2022 they put a public demo online at galactica.org, where anyone with a browser could type a prompt and read back paragraphs of technical prose. The team posted the paper describing the model to arXiv the next day.¹ Yann LeCun, Meta's chief AI scientist and the model's most prominent promoter, told users what to expect: type a text, he wrote, and Galactica would generate a paper with the relevant references and formulas.²

On launch day it wrote a false biography and invented a paradox

The same day the demo opened, Ernest Davis and Andrew Sundstrom, computer scientists at NYU, ran a set of prompts through it and wrote down what came back.³ Asked for an encyclopedia article on the experiment that measured the gravitational constant, the model produced a fluent article about a different experiment. It built a biography to match, sending the physicist Leonard Schiff to a professorship at the University of Maryland. Schiff spent his career at Stanford.³ Asked about the transit of Venus, it listed transit dates from centuries before the telescope. It stated that the last known transit was in 1882, as though the widely watched transits of 2004 and 2012 had never happened.³ Asked to explain the Solomonov-Russell paradox, which does not exist, it did not decline. It invented a coin-flipping thought experiment and set it out in the same even voice as the rest.³

Others found the same thing on subjects they knew cold. Michael Black, who directs a Max Planck institute for intelligent systems, tested the live model and reported the pattern in a public thread. The output was "wrong or biased but sounded right and authoritative," he wrote, and it fabricated citations, hanging invented results on the names of real researchers. He warned it could bring on "an era of deep scientific fakes."⁴ The day after launch, Gary Marcus called the results "pitch perfect and utterly bogus imitations of science and math, presented as the real thing."⁵

None of this slipped out unlabeled. Every result the demo returned carried a printed warning.

The warning was accurate, and the fabrications went out under it anyway. The demo even screened for some content: ask it about racism or AIDS and it refused, replying that the query did not pass its content filters.² It had no equivalent check on whether an answer was true. What it returned looked like a settled reference, and a reader who trusts that form reads straight past a caution printed beside it.

Criticism was public before Meta paused the demo

The demo did not come down ahead of the criticism. The most detailed expert warning was already public when Meta paused it, on 17 November 2022.

1. 17 Nov, 06:47 UTC

   Michael Black calls the model dangerous⁴

   In a public thread he reports that on topics he knows well the output is wrong yet authoritative, and warns of an era of deep scientific fakes.

2. 17 Nov, 15:05 UTC

   Papers with Code pauses the public demo⁷

   The team thanks users for their feedback and keeps the model available to researchers who want to reproduce the paper. It concedes no specific fault.

3. 17 Nov, 17:20 UTC

   LeCun says the demo is offline⁸

   Meta's chief AI scientist writes that it is no longer possible to have some fun by "casually misusing it."

That left the public demo online for about two days, from 15 to 17 November. Much of the coverage counted the calendar dates and called the run three days, and the phrase stuck.²

Meta did not describe the model as broken. LeCun's line put the fault on the critics for misusing it. A year later Joelle Pineau, who led AI research at Meta, gave the fuller account. Galactica had been a research demo, pulled so people were not misled into treating it as a product, and released without the responsible-use guidance that would have set expectations. "The gap between the expectation, and where the research was, was too big," she said.⁹ That account is a real one, and the demo was labeled a demo. It also runs against the model's own paper, which had measured the failure and warned against exactly the use the demo invited.

A science interface makes a guess read as a result

Galactica did not look facts up and report them. Like any language model, it produced the most likely next stretch of text given the prompt, and a citation is just more text to continue. The Nightly Build has a lesson on why a model builds a fake citation the same way it builds a true one; the short version is that nothing inside the model separates a real reference from a plausible-looking one. Both are patterns it learned to continue.

What Galactica added was the surface. Its output wore the form of a review paper: the register, the structure, the trailing list of references. That surface is what a reader leans on to decide whether to trust a claim, so an invention arrived carrying the same signals as a result. The team was not blind to this. Their own paper names hallucination as a property of storing knowledge in a model's weights, where information may blend together rather than be retrieved intact.¹

They also measured it. In an ablation on 1,705 methods, the 6.7-billion-parameter model got the citation right 13.8 percent of the time for papers that had been cited only once, and invented the citation outright more than half the time. For papers cited more than 500 times, it never fabricated the citation at all.¹

The model reproduced the references it had seen often and invented the ones it had barely seen. Obscure work, the kind a real literature search most needs to surface, is exactly where it failed.¹ The same paper's limitations section said the model likely required augmentation before being used in a production setting.¹ The public demo put the unaugmented model in front of anyone with a browser.

Fluent citations still ship in research assistants

The builders had already named the fix. Their paper said the model would need augmentation and verification before any production use.¹ The augmentation they meant is now common: today's research assistants and answer engines pair a language model with a retrieval step, so the citations can point at documents that were actually fetched. That closes part of the gap Galactica left open. It does not close all of it, because the model still writes the claims around those documents, and it can still describe a fetched source as saying something it does not.

This was not Microsoft's Tay, corrupted in hours by users who fed it slurs. No one had to attack Galactica. It fabricated from its training, doing what it was built to do. And the fabrication does not stay on the screen. The clearest picture of where it lands is a courtroom: a lawyer once filed a brief citing six cases a chatbot had invented, and paid for it.

Galactica failed loudly, in front of scientists who could check its work inside a day. A retrieval-backed assistant fails more quietly. It gets most of its citations right, and that is what makes the invented one hard to catch.