Generative models fed only their own output collapse within nine generations

Nine generations from real text to a list of color morphs

In May 2023, Ilia Shumailov and coauthors described a failure mode they called model collapse. Train a generative model on text or images that an earlier model generated, feed that output back in as if it were real data, and repeat.¹ Just over a year later the same team published a refined version of the study in Nature, with the same finding and the same demonstration.² Trained on its own output, they write, a model's data "end up polluting the training set of the next generation," and being trained on polluted data, the model comes to "mis-perceive reality."² The rare, unusual cases in the training distribution disappear first, what they call early collapse. Run the process long enough and the output narrows into something that barely resembles the original data at all, late collapse.²

They tested this on OPT-125m, a small open language model, fine-tuned on Wikipedia text. Fine-tuned once on the real data, the model scored 34 on perplexity, a standard measure of how surprised a model is by real text, where lower means the model predicts the text better.² Feed that model's own output back in as training data, fine-tune a new model on it, and repeat for nine generations with none of the original text kept. Perplexity climbs by 20 to 28 points, roughly 60 to 80 percent worse than where it started.² The words drift along with the score: the boxed example in the 2023 precursor shows a passage sliding, generation by generation, into a list of jackrabbit color varieties by generation nine.¹

The same paper ran a third condition worth flagging early: instead of discarding all the original data, they kept a fixed 10 percent of it at every generation. Perplexity still rose, just more slowly.²

Keeping the old data around stops the collapse across three tasks

Gerstgrasser and coauthors tested a different rule.³ Instead of replacing the training set at every generation and discarding everything older, they accumulated it. Keep all the original real data, keep every earlier generation's synthetic output, and add each new generation's output on top: the pool only grows.³ They ran replace against accumulate on causal language models (GPT-2 and three sizes of Llama-2) pretrained on short stories, on a diffusion model generating molecular structures, and on variational autoencoders trained on faces. Replacing degraded all three; accumulating held test error roughly flat, or nearly flat for the diffusion model, across five to nine rounds of retraining.³

They also proved why, in a simplified case they could work out exactly: a sequence of linear models, each fit on the data available at that step. Replacing at every iteration makes test error grow in direct proportion to the number of iterations. Accumulating keeps it bounded, no matter how many iterations run. Each new generation's added noise gets diluted by a weight that shrinks in proportion to one over the iteration number squared, as the pool of real and synthetic data keeps growing.³ Gerstgrasser and coauthors put the upshot plainly: "these results strongly suggest that the 'curse of recursion' may not be as dire as had been portrayed — provided we accumulate synthetic data alongside real data, rather than replacing real data by synthetic data only."³

A fixed slice of the old data slows the collapse without stopping it

It is tempting to read Shumailov's 10 percent condition as a small version of Gerstgrasser's accumulation, proof the same fix showed up twice. It is a different operation. Ten percent retention keeps a fixed, constant slice of the original data at every generation, the same size each time.² Accumulation keeps the entire growing history: all the real data plus every earlier generation's synthetic output. The fraction that is real data shrinks over time even as the absolute amount of real data never drops.³ Partial, fixed retention is the condition that only slowed Shumailov's collapse. Full, growing accumulation is the condition Gerstgrasser's team showed avoiding it. Reading the first as a miniature of the second overstates what either paper found.

One architecture and one theorem complicate the fix

Gerstgrasser's own paper names an exception to its result. Citing earlier work by Martínez and coauthors, they note that a different architecture, tested on a much smaller dataset, showed fast performance loss "even with accumulating data."³ They call understanding why "an interesting direction... for future research," not a solved problem.³

A later, separate theoretical paper complicates the picture further. Five months after Gerstgrasser's paper, Dohmatob and coauthors proved a related result in a comparable linear-regression setting. Whenever the synthetic share of the training mix stays a fixed fraction instead of shrinking toward zero as the dataset grows, collapse persists, "even the smallest fraction of synthetic data (e.g., as little as 1% of the total training dataset)."⁴ On its face that sounds like it targets accumulation directly. It does not straightforwardly refute it, because the two papers assume different things about how the synthetic share behaves. Gerstgrasser's own theorem has that share shrinking on its own, proportional to one over the iteration number squared, under their specific accumulation scheme, which is close to the condition Dohmatob's proof exempts.³⁴ The two papers do not cite or engage each other on this point. Whether real accumulation schemes behave more like Gerstgrasser's shrinking share or Dohmatob's fixed one is an open, empirical question, not one either paper answers for the other.

The strongest push for provenance is still the original paper's

Who is pressing the collapse worry today, and asking for what? The clearest primary answer is still the one in the original 2024 paper's own discussion. Preserve access to real, human-generated data, and coordinate across the field on tracking where content on the web actually came from.² Yarin Gal, one of the paper's coauthors, put the stakes more bluntly at publication: "Model collapse is the AI equivalent of a feedback loop gone wrong. The more models feed on their own output, the further they drift from reality."⁵ Independent reporting on the same study carried the same framing, that companies retaining access to human-generated content may hold a structural advantage.⁶ Nothing more recent and equally authoritative restates or updates that position: more than two years on, the loudest primary voice on what to do about model collapse is still the team that demonstrated it.

The tension the collapse worry sits inside is that synthetic data has also made models better, when someone curates it deliberately. This paper's lesson on phi-1 covers the case in full: a small code model trained mostly on filtered web text plus a meaningful share of textbooks and exercises an earlier model wrote. It matched or beat StarCoder-Prompted, a code model built with about twelve times its parameters and more than a hundred times its training tokens.⁷ Gunasekar and coauthors never mention model collapse; their subject is data quality, not the replace-versus-accumulate question Shumailov and Gerstgrasser were testing.⁷ It is not a counter-example to either paper. It is evidence that synthetic data chosen deliberately, for a narrow purpose, behaves differently than synthetic data an internet-scale model absorbs indiscriminately. That gap, between curated use and indiscriminate use, is close to the gap between what has been demonstrated and what is still extrapolation about the open web.

Not every researcher reads the collapse literature as settled ground. Schaeffer and coauthors argue the field uses at least eight distinct, sometimes conflicting, definitions of model collapse, which makes it hard to say what any single study actually proved. They also argue that several of the most-cited collapse scenarios rest on assumptions that poorly match how models are actually trained.⁸

Even that reading does not dispute the core demonstration: replacing real data with a prior generation's synthetic output collapses models in every setting both papers tested. The disagreement is entirely about how far that finding travels beyond those settings.