Model collapse

Model collapse is a gradual decline in the quality of generative models that are trained, generation after generation, on data produced by earlier models. Ilia Shumailov and colleagues, who named the effect, describe it as "a degenerative process affecting generations of learned generative models, in which the data they generate end up polluting the training set of the next generation."^[1] Models affected this way first lose rare, unusual examples from their output and may eventually produce output that bears little resemblance to the original data.^[1]

The concern is practical because large language models and image generators are trained largely on data scraped from the web, and a growing share of the web is itself AI-generated.^[1] Low-quality AI content of this kind is often called AI slop. The effect has also been called model autophagy disorder (MAD) and, informally, Habsburg AI.^[2][3]

Researchers disagree about how likely model collapse is in practice. Later studies found that it can be avoided when AI-generated data is added to, rather than substituted for, real data, and one review counted eight different definitions of the term in the research literature.^[4][5]

History and names

Shumailov, Zakhar Shumaylov, Yiren Zhao, Yarin Gal, Nicolas Papernot, and Ross Anderson first described model collapse in a preprint, "The Curse of Recursion: Training on Generated Data Makes Models Forget", posted on May 27, 2023.^[6] A revised version was published in Nature in July 2024 as "AI models collapse when trained on recursively generated data".^[1]

In July 2023, a team including Sina Alemohammad and Richard G. Baraniuk described a similar effect in image generators. They called repeated training on a model's own outputs an "autophagous" (self-consuming) loop, and named the resulting damage model autophagy disorder, by analogy with mad cow disease.^[2]

The informal name Habsburg AI was coined in February 2023 by Jathan Sadowski, who described "a system that is so heavily trained on the outputs of other generative AI's that it becomes an inbred mutant, likely with exaggerated, grotesque features." The name refers to inbreeding in the Habsburg dynasty.^[3]

Mechanism

A generative model learns a statistical picture of its training data — how often words, phrases, or image features appear. No model learns this picture perfectly. When the next model learns from the first model's output instead of the original data, it inherits those imperfections and adds its own. Over many generations the errors compound.^[1]

Shumailov and colleagues identify three sources of error:^[1]

Early and late collapse

The authors distinguish two stages. In early model collapse, "the model begins losing information about the tails of the distribution" — the rare cases. In late model collapse, "the model converges to a distribution that carries little resemblance to the original one, often with substantially reduced variance."^[1]

An analogy is a photocopy of a photocopy: each copy is nearly as good as the one before, but faint details disappear first, and after enough copies the image degrades into something quite different.

Evidence

Language models

In the Nature study, the researchers fine-tuned a small language model, OPT-125m, on a dataset of Wikipedia articles (wikitext2), then trained each new generation on text produced by the previous one.^[1]

The study gave an example of the drift. Given a prompt about medieval church architecture, the first-generation model produced text about Perpendicular Revival architecture. By the ninth generation the output read: "architecture. In addition to being home to some of the world's largest populations of black-tailed jackrabbits, white-tailed jackrabbits, blue-tailed jackrabbits, red-tailed jackrabbits, yellow-".^[1]

The same paper showed the effect in simpler models: variational autoencoders and Gaussian mixture models.^[1]

Image models

Alemohammad and colleagues studied three kinds of self-consuming loops using image generators. They concluded that "without enough fresh real data in each generation of an autophagous loop, future generative models are doomed to have their quality (precision) or diversity (recall) progressively decrease."^[2]

Scaling laws

Neural scaling laws describe how model performance improves as models and datasets grow. In a paper presented at ICML 2024, Elvis Dohmatob and colleagues studied how those laws change when training data includes synthetic data. They predicted several effects, including a loss of the benefit from scaling and the loss of skills, and tested them with transformers trained on arithmetic and with the Llama 2 language model.^[7]

Criticism and debate

Replacement versus accumulation

Early experiments assumed that each generation's data replaces the previous generation's. A 2024 study led by Matthias Gerstgrasser tested the alternative: keeping all earlier real and synthetic data and adding new synthetic data to it. With accumulation, they found that collapse was avoided in language models, molecule-generating diffusion models, and image-generating variational autoencoders. They also showed mathematically that, under accumulation, "test error has a finite upper bound independent of the number of iterations."^[4]

Competing definitions

In a 2025 position paper, Rylan Schaeffer, Joshua Kazdan, Alvan Caleb Arulandu, and Sanmi Koyejo argued that research on model collapse "actually encompasses eight distinct and at times conflicting definitions":^[5]

1. Catastrophic increase of population risk (error on the true data distribution)
2. Any increase of population risk
3. Asymptotically diverging population risk
4. Collapsing variance
5. Change in scaling law
6. Disappearance or entanglement of real data modes
7. Disappearance of real tail data
8. Appearance of hallucinated data

They argued that realistic settings resemble accumulation, because "real data are not deleted en masse" as new models are trained. Under those conditions, they concluded, "population risk will not increase catastrophically or diverge asymptotically." They still expected losses at the margins: "Real tail data and modes will be lost, but how many and how quickly is unclear."^[5]

A statistical inevitability?

Ali Borji argued in a 2024 note on the Nature paper that its results reflect a basic statistical property of repeatedly fitting and sampling from distributions, and that "the outcomes reported are a statistical phenomenon and may be unavoidable."^[8]

Open questions

• Real-world extent. The experiments above use deliberately simplified loops. How much AI-generated content current training datasets actually contain, and whether deployed models have lost measurable quality as a result, has not been established.^[5]
• Loss of rare content. Even researchers who consider catastrophic collapse unlikely expect some loss of rare "tail" data; how much, and how quickly, is unknown.^[5] Answering this would require tracking how often rare facts, styles, and languages appear in model output across successive model releases.
• Provenance. The Nature authors argued that preserving access to original data and to data not generated by language models is necessary to sustain learning, and that this may require "community-wide coordination" among AI developers to track where data comes from.^[1] Whether reliable provenance tracking, such as labeling or watermarking of AI output, can work at web scale remains open.

Analysis: effects on human agency

This section contains value judgments based on the evidence cited above.

The first thing model collapse removes is the tails of the distribution.^[1][5] In human terms, the tails are the unusual: rare facts, minority viewpoints, less common languages and styles. People increasingly use AI systems to find information and to write. If those systems lose the tails of human knowledge, the range of ideas people encounter narrows without their noticing, and choices that rest on less common knowledge become harder to discover. This reduces agency even if average output quality stays high.

The research also shows the lasting value of human-made data. The mitigations tested in these studies depend on keeping a supply of genuine human data alongside synthetic data.^[1][4] This gives weight to the work of people who write, record, and curate original material, and to the institutions — archives, libraries, reference works — that preserve it.