"AI eating AI will be doomed" is a false proposition. The real product risk is the lack of a data error correction mechanism.

New research in Nature reveals the degradation trajectory of AI under extreme conditions, but reality is far more complex than experiments. This article breaks down three cognitive misconceptions from a product perspective, proposes the RAID model framework based on Data Anchoring, Rectification, Intelligence, and Monitoring, reveals the hierarchical defense strategies from medical AI to entertainment products, and further points out the data pipeline design capabilities and emerging market opportunities that AI product managers must master at the current stage.

Model Collapse?

In July 2024, the cover of Nature published the research of Shumailov et al., which systematically revealed the Model Collapse phenomenon: in the completely closed recursive training of AI models, the 9th generation is completely distorted — outputs degenerate from medieval architecture to non-existent rabbit species.

The 2026 Stanford AI Index Report shows that AI-generated content accounts for 51.72% of newly released internet content. AWS research shows that about 57% of online texts have been processed by AI. High-quality human text data may be exhausted as early as 2026-2032. Synthetic data has become an industry reliance due to its cost advantage (synthetic images cost $0.06 vs. manual labeling costs $6).

Against this backdrop, the pessimistic argument that "AI will collapse if it feeds on its own outputs" has spread widely.

However, as product practitioners, we need to look past emotional narratives and analyze this issue from the perspectives of engineering and product design.

Three Cognitive Misconceptions About Model Collapse

Misconception 1: Equating Extreme Experimental Conditions with Reality

The experimental premise of the Nature paper is a completely closed loop — each generation only uses the output of the previous generation's AI, with zero human data. This is a theoretical extreme condition, not equivalent to real-world scenarios.

Extrapolating the conclusions of extreme condition experiments to reality makes the mistake of overgeneralizing from a partial sample.

Misconception 2: Ignoring the Recursive Analogy of Human Civilization

The development of human civilization itself is a recursive process: the vast majority of people use the achievements created by a few, without producing original knowledge. But civilization continues to progress because humans have established error-correction mechanisms.

The recursive model of human civilization:

Few people create original works → Most people apply them → Error-correction mechanism operates (experimental verification / peer review / real-world feedback) → Knowledge iteration and upgrading

The lesson of medieval scholastic philosophy: Pure closed recursion (only quoting classics without experimental verification) → Stagnation for a thousand years. The breakthrough of the Scientific Revolution: Introducing experimental verification (real-world anchoring) → Civilization takes off.

Recursion itself is not the problem; the lack of an error-correction mechanism is.

This principle also applies to AI.

Misconception 3: Ignoring Existing Solutions

In 2025, the LUMIA Lab of Shanghai Jiao Tong University, in collaboration with Tsinghua University, Peking University, and Beijing Academy of Artificial Intelligence, proposed the "Token-Level Editing" method, which was published at ICML 2025.

The core idea is not to reject synthetic data, but to perform fine-grained token-level correction on synthetic data to maintain the long-tail distribution characteristics of real data.

In May 2026, the research teams of the Norwegian University of Science and Technology and King's College London published a paper in *Physical Review Letters*: Adding even a single piece of real data during training can effectively prevent model collapse.

Even if this piece of data is far less than the AI-generated data, even if the generated data increases infinitely.

Product Framework: Design of AI Product Data Error-Correction Mechanism

Based on the above analysis, we can extract a design framework for the training data pipeline of AI products, which I call the "Data Anchoring, Rectification, Intelligence, and Monitoring Model".

【Data Anchoring】—— Real Data Anchoring

In each generation of the training data pipeline, a certain proportion of real human data is forcibly retained as an anchor point.

Product design essentials:

Define the "real data" standard: human-created, authoritatively verified, and with long-tail distribution characteristics

Set a hard threshold for the proportion of real data: ≥50% in key fields (medical/legal/financial), ≥30% in general fields

Establish real data source management: professional content creation, user-generated content (UGC), authoritative data licensing

Design a real data freshness mechanism: regularly supplement new human data to avoid outdated anchor points

Product decision: Real data is a scarce resource, and trade-offs need to be made between cost and quality.

It is recommended to manage products by risk level — high-risk products (medical AI) require a high proportion of anchoring, while low-risk products (entertainment AI) can appropriately reduce it.

【Rectification】—— Iterative Error Correction

Embed an error-correction link in the training loop to ensure quality assessment and feedback correction after each generation of training.

Product design essentials:

Design post-training quality assessment indicators: output diversity, long-tail knowledge coverage, factual accuracy

Establish a degradation detection mechanism: compare the distribution difference between the output of each generation of models and the previous generation to detect early signals of collapse

Design error-correction trigger rules: automatically trigger the error-correction process when the degradation indicator exceeds the threshold

Establish an error-correction execution pipeline: introduce real data, adjust the proportion of synthetic data, apply methods such as ToEdit

Product decision: The frequency of error correction is a key parameter.

Humans correct errors on an annual basis (paper publication cycle), while AI can correct errors on an hourly basis (automatic assessment + automatic triggering).

It is recommended to design an automated error-correction pipeline to reduce manual intervention delays.

【Intelligence】—— AI Data Intelligent Identification

Identify and mark AI-generated content in the training data pool, and establish a traceable system for data sources.

Product design essentials:

Deploy AI content detectors: evaluate the probability of AI generation for crawled/purchased data

Establish a data source label system: mark the source of each piece of data (human-created / AI-generated / mixed)

Design a data pollution early warning mechanism: trigger an alert when the proportion of AI content in the data pool exceeds the threshold

Establish a data traceability chain: the entire chain from data collection to model training is traceable

Product decision: The current accuracy of AI content detection technology is about 80-90%, and there is a risk of misjudgment.

It is recommended to use it as an auxiliary signal rather than an absolute standard, in conjunction with manual review.

【Monitoring】—— Distribution Monitoring

Continuously monitor the distribution characteristics of training data and model outputs to prevent pattern collapse and the disappearance of long tails.

Product design essentials:

Monitor training data distribution: lexical diversity, topic coverage, proportion of long-tail knowledge

Monitor model output distribution: output diversity indicators, repetition rate, novelty score

Design distribution deviation alerts: alert when the deviation between the output distribution and the target distribution exceeds the threshold

Establish a distribution repair mechanism: repair deviations through technologies such as data augmentation and distribution correction

Product decision: Distribution monitoring is the "early warning system" for model collapse.

It is recommended to design a real-time distribution monitoring panel in the product, so that product managers and engineers can intuitively see the data health status.

Model Application Scenarios

Percentage of real data: Medical AI products > Legal AI products > General conversational AI > Content generation AI > Entertainment AI products

Detection accuracy: High precision, Standard precision, Basic precision

Error correction frequency: Single, Regular, On-demand

Monitoring frequency: Real-time, Daily, Weekly, Monthly

Core Judgments

1. Model collapse is not the end of AI.

What it exposes is not the ceiling of AI technology, but the shortcoming of the data pipeline.

2. The argument that "AI will collapse if it feeds on AI" is a false proposition.

The real risk is not recursion itself, but closed recursion without an error-correction mechanism.

The successful recursion of human civilization proves that: recursion with an error-correction mechanism is an engine of progress, while recursion without an error-correction mechanism is a spiral of degradation.

3. Humans have autonomous consciousness, and AI has scalable capabilities.

The two evolutionary paths are different but both feasible.

AI's hourly error-correction frequency is much higher than humans' annual frequency, which is AI's unique advantage.

4. The core competencies of AI product managers are changing.

From focusing on model parameters and functional design, it has expanded to training data quality management and error-correction mechanism design.

The RAID model provides a practical framework for implementation.

5. The biggest product opportunity created by model collapse lies in AI data infrastructure.

Real data asset management, synthetic data quality engineering, human-machine collaborative verification platforms — these are new tracks for AI products.

This article is from the WeChat official account "Everyone is a Product Manager" (ID: woshipm), author: Sen Linyu, published with authorization from 36Kr.