Can generative models also be trained end-to-end? The core turns out to be a for loop.

In 2012, AlexNet ended an era with a landslide victory. Before that, image recognition relied on manually designed hierarchical feature extraction pipelines; AlexNet proved a fact that was repeatedly verified later: handing over the entire task to the model for self-learning end-to-end almost always outperforms the carefully designed phased pipelines by humans.

From image classification to object detection and then to image segmentation, behind every leap of deep learning is the same logic: let it learn the whole task in one go.

There has always been only one exception: generative models.

The most powerful and scalable generative models today (whether autoregressive or diffusion models) are not end-to-end.

During training, they only learn to predict "one small step", but during inference, they need to unfold this step repeatedly hundreds or thousands of times like a recurrent network.

Training and inference do not use the same sampling method. This discrepancy brings up an old problem: the error of each step is fed into the next step, the input gradually drifts away from the distribution seen during training, and errors accumulate layer by layer. Academically, this is called "exposure bias".

In other words, "end-to-end performs better" — this core experience of deep learning — has never been truly applied to generative models for more than a decade.

Recently, a paper from UIUC and Harvard University attempts to fill this last piece of the puzzle.

The authors named this new paradigm Explorative Modeling, abbreviated as XM. Its idea is so simple that it seems almost naive, but it points to a bold conclusion: In addition to parameters and data, generative models actually have a third amplifiable dimension.

Project website: https://explorative-modeling.github.io

Paper link: https://arxiv.org/abs/2607.27372

Code repository: https://github.com/alexiglad/XM

Root of the Problem: Models Only "Take the Average"

To understand what this paper is solving, we must first understand what makes generation so difficult.

For ordinary supervised learning (such as classification), each input basically has only one correct answer, and the model only needs to learn a definite mapping.

But generation is different. If you ask the model to "generate a dog", there may even be an infinite number of correct answers. These valid outputs are individual modes (independent peaks in the distribution) in the data distribution. The difficulty of generation lies precisely in the need to capture so many modes at the same time.

The trouble is that mainstream generative models use reconstruction loss (such as squared error) during training. When an input is randomly matched with many different valid targets, the optimal solution that the reconstruction loss can give is the average of these targets. For most data, the average value is not on the data manifold at all, but falls in the middle of several modes, and resembles none of them.

The figure in the paper is very intuitive: if the model is allowed to perform end-to-end regression directly without any tricks, the three clusters of scattered points will be predicted as a single point in the middle, a photo of a dog will become a blurry mess, and a sentence will degenerate into repeating the word "the" endlessly. This is "mode blurring", which means the optimal solution is exactly the answer that least resembles real data.

How do existing models get around this? The answer is to break the "generation" process into pieces. Autoregressive models predict only one element at a time, diffusion models remove only a little noise at a time, and the target of each small step is split to retain almost only a single mode, so the reconstruction loss will no longer take the average.

This "splitting of the generation process" is exactly why diffusion and autoregressive models can produce high-quality samples, but it also prevents the model from being end-to-end.

The authors thus raise a key question: a generative model has only two things that can be split: how to generate, and how to train.

Since splitting the generation path will destroy the end-to-end property, why not split the training process instead?

A For Loop: The Entire Core of Explorative Modeling

What Explorative Modeling splits is the training loop itself.

Its mechanism can be explained in one sentence: at each training step, the model no longer generates only one sample to forcefully match the target, but generates K candidates, then selects only the one closest to the real data for training and backpropagation of gradients. The paper implements this as a 3 to 5 line for loop, which is so simple that it is almost unbelievable (Algorithm 1).

Why can this solve mode blurring?

Let's take a real-life scenario: guessing the landing position of a dart. If you are only allowed to guess once, your optimal strategy is to guess the average position of all darts, but that is often a spot on the dartboard where few darts actually land. But if you are allowed to guess K times and only score based on the closest guess, the optimal strategy changes immediately: you will spread out these guesses, so that each one covers a different cluster of landing points.

The model works the same way. When it is allowed to explore K candidates, different input noises will each "claim" a different mode, instead of all crowding to the middle to take the average. The model can stably capture as many modes as the number of explorations.

The authors named this long-neglected capability generative expressivity, and pointed out that it is determined by the training objective itself. No matter how large the parameters and data are stacked, it will not increase on its own.

This also explains a strange phenomenon that has long been observed in the industry: why the best models today rely so heavily on "guidance" techniques.

The so-called classifier-free guidance essentially "pushes" the prediction away from the blurry average value. But if the model itself does not produce blurry outputs, why push it at all? The reason why guidance works is precisely the root cause left by mode blurring.

The paper also presents two exploration directions, Forward and Reverse: the former fixes a real target and searches for the closest one among its own generations, which tends to prioritize recall (covering all modes); the latter fixes a generation and searches for the closest one in real data, which tends to prioritize precision, and hardly increases computing overhead, at the cost of possibly collapsing to a small number of modes. The two are complementary and can be used in combination.

The Third Dimension: The Larger the Scale, the Greater the Gain

The most significant conclusion of this paper is verifying "exploration" as a real scaling dimension.

The authors added exploration to diffusion/flow models, Jumpy models and even masked diffusion language models, and observed consistent monotonic performance improvements across three modalities: image, video, and language. More critically, the trend of gains with scale becomes more prominent as the dimension is scaled up.

The figures given in the paper show that as the data scale grows, the gain brought by exploration rises from 7% to 36%; as the model size increases, the gain rises from 13% to 23%; when the computing power triples, the efficiency gain more than doubles.

In terms of specific efficiency, exploration improves FLOP efficiency by 4.1 times, sample efficiency by 6.2 times, and parameter efficiency by 47%. In image generation, it further pushes the current state-of-the-art RAE formulation to an unguided FID of 1.43 on ImageNet, approaching the best level in the industry.

A large model that explores 5 modes can even outperform an XLarge model with 47% more parameters that does not perform exploration.

The implication of this trend is significant. The authors explain that at small scales, the model is mainly constrained by parameters and data, and generative expressivity is not yet the bottleneck; but once parameters and data are scaled to the point where they are no longer limiting factors, generative expressivity will increasingly become the real bottleneck, which is exactly what exploration can directly amplify.

Considering that the training of real foundation models today uses computing power about four orders of magnitude higher than the largest experiment in this paper, the authors believe that the numbers reported in the paper are very likely only the lower bound of the gains at larger scales.

Truly End-to-End Generation

What will happen if exploration is pushed to the limit? The answer returns to the suspense at the beginning: generative models can finally be end-to-end.

The authors used XM as an independent end-to-end model for robot control tasks. On Behavior Cloning, their Explorative Policy achieved and even exceeded the performance of Diffusion Policy, which requires 100 forward passes, with only one single network forward pass; on goal-oriented world modeling, the Explorative World Model achieved better average performance with 16 to 256 times less inference computing power than Diffuser.

The source of this gap is clear: diffusion models use hundreds of generation steps during inference to obtain expressivity, while end-to-end XM moves this cost to exploration during training, so that inference only requires one forward pass. For the same task of "handling multiple modes", you can either split it slowly during inference, or complete all explorations at once during training; this paper chooses the latter approach.

About the Author

The first author of this paper, Alexi Gladstone, is no stranger to the industry. Back in July 2025, his led work Energy-Based Transformers (EBT) sparked extensive discussions on social platforms.