Prompt engineering paper featured at ICML 2026, sparking heated debate among netizens

In this day and age, can prompt engineering work get accepted at a top conference like ICML???

Recently, a user shared a newly ICML 2026 accepted paper on Reddit, and the post immediately went viral, with comment count surging rapidly.

Yet everyone was left confused, thinking: Does that even count?

The authors proposed no new optimization algorithms, nor trained any new large language models — they only did one single thing:

Modified the prompt.

The paper introduces a method called Verbalized Sampling (VS), which can significantly improve the diversity of LLM outputs and alleviate the long-standing Mode Collapse problem that has troubled large language models for a long time, solely by adjusting prompts.

It sounds quite practically valuable, but is it reasonable that a mere prompt trick could get accepted at such a top-tier conference?

Let's take a closer look at the paper first before making a judgment.

A Highly Controversial ICML Paper

Have you ever felt that AI outputs are becoming increasingly homogeneous these days?

If you ask an LLM "tell me a joke" ten times, the answers you get are often highly similar. This homogeneity is not only seen in creative tasks, but also in question answering, and even code generation scenarios...

This phenomenon is collectively referred to as Mode Collapse in the academic community.

Simply put, it means that models tend to output the most probable, safest, and most conventional classic answers, while rejecting unconventional and creative ideas.

In the past, to address this issue, most researchers would prioritize adjusting sampling parameters, modifying decoding algorithms, or retraining the model. However, this paper takes a different, unconventional approach: directly prompting the model to explicitly output its own sampling process alongside the final result.

For example, taking the earlier "tell me a joke" scenario, the authors modified the prompt to ask the model to:

Generate 5 distinct jokes, and assign a plausible probability value to each joke.

As a result, the model can produce far more diverse and less repetitive responses.

It sounds extremely simple, right? In fact, this is the core contribution of the paper — the Verbalized Sampling method. No fine-tuning is required; simply changing the way you phrase the prompt can drastically improve output diversity.

Even so, the authors carried out rigorous demonstrations and validations in the paper.

First, they investigated the root cause that leads to homogeneous model outputs.

Previously, the academic community attributed this problem to algorithm-level flaws, such as imperfect reward models or improperly set KL penalty terms. This paper digs deeper and argues that the real underlying cause lies in the preference data itself.

They proposed a concept called Typicality Bias. From the perspective of cognitive psychology, human annotators are inherently inclined to favor familiar, fluent, and conventional text, and will naturally assign higher scores to stereotypical, widely accepted responses during rating.

Therefore, even if the reward model and optimization algorithm are perfectly optimized, as long as the human preference data used for training carries inherent Typicality Bias, the aligned model will still suffer from Mode Collapse after alignment.

To verify this, the authors conducted repeated tests across five different preference datasets and various base models, and the conclusions remained consistent across all experiments.

Once they clarified this point, the authors realized that since the problem is rooted in the training data, they only need to design a prompt-based solution at the inference stage to correct it: by asking the model to output its full probability distribution in the prompt, they can awaken the diverse output distribution that the model originally possessed during the pre-training phase, and restore output diversity.

They then ran experiments on this method in various scenarios, and the results showed that in creative writing tasks, the output diversity is 1.6 to 2.1 times higher than that of standard prompts, without reducing the factual accuracy of the content or the safety level of the model.

Moreover, the more capable the model is, and the larger its parameter size, the more significant the diversity improvement brought by Verbalized Sampling will be.

So even though the final method presented in the paper is very simple, ICML still accepted the work.

Reddit Users Are Divided Over This Paper

However, the comments on the original post show polarized opinions on this paper.

Many users argued that ICML used to publish only hardcore innovations such as new models, new algorithms, and new theories, and work that only optimizes prompts and inference workflows cannot be considered legitimate machine learning research.

By comparison, the innovation of this work seems relatively weak, and there are several obvious problems:

First, the instruction-writing-like method is not original at all; some users even said they have been writing prompts like this the day before. Second, the theory is difficult to verify, because prompts often stop working when you switch to a different model, which is not as stable as a formal algorithm. Third, the experimental scale is limited, which is not sufficient to prove that this is a universal rule.

Some users directly compared the current situation in the machine learning field to the academic crisis that hit the psychology community more than a decade ago.

Back then, a large number of researchers had a weak foundation in statistics and misused statistical tools, leading to unreplicable conclusions in many papers and a severe industry-wide trust crisis. Now the machine learning community is also highly dependent on empirical experiments and lacks rigorous theoretical support.

The industry is fiercely chasing new methods, while there is a widespread culture of over-tuning parameters and gaming benchmark scores. Many so-called innovative algorithms have almost no practical deployment value compared with mature baseline models, and are packaged as innovative breakthroughs solely based on tiny metric improvements.

Essentially, all these issues are publication problems caused by unclear professional norms after the rapid expansion of the discipline.

But supporters argued that scientific research is not a competition to see whose method is more complicated. As long as the hypothesis is clear, experiments are sufficient, and results are stable and replicable, the work can absolutely qualify as excellent research.

For example, this paper fully explains what Mode Collapse is, and points out that the real problem lies in Typicality Bias — this insight is far more significant than the prompt trick itself.

One of the authors also replied in the comment section, stating that although the paper looks simple, it actually involves a large number of complex processing procedures.

The entire work includes complete problem tracing, new theoretical attribution, mathematical derivation, and multi-dimensional quantitative experiments, and is not a superficial prompt-tuning "salami-slicing" low-quality paper.

Many people also mentioned Chain-of-Thought, or CoT. When CoT first emerged, it was essentially just a single prompt line:

Let’s think step by step.

But now almost all reasoning methods can be traced back to CoT, which just goes to show that prompt engineering is no longer just writing simple prompts — it is becoming a new paradigm for studying model behaviors.

Over the past decade, machine learning research has almost entirely focused on the training phase, but now some inference-phase usage techniques are gradually moving to the core of machine learning research.

Perhaps in the coming years, we will see more and more such papers. They do not add a single line of new training code or a single new model parameter, yet they are still able to push the capability boundaries of large language models.

Research Team Introduction

Finally, let's take a look at the research team behind this work.

This research was completed by the team led by Weiyan Shi at Northeastern University, in collaboration with the Manning Lab at Stanford University and West Virginia University. Jiayi Zhang, Simon Yu, and Derek Chong are co-first authors of the paper.

Jiayi Zhang completed her undergraduate studies at the University of Michigan, earning three bachelor's degrees in Computer Science, Mathematics, and Linguistics, before pursuing her Master's degree in Computer Science at Northeastern University.

Her other paper accepted by the top NLP conference NAACL 2024, titled "Analyzing the Role of Semantic Representations in the Era of Large Language Models", also focuses on semantic representations and large language model research.

Simon Yu is currently pursuing his PhD at Northeastern University, with research focusing on alignment mechanisms and reinforcement learning in large language models. He completed his undergraduate and master's degrees at the University of Edinburgh, and has published multiple top conference papers.

In addition to this paper, his other work titled "Unsafer in Many Turns: Benchmarking and Defending Multi-Turn Safety Risks in Tool-Using Agents" was also accepted by ICML 2026.

Derek Chong holds a master's degree from Stanford University, and is currently a researcher at the Stanford University Artificial Intelligence Lab, with research interests focused on NLP for large language models.