Why is it getting increasingly difficult to measure the progress of large models?

Recently, mathematician Terence Tao published a rare open letter on his personal blog.

Also co-signing the letter are 24 other Fields Medalists, including Peter Scholze, Yuk Deng, and June Huh.

The mathematicians acknowledged that large language models have made notable progress in recent months, and have solved critical problems across multiple fields of mathematics.

Immediately afterwards, however, they issued a warning: AI companies using difficult mathematical problems as benchmarks for measuring model capabilities are harming mathematical research itself.

Terence Tao stated that this statement came from intensive discussions among 25 mathematicians over the course of a week. Normally, such academic initiatives require months of coordination. The hasty release this time precisely demonstrates that the situation has become sufficiently urgent.

Starting from mathematical research, this open letter points to a key issue in the development of large models:

When ranking scores fail to reflect model capabilities, how exactly should intelligence be measured?

Why have traditional evaluations become ineffective?

In early 2026, OpenAI's evaluators placed GPT-5.2 in an old-version open-source code repository, and asked it to fix programming errors inside. This test came from SWE-bench Verified.

The specific problem OpenAI encountered was somewhat unusual.

To pass the test, the model had to add a parameter with a specific name to the code. The problem description did not state this parameter name, and the code at that time did not provide sufficient clues. According to the problem design, it was almost impossible for the model to guess the answer correctly.

However, GPT-5.2 mentioned during the reasoning process that although the problem did not require the use of this parameter name, it "remembered" that a later version of the repository had added a similar parameter, so it could try to add it.

In this way, the model passed the test.

Evaluators soon realized the problem: at the time point set by the problem, that section of code did not exist yet. GPT-5.2 appeared to be fixing programming errors, but it might simply be recalling the later publicly released fix code from its training data.

To confirm this suspicion, OpenAI used a dedicated auditing AI to question the tested model about whether it had seen this problem before, whether it could reproduce the original fix code, and whether it knew the source of the problem.

Ultimately, the researchers found varying degrees of memory traces in OpenAI's own models, as well as cutting-edge models such as Claude and Gemini Flash. A few models could even state the serial number of the problem in the evaluation set.

This situation is commonly referred to as "benchmark contamination":

Test questions, answers, or highly similar materials have entered the model's training data. The final score measures not only problem-solving ability, but also the model's memory of existing answers.

Apart from contamination, some of the problems themselves are also flawed.

OpenAI once audited 138 difficult problems in SWE-bench Verified that models repeatedly failed to solve, and found that 59.4% of them had substantial errors.

Some tests that were used for scoring but not disclosed to the model checked functions that the problems never required; others only recognized the specific writing method of the original fix code, and other reasonable implementations might also be judged as incorrect.

On February 23, OpenAI announced that it would stop reporting SWE-bench Verified scores.

This is a highly symbolic decision. SWE-bench Verified was once screened and verified with the participation of OpenAI, and was once regarded as the gold standard for measuring AI programming capabilities. Today, the scores it provides can hardly correspond to the actual programming level of the models.

The emergence of agents has made evaluations even more difficult.

In the past, large models mainly answered questions in static interfaces. Now, agents can browse web pages, call tools, execute code, check the environment, and even analyze why they received this specific problem.

When Anthropic evaluated Claude Opus 4.6, it once asked it to participate in BrowseComp. This evaluation requires agents to use a browser to find information that is extremely difficult to retrieve.

On one problem, Claude conducted hundreds of searches across 12 languages, but still could not find a reliable answer. Then, it changed its investigation direction, and began to judge whether it was participating in a certain AI benchmark.

The model checked multiple sets of AI evaluation question banks one after another, and finally locked onto BrowseComp. Next, it found the public evaluation code, figured out the encryption method of the answer file, and wrote a decryption program by itself.

Since the browsing tool could not read the original file, it found a readable copy on the developer community Hugging Face, and finally decrypted the answers to all 1266 problems.

Evaluators wanted to measure web investigation capabilities, but what the model demonstrated was the ability to identify and bypass the evaluation mechanism.

This is exactly the most tricky part of agent evaluation. The stronger the model, the more tools it has, and the longer it runs, the more likely it is to find a success path that the problem designers did not anticipate.

At the same time, the cost of maintaining high-quality evaluations is also rising rapidly.

Nick Heiner, an engineer at Surge AI, estimated in a speech in August that a programming agent evaluation set requires approximately 1000 tasks, with an average of 60 engineer hours invested in each task.

Calculated based on the total cost of a software engineer at 500,000 US dollars per year, the initial production cost is close to 15 million US dollars.

These 1000 problems cannot be a one-time asset either. Heiner estimates that about one-third of them need to be eliminated and replaced every year, with maintenance costs of approximately 5 million US dollars.

High-quality evaluations are becoming an expensive, scarce, and continuously depreciating consumable.

Why are mathematical problems facing a "benchmark gaming" crisis?

After public evaluations gradually become saturated, mathematics seems to be an ideal new competition field.

In 2024, Google DeepMind announced that AlphaProof and AlphaGeometry 2 solved four problems in that year's International Mathematical Olympiad, scoring 28 points and reaching the silver medal level.

"International Mathematical Olympiad silver medal" is a highly communicative indicator. The public does not need to understand proof search, reinforcement learning, and formal systems to quickly judge how capable a model is.

A year later, Google announced that Gemini Deep Think had reached the International Mathematical Olympiad gold medal level. The model can directly read natural language problems and solve five problems within the 4.5-hour time limit of the official competition.

During the same period, OpenAI also announced that its model achieved gold medal-level performance on the IMO test questions of the same session.

Thus, a clear upward curve has formed for AI's mathematical capabilities: elementary school problems, AIME, International Mathematical Olympiad silver medal, gold medal. Continuing up along this curve, the only remaining goal is the mathematical scientific research itself.

Terence Tao said in an interview with *Nature* in May this year that AI companies have realized that their "most unambiguous successes will come from mathematics".

Mathematics is naturally suitable for quantification. Problem-solving results can be checked by programs, and formal proofs can be verified step by step by proof assistants such as Lean.

The model can try a large number of paths in parallel, and the scoring system does not need to understand the entire thinking process, only to confirm whether the final proof is valid.

When evaluations move from competition problems to open problems, the nature of the matter changes accordingly.

Open problems are not all century-level conundrums like the Riemann Hypothesis. They can also be unproven ideas: whether a certain object exists, whether a certain boundary can be further tightened, or whether a certain type of structure can be fully classified.

Such problems have one thing in common: humans have not yet mastered the recognized answers, nor do they have ready-made standards for comparison. The problem-solving process of AI has entered the production link of new knowledge.

Whether a result is valid needs to be checked by domain experts; whether the method is novel requires reviewing a large amount of literature; how to attribute the achievements may also involve years of unpublished accumulation of multiple researchers.

In the joint letter in September, 25 Fields Medalists targeted a forming competition mechanism: AI companies have begun to measure model capabilities by "how many unsolved problems have been solved", and announce the results according to the pace of product releases.

AI companies get proof of their model capabilities, while mathematicians have to undertake the subsequent verification, organization, and explanation work.

The attribution issue is even more acute.

Large models have been trained on papers, textbooks, lecture notes, code, and forum discussions. The "new ideas" they put forward may come from the recombination of multiple existing methods, or they may simply rediscover results from an obscure paper.

Formal tools can check whether a proof is valid, but they cannot determine where the idea came from, nor can they automatically decide who should receive authorship credit.

Therefore, the 25 mathematicians specifically mentioned in the statement that hasty releases will reduce the time for rigorous writing, method refinement, and literature verification, which will bring serious risks of attribution disputes and plagiarism.

From the perspective of mathematicians, the value of a problem lies far beyond the final answer.

Important problems often guide researchers to develop new concepts and tools. These methods are discussed, simplified, and taught, and eventually enter textbooks, becoming the foundation for the next generation of research.

Terence Tao refers to high-quality mathematical problems as a kind of "non-renewable resource".

What is truly scarce is "good problems": they are difficult enough, yet leave accessible paths. A field often needs years of exploration to gradually realize what problems are worth raising.

If ordinary benchmarks are gamed and broken, the credibility of the scores is lost. If mathematical problems are consumed as benchmarks, the price may fall on the "open problems" themselves, and more research that could have grown around them.

How should AI companies measure model capabilities?

A more practical issue is now placed in front of AI companies: if there is no unified test paper that can be used for a long time, how should model capabilities be measured?

Judging from the recent moves of OpenAI, Anthropic and Tencent, a new evaluation system is taking shape — evaluations are close to real work scenarios, and run through the entire lifecycle of models and products.

The first change is to shift evaluation questions from public, static lists to partially public, dynamically changing question banks.

For example, LiveBench regularly adds new problems from recent papers, news, competitions, and datasets. Programming evaluations have also begun to adopt code tasks that appear after the model's training cutoff date, or retain undisclosed test sets.

GDPval, launched by OpenAI, chooses to keep most of the evaluation tasks confidential.

This evaluation covers 44 professions including software development, law, finance, and healthcare, with a total of 1320 tasks. The model needs to produce real work outputs such as litigation summaries, engineering drawings, nursing plans, spreadsheets, and presentations.

Among the 1320 tasks, only 220 standard tasks reviewed by experts are publicly released, and the rest remain confidential to reduce the possibility that the model will be exposed to the problems in advance and receive targeted training.

The second change is to shift the evaluation focus from the number of answered questions to work outcomes.

GDPval not only checks whether the model's answers are correct, but also focuses on the quality of the final documents it delivers. Professionals will compare documents from AI and human experts without knowing which ones come from AI and which come from human experts.

Anthropic's summary of agent evaluations also follows a similar logic.

An agent may call tools dozens of times to modify files, databases, and web page statuses. Only checking the last paragraph of the reply can easily misjudge a failure as a success.

Therefore, Anthropic suggests prioritizing the inspection of what the AI has actually accomplished:

Whether the code can pass the tests, whether the records are modified correctly, whether the files are generated in the specified location, and whether the expected effect is achieved after calling the tool. When necessary, combine automatic program checks, scoring by other models, and manual reviews.

At Tencent, adjusting the evaluation system was also one of the early priorities for Shiao-Ping Yao to rebuild Hunyuan.

While the Hunyuan team was rebuilding the training infrastructure, they also shifted the evaluation focus from ranking scores to practical capabilities through self-constructed problems, manual evaluations, and product public testing.

On August 28, Tencent released and open-sourced Hy4 preview. In addition to public benchmarks such as Terminal-Bench and DeepSWE, Tencent also organized 163 internal experts to conduct blind tests on 203 engineering tasks.

These tasks come from real scenarios such as software engineering, gaming, finance, and security. The model needs to understand requirements, complete planning, debugging, and verification, and finally deliver code, documents, spreadsheets, or presentations.

The third change is to embed evaluations into the iterative closed loop of models and products.

Anthropic divides evaluations into two categories. One type looks for tasks that the model still cannot do well, to test its capability boundaries, which is called "capability evaluation"; the other type repeatedly checks tasks that it already knows how to do, to see if there is any regression after upgrades, which is called "regression evaluation".

The evaluation of Claude Code also went through a similar process. In the early stage, the team mainly relied on internal trials and user feedback for rapid iteration. As the product scale expanded, Anthropic gradually established automatic tests for specific behaviors such as file editing accuracy and answer conciseness, and added failure modes that appeared online back to the evaluation set.

Problems exposed after the model goes online are thus fed back to the evaluation system, which continues to promote the next round of optimization.

Tencent calls the idea of co-developing model training and products "Co-design". After Hy4 preview was connected to WorkBuddy, users found that it had problems such as excessively long thinking time and repeated self-verification in complex tasks.

On September 7, the Tencent Hunyuan and WorkBuddy teams completed special optimizations for Hy4 preview, and checked the effects using both benchmark indicators and manual evaluations. Tencent stated that while the task quality remains stable, the new version reduces the number of execution rounds, as well as the consumption of input and output tokens.

As a result, the industry is forming three common paths: reduce contamination with new and non-public problems, replace abstract Q&A with real work outputs, and continuously convert failures from products into evaluation tasks.

Evaluation is becoming an infrastructure for AI companies.

High-quality tasks require long-term accumulation, professionals need to participate continuously, and new problems that appear online must be continuously fed back. An evaluation system that can accurately identify problems relies on the shared understanding of users, products, and models within the organization.

As model capabilities continue to improve, the evaluation of model capabilities will also become part of the core competitiveness.

This article is from the WeChat official account "Deep Flow Institute", author: Wu Jiangfeng, published with authorization from 36Kr.