Stop conducting blind AI research. Meta has launched the "AI Research Preference Model" (RPM): why not try small-scale pilots first?

AI Research Agent (AIRA) is already capable of autonomously proposing, implementing and evaluating machine learning experiments, but its progress on cutting-edge tasks has always been constrained by costs: drafting a candidate solution only takes a few minutes, while verifying its effectiveness may take hours or even days of GPU time.

Nowadays, the number of candidate solutions that agents can propose far exceeds the number that they can afford to execute, which makes the research preference of "how to allocate computing power under a fixed execution budget" a key factor that determines the progress of research.

To address this challenge, the Meta FAIR team and its collaborators proposed "AI Research Preference Models" (RPMs), a model that can predict which of multiple candidate solutions is most worthy of investing computing power to execute without actually running the candidate solutions.

Paper link: https://arxiv.org/pdf/2608.13940

The research team observed that language models are not reliable when predicting absolute metrics or execution results. Therefore, RPMs will not be used to guess what final score a solution can achieve, but to reformulate the problem as a relative ranking problem: rank candidate solutions before actually investing computing power to find the most promising path.

Specifically, they first built two different forms of RPM based on a frozen pre-trained language model (without task-specific training): one inference-only model that reasons about candidate plans, codes and previously executed solutions, and one agentic model that performs small-scale trials before making subsequent decisions; then they integrated these two types of models into an AIRA-dojo search agent.

In the research of Agentic RPMs, the research team found that the pilot experiment agent tends to be overly conservative when allocating time budgets, resulting in a large amount of idle budget. To this end, they adopted two mechanisms: one is to intentionally exaggerate the remaining time budget in the prompt (the reported value is several times the actual value) to prevent the agent from stopping prematurely; the other is to introduce an independent feedback model after each submission to review the existing findings, propose the most informative experiment for the next step, or judge that the evidence is sufficient and terminate the loop.

The evaluation results on AIRS-Bench show that: the two schemes increase the average normalized score from 0.684 to 0.711 and 0.729 respectively, and can reach the level that the unguided agent can reach in 24 hours in about 15 hours, with less than 2/3 of the execution budget of the latter. It is worth mentioning that their optimal RPM model also achieved SOTA results on two AIRS-Bench tasks.

Figure | Evaluation results of RPM-enhanced AIRA-dojo

Of course, this study also has certain limitations. For example, the current experiment assumes that the LLM inference cost is negligible, but there is actual latency — taking the Inference-only RPM as an example, after deducting 0.660 hours of inference latency, the normalized score drops slightly from 0.711 to 0.708, and the specific overhead varies depending on the hosting infrastructure and model scale; the offline evaluation data comes from previous greedy search runs of different backbone models and different task modalities, which is off-policy data, so the main conclusions of the research paper are still based on end-to-end results.

Moreover, the research team only limited the integration scope of RPM to the child node creation stage. Preliminary experiments show that since the Hidden Consistent Evaluation protocol has ensured that the validation set is highly consistent with the test set, the application of RPM in final node selection has no significant improvement compared with the default strategy of "selecting the node with the highest validation score"; the application in the parent node selection stage is left for future work.

In addition, RPM has only been verified under the AIRA-dojo framework, using a single backbone model and on a single benchmark, and its transferability on other frameworks and backbone models needs to be verified in subsequent work.

In the future, the next stage of competition for AI research agents may not only lie in whether they can propose more potential solutions, but also in whether they can more accurately determine which solutions are worthy of investing research resources.

RPM provides a direct and systematic path: replacing fragile absolute prediction with relative comparison, supplementing judgment context with historical trajectories, and obtaining more evidence through small-scale experiments when necessary. When experimental evaluation is far more expensive than solution generation, investing test-time computing in candidate selection itself may become an important direction to improve the efficiency of automated research.

This article is from the WeChat official account "Academic Headlines" (ID: SciTouTiao), author: Academic Headlines, published by 36Kr with authorization.