New Breakthrough in RSI: Teams from UC Berkeley and Sakana AI Propose "Multi-Agent Self-Supervised" MASS

Improved sphere packing bounds, new gluon scattering formulas, and experimentally validated nanobody designs... Cutting-edge AI systems have begun to produce original achievements in scientific research.

How can this process be sustained and research efficiency further improved? Recursive Self-Improvement (RSI) is regarded as a direction with great potential.

RSI is primarily driven by two core components: evaluation and improvement. However, both links face practical difficulties in cutting-edge scientific research: on the evaluation side, if research outputs exceed the judgment capability of existing human experts, or experimental feedback requires a very long waiting period, it is difficult to obtain reliable evaluation results; on the improvement side, models need to learn from high-quality execution trajectories, and the quality of these trajectories depends on whether the workflow is reasonable. Adjusting task decomposition, tool usage, result verification and other links one by one for different tasks and fields incurs extremely high labor costs and is difficult to generalize.

The problem also lies in that models often need to rely on themselves to judge and improve their own performance. Task generation, result evaluation and workflow adjustment are all built on the existing knowledge and judgment of the model. If these judgments are biased in the first place, self-feedback may cause errors to persist continuously.

To address this issue, the University of California, Berkeley and the Sakana AI team proposed "Multi-Agent Self-Supervision" (MASS), which uses multi-agent coordination to generate self-supervision for RSI.

Paper link: https://arxiv.org/pdf/2610.12176

Specifically, they let the same model alternately act as the task-solving agent, workflow optimizer and evaluator, first optimize "how multiple copies of itself collaborate" under the premise of frozen weights, and then distill the trajectories generated by collaboration back to the shared weights.

After two rounds of MASS cycles, Qwen3.6-27B achieved a win rate against external strong model judges on three test tasks rising from 53.9% to 69.9%, and the score per output Token on four public scientific research benchmarks reached 1.2–1.6 times that of the base model.

In addition, they also found that the optimized workflow will arrange information transmission and agent routing according to the characteristics of tasks. In the first round of experiments, the multi-agent student model achieved a 68.3% win rate over the base model. As a control, the single-agent student model was trained with approximately 1.4 times more processed supervised Tokens, with a win rate of 64.0%. This indicates that joint learning orchestration and bounded sub-agent execution can provide more effective supervision for RSI.

These results show that optimizing the collaboration mode between agents and letting each sub-agent perform work within a clear task scope helps improve the quality of training data and provide more effective supervision signals for the continuous improvement of the model.

1. Research Methodology

MASS first stimulates the collective intelligence of the multi-agent team by optimizing the workflow, and then distills the team experience into the shared model. The inner loop optimizes the team workflow while keeping the model weights fixed, and the model itself evaluates and judges which workflow is optimal; the outer loop updates the weights by learning the execution of the optimized workflow. The following animation demonstrates this process:

Video | MASS animation demonstration. Workflow search changes the collaboration mode between agents; training based on the selected execution trajectories will precipitate these experiences into the shared weights; the updated model immediately takes on all three roles again.

1. Preparation Phase: Workflow as Prompt

A workflow is a written plan on how the multi-agent team collaborates. The orchestrator is responsible for assigning work to subagents and integrating results. This plan will be appended to the task Prompt, so that the organization mode of the team can be inspected and modified by the model, including four parts:

• Role: The responsibilities assigned to each subagent, such as preparing data.
• Instruction: The work that the subagent should perform.
• Contract: The information and artifacts it must return, including any required checks.
• Hop: The call order of subagents, including rollback to previous steps.

If historical decisions use future information, a backtest can seem very convincing. This contract requires data specialists to check for such errors and report whether the time segmentation is valid. It also tells subsequent agents what evidence they should receive before proceeding. Thus, part of the research plan is expressed as "what agents must establish and what they must pass to each other".

2. Inner Loop: Search for Better Workflows

For each training task, MASS retains a champion: the best workflow found so far according to the judgment of the current model. Each iteration has three steps:

• Execute: The team executes according to the candidate workflow and produces a workspace.
• Evaluate: The evaluator compares this workspace with the champion's workspace and explains which one is better. The winning candidate becomes the new champion.
• Revise: The optimizer uses the feedback history to propose the next workflow.

The evaluator judges the completed work; the optimizer uses this judgment to revise the process that produced this work. The optimizer's system Prompt emphasizes the contract and call order (hop), guiding workflow search to "what information is exchanged between agents and when the information is received".

3. Outer Loop: Learn from the Selected Team

Even with the optimized workflow, the quality of long-term research work may still fluctuate between runs. Therefore, MASS executes the self-assessed optimal workflow (i.e. the champion workflow, referring to the optimal workflow found so far) with multiple random seeds, ranks the generated workspaces using the current model, and retains the top-ranked trajectories. Each trajectory records the messages, tool calls and tool results of the orchestrator and its sub-agents.

• Run the champion workflow 18 times for each task.
• Compare and rank the generated workspaces using the current model.
• Retain up to 16 trajectories for each task: 15 for training and 1 for validation.
• When constructing training samples, remove the workflow text from the orchestrator's initial Prompt.
• Fine-tune the shared model on the dialogues between the orchestrator and sub-agents using LoRA. The output of the model is the training target; the task Prompt and tool results provide context.

Removing the workflow from the orchestrator's training Prompt can prompt the model to learn to organize work starting from the task itself.

2. Experimental Results

The research team used Qwen3.6-27B as the base model and ran two rounds of MASS in the qwen-code programming agent environment. The experiment used 12 synthetic research tasks covering the finance, robotics and pharmaceutical fields, each requiring delivery of code, quantitative results and research reports, of which 9 were used for training and 3 were reserved for testing. GPT-5.5 and Claude Opus 4.8 served as external judges, whose judgments did not participate in any link of MASS.

On the reserved test tasks, the model's win rate against the base model rose from 53.9% in the first round to 69.9% in the second round, and the win rate of the second round against the first round was 60.8%.

On 6 public benchmarks, after two rounds of training, the score per output Token of the model compared to the base model is: 1.60 times on MLR-Bench, 1.35 times on DSBench, 1.28 times on AstaBench E2E-Hard, and 1.20 times on ScienceAgentBench. In contrast, Terminal-Bench 2.0 and SWE-bench Verified reached 0.96 times and 0.94 times respectively, indicating that the improvement is not balanced across different tasks. The research team speculates that adding software engineering tasks to the training data may improve performance on these two benchmarks.

It is worth noting that the training data only contains task-solving trajectories, but the capabilities of the two roles of workflow optimization and self-assessment are also improved simultaneously:

• Self-assessment: The consistency rate between the evaluator and external judges rose from 0.73 to 0.91, and then to 0.93.
• Workflow optimization: Even when executed by the base model, the workflow generated by the first-round model outperformed the workflow generated by the base model itself in 64% of the comparisons.
• Search speed: The trained model can find effective workflows earlier.

Multi-agent trajectories also have higher training efficiency. M+ (the first-round MASS model) processed 28.8M supervised Tokens and achieved a 68.3% win rate against the base model; S++ trained with single-agent trajectories processed 42M supervised Tokens and had a 64.0% win rate. The former used 31% less supervised Tokens while achieving a higher win rate. This Token count does not include the overhead of data generation, workflow search and evaluation.

In the comparative experiment between the optimizer and the evaluator, the executor was fixed as the base model. After both the evaluator and the optimizer were replaced with GPT-5.5, workflows better than the no-workflow base model were found for all 12 tasks within 5 iterations; when only the evaluator was replaced, the benefit was significantly smaller. The homogeneous closed loop where all roles are served by the base model is slower, but it also covered 11 tasks by the 13th iteration. The research team believes that when the budget is limited, priority should be given to upgrading the optimizer.

The above results show that MASS can provide effective self-supervision for RSI. The research team stated that how to accelerate the speed of performance improvement in each cycle remains an important direction for future work.

This article is from the WeChat official account "Academic Headline" (ID: SciTouTiao), author: Academic Headline, and is published by 36Kr with authorization.