96 robots evolve together, with their success rate surpassing that of humans. Tsinghua University, Agibot and InfinityCore jointly propose "Collective Experiential Learning"

Every execution of robots in the real world generates experience. How to make use of these experiences has long been a tricky problem. The traditional approach relies on manual correction: remote operators monitor robot execution, take over when the policy makes mistakes, and give correct actions. This mode can be maintained for a small number of robots, but as the scale of the cluster expands, manual supervision will fail to keep up with the growing demand sooner or later.

To solve this problem, the joint team of Tsinghua University, AgiBot and Wuyuan Core Qiong proposes Collective Experience Learning (CEL), a learning paradigm that allows deployed robots to share execution experiences and jointly improve the same policy.

Paper Link: https://arxiv.org/abs/2610.25736

The algorithm implementation of CEL is Failure-guided Experience Retrieval Fine-tuning (FERT): using the context when the robot fails to execute, retrieve relevant trajectory segments from the successful experiences shared by the entire cluster, and use the recorded actions in them as supervision signals to update the policy. The whole process does not require additional collection of expert demonstrations or corrective actions. Humans only need to label each execution as success or failure and reset the environment, and all actions are provided by the robot itself.

Figure | Three experience sources for robot policy learning: human demonstration, expert correction and cluster collective experience

To support continuous learning on large-scale clusters, the team also built a Fault-tolerant Asynchronous Training System (FATS). In this way, even if individual robots in the cluster are interrupted due to failure, reset or maintenance, experience collection, policy optimization and policy distribution can continue to proceed.

On five real-world operation tasks including mobile phone packaging, flower arrangement, and clothes folding, the research team conducted 120 trials for each task. None of the robots participating in the evaluation were involved in training data collection, and the success rate of the final policy reached 93.3% to 99.2%. When the collection time of each robot is fixed and only the number of robots is increased, the larger the cluster and the more experiences, the better the performance of FERT in general. In the multi-stage mobile phone packaging task, after learning with 96 robots, the task success rate increased from 79.2% to 98.3%.

These results show that collective experience can be a feasible and scalable foundation for robot learning: as the deployment scale expands, the supply of experience grows synchronously, without the need for expert supervision to expand proportionally. This provides a path for robots to continuously improve policy reliability during deployment.

1. Research Methodology

The algorithm and system implementation of CEL are FERT and FATS respectively.

The core idea of FERT can be summarized in one sentence: success provides the imitation object, and failure determines what to learn. The specific process is divided into the following four steps:

• Step 1: Asynchronous experience collection. Robots in the cluster continuously and autonomously execute tasks, and each completed trajectory is labeled with a binary result of success or failure. Successful trajectories enter the collective experience buffer as learning materials; failed trajectories do not provide actions and are only used as retrieval clues. This ensures that the system can learn from failures without imitating failed actions.
• Step 2: Experience encoding. Normalize the output (after spatial pooling) of the frozen VLA visual encoder and the joint state of proprioception respectively, and then splice them to form a unified retrieval representation, so that experiences from different robots can be compared with each other.
• Step 3: Failure-guided retrieval, which is the key of this method. The binary label can only indicate that a trajectory has failed, but cannot locate where the failure occurs. FERT takes the observation sequence about 3 seconds before the failure end point as a query, and finds time-aligned and state-similar successful segments in the successful experience library through subsequence dynamic time warping with slope constraint (S-DTW). Intuitively, it answers the question "how to succeed in similar situations".
• Step 4: Priority fine-tuning. Segments hit by retrieval get higher sampling priority, and repeatedly occurring failure situations will make relevant successful behaviors be learned more frequently; at the same time, the priority decays exponentially over time, ensuring that learning always focuses on the current most failure-prone links of the policy. Policy update adopts log-likelihood fitting of action blocks, without the need for value function, critic or advantage estimation. Theoretically, the team proved that under the conditions of successful replay and retrieval alignment, the update direction of FERT is consistent with the goal of improving task success rate, and retrieval weighting brings a first-order positive contribution.

Figure | Overall process of FERT

To run this FERT learning loop on a real cluster, the engineering difficulty is no less than that of the algorithm itself. The team points out two key features: First, the timing requirements for interaction and experience are different. The robot must receive the inference result before the next action (millisecond level), while training requires complete episode records (GB level, which do not need to arrive immediately). Second, physical robots may be interrupted at any time. In the actual operation of 96 robots, one robot fails on average every 22 minutes.

Figure | FATS Architecture

To solve these two problems, FATS adopts a dual-plane architecture.

Among them, the data plane SwarmLink connects robot execution to shared cloud inference and training. Inference requests are aggregated into a public request pool, and suspended robots automatically release computing power; episode data is uploaded in streams during execution, and observations that have been used for inference are directly reused in the cloud to avoid repeated transmission; interactive traffic has priority over batch training data.

The control plane HotSwarm models the cluster as dynamic membership. The life cycle of "station", which refers to the fixed position binding nodes, devices and robots, is separated from that of "worker", which refers to the process occupying the station. The station runs through the whole training process, and the worker can be replaced at any time. When a robot fails, the system restarts its worker locally, rebuilds the connection with a new version number, and replays the interrupted call, while other robots are not affected; expired actions calculated for failed workers will be identified and discarded to prevent data with mismatched actions and observations from polluting training. The recovery strategy is also designed in layers: environment workers can be restarted locally at any time, with a maximum cost of one episode; training workers are archived periodically and recovered from the latest checkpoint, so that the optimizer state accumulated over several hours can be preserved.

2. Experimental Results

The experiment covers five real desktop operation tasks: flower arrangement (flexible objects), clothes folding (deformable fabric), mobile phone charging (plugging and unplugging with precise alignment), stool assembly (small objects and tool use), and mobile phone packaging (multi-stage placement, taking 2-3 minutes). The hardware is the dual-arm AgiBot G2 robot, and the base model is π0.

At the maximum cluster size of each task, multi-robot HG-DAgger increases the average success rate of the five tasks from 36.3% of SFT to 81.0%; without using any additional expert data, FERT further increases the average value to 97.3%, and the average failure rate is relatively reduced by 86.0%. Among them, the success rate of mobile phone charging increases from 60.0% to 97.5%, and the relative failure rate decreases by 93.8%; the success rate of mobile phone packaging increases from 79.2% to 98.3%.

Figure | Main results of the five tasks

FERT outperforms human teleoperation on all five tasks, with an average lead of 16.7 percentage points, and a lead of 25.8 percentage points on mobile phone charging; it also comprehensively outperforms the two baselines: single-robot online reinforcement learning RLT (average lead of 12.3 percentage points) and offline reinforcement learning RECAP which also uses cluster data but retains expert correction (average lead of 11.5 percentage points).

When the collection time of a single robot is fixed and only the number of robots is increased, the success rate of FERT continues to rise in general across the five tasks: in the mobile phone packaging task, when the training cluster expands from 16 to 96 robots, the success rate rises from 86.7% to 98.3%. In contrast, HG-DAgger that relies on manual intervention saturates or even drops after the cluster expands, and the limited human monitoring and intervention capacity constitutes the performance upper limit. The experience supply of FERT grows naturally with the expansion of cluster scale, so the scaling curve of "the more robots, the better the learning" is valid. It is worth noting that FERT may be lower than the initial policy in small clusters, because the successful experiences are not enough to cover various failure scenarios encountered by the policy.

Figure | Scaling with cluster size

Finally, the research team tested the contribution of each design choice through ablation experiments. The ablation verification on the mobile phone packaging task (16 training robots, the success rate of the full configuration is 86.7%) verified the role of each design. After removing the failure-guided retrieval, the success rate drops to 64.2%, a decrease of 22.5 percentage points, which is the largest drop among all ablation items, directly verifying the value of "selecting experiences according to failure scenarios"; removing the action context expansion before and after the retrieved segments, the success rate drops to 57.5%, indicating that the action context before and after the matched segments is equally important for learning successful behaviors. The optimal policy synchronization frequency is every 500 optimizer steps: too sparse will delay the use of new policies, while too dense will lead to insufficient accumulated experiences under each policy version; the optimal priority retention factor is the medium value 0.88, which reflects the trade-off between retaining historical retrieval information and responding to the latest failures.

Figure | Results of ablation experiments

At the system level, 96 robots collected a total of 9,834 episodes in nearly 8 hours of operation, during which 30 failures only caused 1.3% of robot time loss. For comparison, if synchronous collection is adopted, the collection volume will only be 33% of the measured value; if the whole machine is restarted for 5 minutes after each failure, only 86% of the time can be preserved. The value of the fault-tolerant asynchronous design is thus quantified.

The research team also pointed out that the evaluation is limited to five task-specific policies on a single platform (AgiBot G2). Broader task distributions, heterogeneous robot platforms, and adaptation to environmental changes in longer operation cycles are left for follow-up work.

Overall, CEL brings a new deployment logic. Robot clusters not only contribute to task execution, but also generate experiences that can be learned and improved by the entire cluster. Through a shared policy, the experience of a single robot can be transformed into the capability of the entire cluster. As the deployment scale expands, the source of experience grows synchronously. Therefore, the more robots there are, not only the greater the production capacity, but also the better they can learn, without being restricted by the supply of expert supervision.

For more technical details, please refer to the original paper.

This article is from the WeChat official account "Academic Headlines" (ID: SciTouTiao), written by Zhang Jiayi, and published with authorization from 36Kr.