Figure rented 30 apartments and deployed robots to conduct "housework assessment" at each household one by one.

On September 17 local time, humanoid robotics company Figure released its neural network model Helix 2.5, and announced a set of zero-shot full-body autonomous experiment results: without any new data collection and environment adaptation, the zero-shot success rate of the robot has skyrocketed from 9% to 56%.

Brett Adcock, founder and CEO of Figure, said that the company has rented 30 residences in the San Francisco Bay Area to let Helix 2.5 directly perform tasks without collecting new data, additional training, or environment adaptation. What Figure wants to answer is: can a robot walk into a home it has never seen before and start doing housework directly?

The official blog calls Helix 2.5 the most advanced neural network Figure has developed to date, aiming to verify whether humanoid robots can independently complete long-term tasks in unadjusted environments only with existing knowledge.

Humans can walk into an unfamiliar house and start working immediately because their cognition of the physical world has the cross-environment transfer ability, and there is no need to re-learn due to changes in bed height or furniture layout. Traditional robots usually rely on collecting data in specific environments and conducting targeted training. Figure hopes to break this paradigm: first let the robot learn from large-scale human behavior experience, and then directly transfer its general capabilities to unfamiliar real environments.

01 Rent 30 houses, no data collection, work directly

In the official demonstration, the Figure 03 humanoid robot entered a completely unfamiliar house and continuously completed three long-term full-body tasks: tidying the living room, making the bed, and folding towels.

Helix 2.5 completes three full-body mobile manipulation tasks in real homes: tidying the living room, moving through narrow spaces to make the bed, and folding towels

When tidying the living room, the robot picks up the toys scattered everywhere one by one and puts them into the basket. Adcock emphasized in related videos and posts that the robot had never entered these rooms before, and the company did not collect data for these new environments. The robot needs to move while searching for target objects, adjust its body posture, and then complete grasping and placing.

The bed-making task is more difficult. The robot has never entered this room, never seen this bed, and never seen these bedding items. It must grab the pillow, pull the quilt, move the relevant objects to the designated position, and smooth the quilt. Figure uses the towel-folding task to test the ability to operate deformable objects: the robot needs to grab a towel it has never seen before, complete the folding, and put it into the basket.

These three tasks combine active perception, movement and manipulation in different unfamiliar homes

Figure stated that Helix 2.5 is the most advanced neural network the company has ever built. A base model pre-trained with Index is used for three different tasks, covering movement, manipulation of rigid and deformable objects, two-handed coordination, and active perception.

These tasks are selected because they require solving multiple classic robot problems at the same time: perception, movement, manipulation, two-handed coordination and full-body control, and all must be completed in real time. Most robot generalization studies are built in environments constructed around robots. For example, desktop robotic arms are limited to fixed working spaces, and wheeled robots need enough open ground for the chassis to move and turn. However, the home environment will not adapt to the robot. Humanoid robots must make their own bodies part of the task, moving, observing, reaching and manipulating objects in narrow and cluttered spaces.

There is another concept that needs special explanation: the "zero-shot" mentioned by Figure mainly refers to the evaluation environment and the manipulated objects. No data was collected for the 30 evaluation residences in advance, and the toys, towels and bedding used for evaluation did not appear in the task-specified data. The robot directly uses the sofas, beds and towels to be folded in each unfamiliar home to complete the tasks.

In other words, Helix 2.5 is not completely untrained for the tasks of "tidying the living room, making the bed, and folding towels". The three behaviors themselves have been adapted with task-specified data collected in other environments. What Figure really wants to verify is: can these learned behaviors be directly generalized to residences and objects that have never been seen before, without re-training on site.

The evaluation criteria are extremely strict: tidying the living room requires all 13 to 15 toys to be put away, folding towels requires complete folding and placing into the basket, and making the bed requires the pillows and quilt corners to be pulled to the top one-third of the bed and smoothed. The task requires end-to-end complete success, no partial scores are given; each task uses the same fixed model node (test version), and no model parameters are adjusted for the new environment; once a human safety intervention occurs during the process, the task is directly judged as failed.

Example of resetting the scene to uncontrolled initial conditions for zero-shot evaluation

Figure stated that it deployed Helix 2.5 to 30 residences in the Bay Area that it had never entered before. Before each test, the evaluation objects are placed in advance, and the AI model combined with manual review confirms that these objects do not appear in the task-specified data. Each test also uses an independent initial configuration, and the same reset conditions are applied to all evaluated strategies to ensure fair comparison.

At the test level, the key data given in the official blog is: the zero-shot success rate has increased from 9% to 56%, which is more than 6 times the original level. The "success" here refers to the completion of the full task, not part of it.

02 Index Pre-training, Human Experience is the "Breakthrough" for Robots

The key of Helix 2.5 is not only the model, but the Index behind it.

The logic of Figure is very simple: one of the important reasons why large language models can continue to expand their capabilities is that the Internet provides massive amounts of human data, enabling pre-training to continuously gain benefits as the scale of data and computing power expands. But robots do not have Internet-scale behavior data. Therefore, Figure tries to build Index, a large-scale human behavior dataset, so that robots can first learn from the experience of how humans act, grasp and manipulate objects in the real world.

In the previous introduction to Index, Figure positioned it as a global-scale human behavior dataset. Helix 2.5 further verifies: can these human experiences be transformed into real robot behavior capabilities.

Figure stated that currently Index can generate about 35 minutes of new human experience data per second. At the same time, the company has promised to invest 3.5 billion US dollars in computing power to train Helix.

The most critical question is: can these human experiences really be transformed into robot behaviors?

Figure conducted a set of controlled experiments: two models use exactly the same task-specified data, with consistent architecture, optimization methods, hyperparameters, downstream data and evaluation methods, and the only variable is whether Index pre-training is performed.

The experiment compared two models: one is a baseline model with randomly initialized weights from scratch, and the other is the Helix 2.5 model that is pre-trained with massive Index data first. Helix 2.5 itself starts completely from random weights and is pre-trained only with Index data, which is completely different from the previous generation Helix 02. Helix 02 was initialized based on existing pre-trained vision-language models.

Comparison of zero-shot success rates in 30 homes (model trained from scratch vs. model pre-trained based on Index)

The results are very obvious. The strategy without Index pre-training has a zero-shot success rate of only 9%; after adding Index pre-training, the success rate reaches 56%, more than 6 times that of the former. Since pre-training is the only experimental variable, Figure believes that this gap can directly measure the contribution of Index pre-training to zero-shot capabilities.

Moreover, the training data of Index is not specifically constructed around these three test tasks. Figure stated that none of the evaluation tasks accounts for more than 1.90% of the Index pre-training data. This means that the pre-training experience obtained by the model is not simply "repeatedly learning how to fold towels, make beds and put away toys", but trying to form transferable capabilities from a wider range of human behaviors.

Comparison between Helix 2.5 and Helix 02 that performs the same tasks

Data efficiency has also changed. Figure compared Helix 2.5 with a Helix 02 strategy that performs the same tasks: the latter uses data collected directly in the actual evaluation environment, while Helix 2.5 only needs half of the adaptation data to achieve a comparable success rate, and further generalizes the behavior to 30 residences that have never been seen before.

Figure therefore summarizes this result as: the cost of behavior specification is reduced by 2 times, and the deployment scope is expanded by 30 times.

Another noteworthy change is the full-body self-correction capability of Helix 2.5. In long-term tasks, the robot cannot complete every step at one time. If there is a deviation in grasping or folding, it may step back to reposition, change its standing posture, or even walk around to the other side of the bed to continue completing the task. Figure regards this ability to "recover and continue advancing after making mistakes" as one of the important effects brought by Index pre-training.

In the longer term, Figure also reported a "human-to-humanoid robot" transfer scaling law. The team trained four models on nested subsets of Index, with the scale of pre-training data gradually expanding, the maximum span reaching 8 times, while keeping the model scale and downstream training unchanged. The experimental results show that with each doubling of the amount of Index data, the loss value of the downstream robot's prediction of the next action shows a smooth and predictable downward trend.

Chart of the human-to-humanoid robot transfer scaling law

More critically, this downward trend shows extremely high predictability. Figure stated that based only on the test results obtained from training with a small amount of data, the test loss value of the maximum-scale model can be accurately predicted to four decimal places before the training of the maximum-scale model starts, and the prediction error only accounts for 0.54% of the full 8-fold data volume change range.

Figure claims that this is the first time that the "human-to-robot transfer scaling law" has been measured on humanoid robots. It means that the idea that previously mainly existed in the field of large language models, which is to predict the improvement of future model capabilities by expanding the scale of pre-training data, may also be extended to humanoid robots.

Of course, what is measured here is only the scaling relationship of data volume, and the model scale and downstream training remain unchanged. Therefore, it cannot directly prove that "the larger the robot model, the stronger it will definitely be", but it at least shows that in the current experiment, there is a predictable relationship between the increase of human experience data and the robot's motion prediction ability.

03 Redefine the boundary between "learning" and "deployment"

From Helix 02 to Helix 2.5, Figure is actually changing a premise: robots do not have to re-learn every time they enter a new place.

Helix 02 once demonstrated full-body long-term control capabilities, from unloading the dishwasher to autonomously running 200-hour logistics tasks, but the data learned by these systems came from the places where the robots will operate. Helix 2.5 tries to prove that full-body intelligence can be learned from human experience and transferred to new scenarios, instead of being re-established every time a new environment is entered.

This is also the real noteworthy difference between Helix 2.5 and previous robot systems.

The traditional data closed loop of robots is often "the robot enters the environment to collect data, trains the model, and then enters the environment". This method can continuously improve the performance in a single scenario, but if the robot eventually needs to enter thousands of households, it means that every family may become a new object of data collection and training.

What Figure wants to establish is another path: humans first accumulate experience in the real world, Index scales the experience, Helix 2.5 pre-trains from human experience, and the robot brings behavioral knowledge to unfamiliar environments.

If this model continues to hold, then the source of robot data will no longer be limited to the robot itself. Humans have accumulated millions of years of physical interaction experience in the real world, and what Figure wants to do is to convert part of it into training data that machines can learn.

Figure also clearly emphasized in the official blog that this does not mean that the general humanoid robot problem has been solved. It believes that Helix 2.5 only provides the first evidence: full-body intelligence can be learned from human experience and transferred to new scenarios without re-construction every time.

The conclusion given by Figure is very straightforward: if Helix 2.5 verifies this logic, then investing more data and computing power in the future will allow robots to master more cognition about the physical world before stepping into any new home.

As Adcock said at the end of the official video: "It's time to scale up." When pre-training draws on massive human experience and deployment is applied in unfamiliar real homes, the boundary of embodied intelligence is being redefined: robots do not have to re-learn every time they arrive at a new home, but learn from humans first, and then reach thousands of households.

This article is from the WeChat official account "Tencent Tech", the author is the person worth paying attention to, and it is released with authorization from 36Kr.