The most difficult robot demo this year has a "robot content" of 0.

In May this year, Jim Fan, head of NVIDIA's robotics division, declared in a public speech: VLA and teleoperation are dead.

At that time, the first reaction of the vast majority of people was: the AI industry, which never lacks radical claims, has added another new bold assertion.

Unexpectedly, a few months later, this statement that sounded a bit exaggerated back then has truly begun to come into reality.

Just now, X2Robot released its brand-new dexterous operating system TwinDex. With zero real-machine teleoperation data in the post-training phase, the robot continuously completed multiple fine operations in chemical experiment scenarios, including screwing bottle caps and injecting with syringes.

It is worth noting that in the past, to make robots learn such fine operations, real-machine teleoperation data was absolutely necessary for "hand-in-hand" teaching.

But this time, TwinDex completely eliminated this part of data requirement, and replaced it with only a few hundred pieces of agent-free data, with almost zero robot participation~

How is this achieved?

Teleoperation may really be in jeopardy

As the saying goes, a good Demo is worth savoring carefully.

In the Demo released by TwinDex this time, the most complete segment is a one-shot chemical experiment:

From opening the bottle to take samples, operating droppers and test tubes, to guiding flow with a glass rod, shaking evenly for observation, the whole process is executed autonomously by the strategy.

The entire experiment contains 24 sub-actions, spans three types of tools, involves multiple bimanual coordination and tool switching, and each step requires millimeter-level positioning and stable force control.

When broken down, the three fingers of TwinDex actually demonstrate several types of capabilities that were the most difficult to tackle in past dexterous operations.

First of all, it is fine operation with extremely high precision requirements.

When opening the toolbox, the robot needs to align the left and right index fingers with two narrow buckles at the same time, insert them accurately and then pry them down, then pinch the handle with the index finger and thumb to open the box cover.

The difficulty here lies in whether the robot's hand can accurately enter a space with very small tolerance to complete the operation.

Similarly, syringe injection follows the same logic.

The index finger and middle finger fix the barrel, and the thumb aligns with the push rod to apply force forward, which not only ensures that the position does not shift, but also accurately controls the injection direction and force.

As embodied intelligence gradually enters laboratories, factories and homes, such operations with small tolerance, high precision and strong contact will increasingly become the basic skills of robots.

Second, it is three-finger collaboration.

For example, in the broom and dustpan task, the three fingers form an envelope around the handle, allowing different fingers to share support and control together.

Compared with only two fingers, the third finger provides an additional contact point and another way to stabilize the object.

Especially in tasks such as tool use and long object grasping, this extra support point is quite critical.

The third category is flexible operation and in-hand adjustment that is closer to human hands.

When unscrewing the bottle cap, TwinDex pinches the cap with the index finger and thumb, and then completes the rotation by the lateral swing of the fingers themselves, hardly needing to turn the wrist greatly.

When turning pages of a book, the robot's thumb first rubs out the top book, then completes pinching, two-hand transfer, placing and page turning.

After watching all these demonstrations, I have a very intuitive feeling:

In the past, when dealing with such actions, people would often subconsciously think of five-finger dexterous hands with more complex structures, and massive real-machine teleoperation data.

But the Demo above just illustrates one thing: many fine operations that were previously considered to require five fingers can actually be done with three fingers.

And to some extent, these three fingers are already more skillful than many five-finger dexterous hands.

However, all the above mentioned are only the side of TwinDex standing in the foreground as the robot's actuator.

Before the robot actually starts to work, it has another identity, a set of wearable agent-free data collection system matching the execution end.

Compared with many solutions that use two different sets of hardware in the data collection and robot execution phases, TwinDex directly adopts the design of isomorphism of data collection and execution.

To put it in plain language, these three fingers are not only responsible for performing operations, the same structure is also used for data collection, which is the so-called "what you collect is what you get".

The actions made by the collector at the collection end can be more directly mapped to the robot execution end, without the need to cross a completely different set of hardware in the middle for complex data migration.

It is precisely because of this that those fine operations with extremely high requirements for fingertip position and force can be performed more stably by the robot.

At the same time, compared with directly teleoperating a bulky robot, collectors can complete actions more naturally and quickly, the data is easier to collect, and the cost is lower.

The experimental results have indeed verified these two points.

In terms of collection efficiency, the effective trajectories that TwinDex can produce per unit time are about 5.3 times that of traditional real-machine teleoperation.

With the same collector and the same period of time, TwinDex can quickly accumulate a batch of data that can be truly used for training.

In terms of data utilization efficiency, X2Robot found that:

As the amount of data increases, the policies trained with agent-free data and real-machine teleoperation data will continue to improve at the same rate, and finally converge to similar performance.

In other words, in this set of experiments, agent-free data can almost 100% replace real-machine teleoperation data when training the model.

Therefore, as a data collector, what TwinDex solves is not just "how to collect data faster".

More critically, while improving collection efficiency, it also minimizes the accuracy loss when these data are actually migrated to the robot.

Both of these two things point to the same problem in the end:

How to make agent-free data truly become data that robots can use.

TwinDEX: Align Embodied Data In Advance

Strictly speaking, TwinDEX is not just a three-finger dexterous hand.

It is more like a dexterous operating system composed of data collection hardware, robot end effector, data processing pipeline and model training process.

Moreover, simply making the data collection end and the robot execution end look the same cannot naturally complete data migration.

The truly key point of TwinDEX is to start from the goal of completing post-training only relying on agent-free data and eliminating real-machine teleoperation data, and then design the entire system in reverse.

From the configuration of the dexterous hand, the motion mode of the collection device, to visual observation, time synchronization, data processing, and finally model training, everything is actually carried out around the same problem:

How to make the collected data as close as possible to the data that the robot actually needs from the very beginning.

With this understood, let's look at the most intuitive hardware part of TwinDEX next.

Judging from the functional division in the actual Demo, the thumb and index finger undertake a large number of opposing, grasping, screwing and fine operations, and the third finger can further provide enveloping, support and stability.

The design of the entire TwinDEX can be summarized into three characteristics: dexterity, consistency and scalability.

First, it is the dexterity that has been mentioned repeatedly before.

TwinDEX did not choose the common two-finger gripper like UMI, nor did it pile up to five fingers completely imitating human hands, but took a compromise between dexterity, stability and engineering complexity.

As we all know, the more fingers there are and the closer the structure is to human hands, the higher the theoretical upper limit of dexterous operations will of course be.

But at the same time, more joints, drivers, sensors, and the subsequent calibration, control and maintenance costs are also increased.

Conversely, although the two-finger gripper is simple enough, it is difficult to cover fine operations such as screwing, in-hand adjustment, and multi-point contact.

Therefore, after basic grasping, in-situ screwing, tool use and in-hand operation and other meta-operation tests, and compared with a variety of candidate configurations, X2Robot finally found that:

Three fingers with nine degrees of freedom is the current sweet spot, which can cover the dexterity capabilities actually required by most current tasks with relatively controllable complexity.

Second, which is the most critical design idea of TwinDEX, consistency.

The consistency here means that the data collection end and the final execution end of TwinDEX are as consistent as possible in motion mode, contact mode, visual observation, collection accuracy and time synchronization.

This is also the key for TwinDEX to reduce the demand for real-machine teleoperation data.

In the past, although many agent-free data were easy to collect, when they were actually migrated to robots, they often encountered a problem: the collection end and the execution end are not the same agent.

How human hands move and how the collection device records may not be reproduced exactly on the real robot, and there are usually kinematic differences and accuracy losses in the middle.

Therefore, although this type of data is easy to scale, it is difficult to directly use it to train the target robot. Most of the time, a batch of real-machine teleoperation data has to be supplemented in the end to complete agent alignment and post-training.

The consistent design of TwinDEX is intended to solve exactly this step.

It does not wait until the data is collected to find a way to adapt the data to the robot, but makes the collection end and the execution end as isomorphic as possible at the hardware design stage.

In other words, instead of waiting for the data to be collected and then trying to adapt it to the robot, we first align the two sides as much as possible when designing the collection device and the dexterous hand.

In other words, the "agent alignment" that originally occurred in the data processing stage is advanced to before the data is generated.

In this way, the loss of agent-free data when migrated to the robot can be minimized, which reduces the need for additional supplementary real-machine teleoperation data.

It is precisely because of this that in the tasks covered by this experiment, TwinDEX can complete post-training with only a few hundred pieces of agent-free data without adding new real-machine teleoperation demonstrations of the target robot, and perform the fine operations mentioned above.

The third characteristic is scalability.

The data collection end of TwinDEX is a wearable three-finger exoskeleton.

Its biggest advantage is that data collection does not need to occupy a real robot, nor does it need to be done around a fixed station.