Zhizai Wujie and Peking University Team Achieve New Breakthrough: Endowing Robots with Human-like Tactile Sensation?

「StarLink Capital」 is a venture capital fund focusing on the large model ecosystem, emphasizing early - stage investments and connecting industries.

When robots can “see” the world, we begin to wonder: When will they be able to “touch” and “understand” the world?

Tactile sense is the cornerstone of human dexterous manipulation, but it is a huge gap for robots to achieve true intelligence. Collecting tactile data for robots is time - consuming and labor - intensive, and the vast amount of human tactile experience is blocked on the other side due to morphological differences.

Today, a bridge has been built — UniTacHand, a breakthrough research from BeingBeyond and a team from Peking University, announces that with only 10 minutes of human - robot pairing data, it can achieve “zero - shot” lossless transfer of human tactile skills to multi - fingered dexterous hands.

This means that the “fingertips” of robots are about to truly feel the texture and force of the world for the first time. The Forbes China 30 Under 30 list aims to discover and recognize young talents under 30 in China who have demonstrated outstanding leadership, innovation, and industry influence in different fields. With its strict evaluation criteria and forward - looking vision, this list has become an important benchmark for measuring young entrepreneurs and industry changemakers.

01 The Key to Breaking the Deadlock

Current robot tactile research is deeply trapped in the dual dilemmas of the “data desert” and the “morphological gap”.

The breakthrough of UniTacHand starts from a core insight: Putting aside morphological differences, the physical logic of human and dexterous hand manipulation of objects is essentially the same.

The research team creatively uses the UV mapping of the MANO hand model as a “universal language”. Whether the data comes from human tactile gloves or dexterous hand sensors, it is uniformly “translated” onto this standard two - dimensional tactile map, thus smoothing out the differences in hardware and morphology.

Merely mapping the data to a unified space is not enough to achieve true tactile transfer.

A key problem lies in: Even when humans and machines perform the same task, their operation strategies are often different. For example, when a human hand grasps an object, it tends to wrap the whole palm to increase stability, while dexterous hands such as Inspire often use the fingertips to pinch.

This results in essential differences in the activation areas of tactile signals and pressure distribution patterns when they contact the object.

UniTacHand designs a core cross - domain contrastive learning framework. Its goal is not to force the data to correspond one - to - one in space, but to teach the model to understand the physical semantics and task intentions behind touch.

This framework uses a dual - branch encoder architecture to process human and robot data respectively. Each branch contains a tactile encoder and a hand pose encoder to ensure that the model understands both “what the tactile sense is” and “what the hand is doing”, and is collaboratively optimized with a carefully designed triple loss function.

Through this process, the model gradually learns to map “the comprehensive pressure distribution of the human palm” and “the concentrated force signals of dexterous fingertips” into high - dimensional features with the same semantics in the shared latent space it constructs internally.

At this point, tactile transfer is no longer a simple signal conversion, but rises to a cross - morphological tactile semantic understanding. Robots can not only “see” the tactile images in a unified format, but also “read” the physical effects and operation purposes contained in them, thus achieving true perceptual alignment and skill inheritance.

02 Real - Machine Verification

To systematically verify the effectiveness of the UniTacHand framework, the research team designed five representative tactile interaction tasks for testing on an integrated platform of an Inspire dexterous hand equipped with tactile sensors and a RealMan robotic arm.

01 Zero - Shot Transfer:

A model trained only with human data can be directly deployed without any robot tactile data.

In the object positioning task, the success rate reaches 100%, and in the soft and hard object classification and placement task, the success rate reaches 85%, significantly surpassing traditional baseline methods.

02 Transfer with a Small Amount of Paired Data:

In the compliance control task, the accuracy of the robot in understanding the force application direction through touch reaches 40%; in the task of classifying 10 unseen objects, the accuracy reaches 38.6%, showing excellent generalization ability.

03 Single - Sample Hybrid Training:

Combining only human data and one piece of real - machine data, the model has a success rate of up to 73.3% in the tactile discrimination task of visually confusing objects (such as empty and full water bottles), not only significantly surpassing the method trained with pure robot data, but also significantly better than other hybrid training baselines.

The results of these five experiments together show that the “human tactile sense → unified representation → robot skills” path constructed by UniTacHand has achieved important breakthroughs in data efficiency, transfer performance, and generalization ability.

Conclusion:

「BeingBeyond」 is committed to “promoting humanoid robots from the laboratory to daily life”. It is dedicated to building a general model framework for humanoid robots through human data and multi - modal large model technology, solving the core technical problems of embodied intelligence, and leading the technological revolution of humanoid robots.

Content source:

1. BeingBeyond “BeingBeyond's Latest Achievement: First Realization of Human - Robot Tactile Data Transfer, Addressing the Pain Point of Difficult Tactile Data Collection for Dexterous Hands”

This article is from the WeChat official account “StarLink Capital”, author: StarLink Capital, published by 36Kr with authorization.