The Eve of the Robotic Tactile Sensation Boom: Five Technical Routes, Data Dilemmas and Model Rivalries

Dare you let a robot give you a massage? After trying it out at this year's WAIC, we found it surprisingly comfortable. What makes this massage robot stand out is that it is equipped with tactile sensors.

Jensen Huang once said that "tactile sense" is the last missing piece of embodied intelligence. Is that really true? Can robots become more powerful and smarter at work once they have tactile perception? If the answer is yes, why do most robots on the market still lack tactile capabilities? What are the different technical routes and conflicting views in the industry?

In this article, we will take you into the laboratory to explore the cutting-edge research of tactile sensors, and discuss how far the tactile technology of robots has developed today.

01

Definition and Significance of Tactile Sense

The first question is: why do robots need tactile sense? Let's start with a real case.

Among a pile of peanuts, there are several fake ones. If you can only look at them without touching, it is extremely difficult for humans to tell the difference. But a robotic hand equipped with tactile sensors can pick out all the fake peanuts accurately.

As long as you can touch them, humans can also quickly distinguish the fakes. Because the fake peanuts are made of stone, their surface is colder and smoother, they are heavier, and they are solid with no vibration from the real peanut kernels when shaken.

This is exactly the importance of tactile sense: there are many tasks in the real world that cannot be completed solely through vision.

Eric LI Zhiqiang

Founder and CEO of DeepSight Tech

Some experiments have been done in the industry: when a normal person's hand is given local anesthesia to temporarily paralyze the tactile sensory nerves, the human hand will fail to perform many operations, including twisting objects, grasping, and even buttoning clothes.

You may wonder: don't some robots without tactile sense already have very high success rates in many tasks? But industry insiders point out that a high success rate does not equal strong usability.

Tao YU

Director of Artificial Intelligence / Head of Dexterous AI, Analog Devices (US)

When using vision only, the robot can slow down, make a tiny 1mm adjustment each time, and finally complete the task after constant attempts. So if you only look at the success rate, it is actually a false proposition. In this case, force is extremely important. For example, how tight you should hold an object, whether two parts are properly engaged, and you cannot wait until the screw is worn out to realize you applied too much force when tightening it. When performing flexible operations, the softness, hardness and material of each thin line are also very critical.

What exactly is tactile sense? It sounds like a single type of perception, but if we break it down, we will find it contains two dimensions of information: one is the perception of "force" itself, and the other is the perception of "morphology".

Among them, force perception is further divided into static force and dynamic force. Static force is the continuous pressure exerted by the fingers when grasping an object, which tells you: are you holding tight enough? How hard is the object? Dynamic force is the high-frequency signal generated when the object slides, which tells you: is the object about to slip? Is the friction sufficient?

Morphological perception refers to the ability to distinguish different materials. For example, to tell an apple from an orange by touching, you need to judge whether the surface is smooth or rough, the vibration frequency during friction, and so on. With so many complex dimensions, how can robots recognize them?

02

Implementation Paths for Tactile Sense

▍ Five Main Technical Routes

There are currently five mainstream solutions on the market: piezoresistive, piezoelectric, capacitive, optical, and magnetic induction.

The first one is the piezoresistive type, which is the most traditional and mature route. When the material is stressed, it deforms and its resistance value changes. By measuring the change in resistance, we can deduce the applied pressure. Common electronic scales work based on this principle.

Its biggest advantages are maturity, low cost and low technical threshold, but it can only measure the force in the vertical direction, and its readings will drift with changes in temperature and humidity.

The second one is the piezoelectric type. When special materials such as quartz and PZT ceramics are stressed, the centers of their internal positive and negative charges shift, generating a voltage signal, just like the igniter in a lighter.

Its most prominent feature is that it is only sensitive to changing forces: the moment you press it, it generates a voltage signal, but if you keep pressing it without moving, the signal will gradually decay to zero, and when you release it, it generates a reverse voltage signal. Therefore, the piezoelectric type is very suitable for detecting high-frequency dynamic signals such as vibration and impact, but it is difficult to measure static force.

The first two routes directly convert "force" into "electrical signals", while the latter three convert "force" into "deformation" and then measure the degree of deformation. A major benefit of this approach is that it is no longer limited to the vertical direction, but can detect three-dimensional force.

The principle of the capacitive type is based on parallel plate capacitors. When the material is stressed, the elastic material deforms: for example, when you press it, the upper and lower layers get closer, the capacitance increases, and then the vertical force can be measured; or when sliding, the plate area changes and the capacitance also changes, so the force on the plane can be sensed. Its advantages are extremely high sensitivity, fast response speed and relatively low cost, but it has a limited dynamic range and is easily affected by electromagnetic interference.

The fourth one is the optical type, also known as visuo-tactile. A layer of flexible material covers the top of the sensor, which deforms when it comes into contact with an object. There is an illumination layer on the back of the material, and a camera hidden at the bottom detects the state of the material in real time, calculates the deformation through algorithms, and then converts it into force distribution. Therefore, this is a type of sensor that can solve both contact morphology and contact force at the same time. It is the most popular and highest-precision method in the current industry, but its cost is relatively high.

The fifth one is magnetic induction. Its structure is similar to the optical type, except that magnetic particles are added to the surface elastic material, the intermediate illumination layer is removed, and the bottom camera is replaced with a magnetic sensor. When the surface elastomer deforms, the magnetic field also changes, and the bottom sensor captures the change to calculate the deformation, which can also deduce three-dimensional force. However, magnetic induction is vulnerable to electromagnetic interference, the surrounding magnetic field environment will affect its accuracy, and its resolution is limited.

There are other less commonly used methods, such as magnetofluid and ultrasound, which we will not elaborate on. Each of the above methods has its own advantages and disadvantages. The guests we interviewed believe that for robot tactile sense, the combination of multiple sensors may be a better solution.

▍ Multi-sensor Integration

Every inch of our skin except our fingertips has perception. If robots want to reach the human level, do they also need to be covered with tactile sensors all over their bodies?

But if we use the most advanced technology all over the body, the cost will be very difficult to control, and different parts have different requirements, so a single technical route cannot be applied to the whole body. For example, the fingertips require extremely high precision, and the optical solution can achieve higher resolution, but for other parts such as the palm and arm where the precision requirement is not that high, using the expensive optical solution is not cost-effective.

Tao YU

Director of Artificial Intelligence / Head of Dexterous AI, Analog Devices (US)

The palm, arm and other parts have relatively large areas. For example, if we consider the visuo-tactile solution, the design of the optical module will be very difficult, but if we use piezoresistive or capacitive technology, it is easier to cover such a large area.

In some relatively simple application scenarios, we can also choose routes with lower precision. For example, at the WAIC exhibition, we saw that some capacitive tactile sensors have been mass-produced for product sorting and inspection in factories, because their cost is relatively low, and their purchase price is only 1/4 to 1/2 of that of the highest-precision optical type.

Moreover, even for the same part, there is an opportunity to integrate multiple technologies in the future, provided that the technology is sufficiently mature.

Tao YU

Director of Artificial Intelligence / Head of Dexterous AI, Analog Devices (US)

Capacitive technology has very high sensitivity, but the problem is that after the material is compressed, the perception range of force will be very limited. But piezoresistive is a technology that requires hard contact. Therefore, the combination of the two technologies is a feasible solution, but whether the manufacturing process, yield and mass production capacity can keep up with the demand is still a problem. At this stage, everyone is still trying their best to improve the defects of a single technical route.

In addition to force perception, tactile sense in a broad sense also includes vibration, material perception and so on, which needs to be supplemented by other sensors. For example, when humans want to distinguish silk from cotton cloth, they will generate vibration by touching and sliding to perceive the fine granularity of the material. How can robots achieve this?

Tao YU

Director of Artificial Intelligence / Head of Dexterous AI, Analog Devices (US)

We will add sensors such as accelerometers or microphones, which can more effectively capture the signals generated by high-frequency vibration.

Therefore, a complete tactile solution is not as simple as choosing only one technical route.

At present, the most cutting-edge and highest-precision technology in the industry is the optical type. The company founded by our interviewee Eric, DeepSight Tech, also chose this direction. Next, we will walk into his laboratory to explore the principle of the optical type.

03

Inside the Lab: 4 Major Difficulties of Optical Tactile Technology

Around 2000, a group of researchers began to try a bold idea: can tactile sense be "seen" by a camera just like vision?

One of the most