Sudden automatic braking that triggers unexpectedly while driving, the "hypersensitivity" of intelligent driving systems is nearly driving car owners crazy.

From the emergence of intelligent driving to today, the hardest part is not teaching vehicles how to move, but how to stop.

You are sitting in a vehicle running smoothly under intelligent driving control, with open road conditions and bright sunshine in front of you, and the blue line on the central control screen extends steadily. Suddenly, without any warning, the vehicle slams on a "death emergency brake" sharply, the seat belt tightens instantly, the mobile phone on the back seat flies towards the windshield, and there is nothing outside the window.

This is exactly the most eerie scenario experienced by countless car owners in intelligent driving applications: "phantom braking".

Over the past few years, intelligent driving technology has evolved from stacking code rules to the "end-to-end" mode that simulates human intuition, and even started to build a "world model". However, these vigorous technological evolutions seem to be trapped in the tiniest moment: when is the most appropriate time for a vehicle to step on the brake?

Behind countless such subtle moments that are hard to judge, there is not only a technical problem, but also a problem of trust. On one side are automakers eager for exponential growth in driving mileage, and on the other side are users who will completely abandon the function due to a small mistake. When the development of a new technology is related to user safety, how to balance technology and trust?

Where does the "phantom" of phantom braking come from?

Intelligent driving turning into "unintelligent state" usually starts with the failure to brake properly.

Take my first experience with robotaxi as an example: About three years ago, I took my first ride on a robotaxi from a major domestic manufacturer. After we finished a lap, when the vehicle was returning to its position, a pedestrian appeared on the roadside. In fact, the pedestrian was still very far away from the vehicle, but the vehicle suddenly braked sharply. The braking force was so strong that I, who was sitting in the back seat recording a video, threw my mobile phone flying directly.

This phenomenon is called "phantom braking", a kind of "paranoia" that intelligent driving seems to suffer from, and it is also the most complained scenario for intelligent driving on forums at home and abroad such as X, Reddit, and Xiaohongshu.

A domestic smart car owner complained to me that when changing lanes on the highway, a car passed by on the right. After that car had overtaken his own car for several seconds, his car reacted and braked. This kind of braking error caused by hesitation in lane changing also occurs frequently in FSD. A blogger who specializes in testing Tesla FSD on Reddit once posted a complaint that when FSD is turning, the turn signal flashes for a long time but it still hesitates to merge into the lane, and it does not change lane until the car behind is impatient to overtake, which is very dangerous.

Some people complain that when facing a stop sign, FSD does not know the most appropriate distance to stop, and often brakes far away then moves forward step by step, looking extremely "unintelligent". Others pointed out that at dusk, early morning and other moments with dim sight, FSD often has "phantom braking" or even exits the autonomous mode directly...

Combining Waymo's safety reports, NHTSA data aggregation and analysis from the third-party insurance agency ConsumerShield, the three scenarios with the highest accident rate of intelligent driving currently are: lane change failure at complex intersections, phantom braking, and failure to handle sudden objects such as "suddenly appearing pedestrians". In the final analysis, all of these are caused by the unclear judgment on when to brake.

Why is it even harder to teach a car when to brake than to teach it to drive? This is not due to insufficient computing power, but because the braking action is essentially not a mechanical execution, but a "prediction"-related gamble.

The most fundamental difference between human drivers and autonomous driving systems is that humans can judge intentions very easily.

When a human sees a pedestrian on the roadside looking down at his mobile phone, he will predict that the pedestrian is most likely not to move; when seeing someone with wandering eyes and unsteady steps, he will predict that the person might rush out while drunk. However, most intelligent driving systems can only calculate based on the moving speed and direction of objects. When the pedestrian really rushes out and has a lateral speed, the system starts to calculate the probability of physical collision. The "calculation delay" plus the mechanical execution time in this process often leads to the lag feeling of "heavy braking only when the accident is about to happen".

But this is very difficult for autonomous driving systems. In the logic of engineers, the intelligent driving system is constantly facing a cruel multiple-choice question every moment: to "over-brake" or to "under-brake"?

If for absolute safety, the sensitivity of radar and vision is pulled to the maximum, then a plastic bag floating by the roadside, a glint of accumulated water on the manhole cover, or even a cloud of black smoke from the exhaust pipe of the truck ahead, may be rendered as a hard boulder in the "eyes" of the system. To save lives, it can only slam on the brake desperately - this is where "phantom braking" comes from.

On the extreme contrary, if for the sake of comfort and traffic efficiency, the perception sensitivity is reduced and those signals that seem to be "clutter" are filtered out, then when a stationary white truck is lying across the road, or a pedestrian in dark clothes suddenly appears from the dark at night, the system may "confidently" think that there is no obstacle ahead, which will eventually lead to a tragedy.

Human drivers have "physical intuition" based on common sense. We can tell at a glance that it is okay to hit a flying newspaper, but machines do not have this kind of common sense. In their world, that is just a pile of unknown "obstacles" occupying spatial voxels.

Who set such a difficult question? No correct answer can be found anywhere

Intelligent driving has tried many paths to cultivate the machine's "physical intuition".

Initially, the traditional intelligent driving research solution divided driving into three steps: perception, planning, and decision-making. Information is passed between these modules like a relay baton - the perception layer says: There is an obstacle 10 meters ahead, with a confidence of 80%.

The prediction layer says: This thing seems to be not moving.

The decision-making layer checks the 502nd rule written by the programmer: If there is a stationary obstacle ahead, brake immediately.

The control layer executes the command: Slam on the brake hard.

However, there is always a problem in the planning and control link: many things do not have an optimal solution.

For example, the vehicle needs to turn right now, but there is a pedestrian walking on the right, and the pedestrian is still a certain distance away from the vehicle. If a human driver is teaching a novice, it is okay to step on the accelerator to pass at this time, or wait for the pedestrian in place. This kind of "can be decided according to the situation" is very simple for humans, but very complicated for machine learning. Because this will lead to the next problem: which situation should we accelerate to pass, and which situation should we wait in place.

What traditional planning and control does is to do the work of "analyzing specific problems in detail", manually marking out the corner cases (extreme situations in special scenarios) encountered and telling the machine what to do. This will lead to two other problems: 1. What if the corner cases cannot be exhausted? 2. What if there are too many codes and too much manpower is required?

The phenomenon of "phantom braking" is very easy to occur under traditional research methods, because there will always be situations like "I have not handled this case before, so I don't know what to do best".

Elon Musk, who adheres to the "first principle", overthrew the traditional R&D path and proposed another solution: If I can directly make the machine think like a human, instead of manually marking and teaching it how to handle each corner case, can I truly realize the original intention of intelligent driving.

Thus, in the summer of 2023, Musk held a FSD live broadcast with not very clear picture quality in Palo Alto, California. But what really shocked the industry was not the picture quality, but the sentence he said: "There is not a single line of code in FSD v12 that tells the car what a red light is and what an intersection is. We deleted more than 300,000 lines of C++ control code."

Since then, end-to-end has gradually become the mainstream solution for realizing intelligent driving.

After the launch of the end-to-end solution, the entire planning and control department of the FSD team was disbanded, the former person in charge left, and the remaining personnel were transferred to data cleaning and scenario mining. The team's four old lines of perception, planning and control, mapping, and testing were all broken up and merged into three modules: "single model + data engineering + cloud training".

By 2024, end-to-end has gradually become the consensus in the industry. Intelligent driving R&D teams such as Huawei, Xpeng, Li Auto, and Baidu have started adjustments one after another. Xpeng revoked the original three departments of perception, planning and control, and mapping, and newly established three major sectors of "AI model development, AI application, and AI efficiency"; the ADS 3.0 organizational structure was fully integrated in May 2024, with perception/planning and control/mapping merged into the "End-to-End Architecture Department", which is directly managed by Richard Yu; Baidu is now operating in a dual-track way, with some people in the traditional algorithm team, and an additional ADFM team established, some of the original perception personnel were transferred there and expanded recruitment was also carried out.

The so-called "end-to-end" means that driving is no longer disassembled into "perception, decision-making, control", but all these links are integrated into a huge neural network. The input end is the original video stream captured by the camera, and the output end is directly the angle of the steering wheel and the force of the accelerator and brake. What happened in the middle? There are no longer codes written by programmers such as "stop at red light and go at green light", but a black box containing billions of parameters is operating frantically.

This is like evolving from "memorizing traffic rules" to "practicing driving sense". The biggest advantage brought by this technological change is the generalization ability for ambiguous scenarios.

However, two years after the end-to-end concept was proposed, why does intelligent driving still have "unintelligent" moments such as "phantom braking"? Because although end-to-end solves the "intuition" problem, it brings two new and more terrifying defects: inexplicability and lack of logical reasoning.

All large models have their inexplicability, just like we can never explain why when you ask Gemini to change the color of the table, it feeds you a picture of a dog head; or when you ask ChatGPT to extract data from web information to make a table, it feeds you two full pages of code...

In ChatBot products, this kind of inexplicability will be regarded as a joke of "unintelligent moment", but in the intelligent driving scenario, this kind of inexplicability may cause major accidents.

Therefore, based on end-to-end, some paradigms have been derived. For example, VLA — Vision-Language-Action model.

If we compare the machine trained by end-to-end to a racing driver, the meaning of VLA is to make this racing car better adapt to daily traffic rules. We can simply understand "VA" as end-to-end, and "L" as a large language model, so it is easy to understand that VLA is to install a "language center" for autonomous driving. It not only looks at the road, but also can "read" the road. It can understand the literal meaning of traffic signs, and understand the social contract behind the gestures of traffic police. When VLA intervenes, the car no longer drives by intuition, but can have an inner monologue: "I see the police car flashing its lights. Although it is a green light, I should give way, because the law stipulates that police cars have priority." This makes up for the logical chain that end-to-end is most lacking.

This year, as the concept of "Physical AI" has become more and more popular, the concept of world model has been mentioned many times. Because people find that when the model parameters are large enough, it can not only generate pixels, but also emerge the understanding of physical laws. For example, in the video generated by Sora, the gravity sense of liquid flow and the perspective relationship of object occlusion do not need to be rendered by manually written physical formulas, but are "realized" by the model itself.

The world model is also applied to intelligent driving R&D to solve the hallucination problem of "end-to-end". The world model can act as a "referee". Before the car takes action, the world model first deduces in the virtual space: "If I do this, what will happen in the next five seconds?" So as to verify whether the end-to-end decision is safe.

As a human driver, what should I do

A user wrote on Reddit: "99% of the time I think FSD is like a god, but in that 1% of the time, it tries to drive my family into a concrete barrier at the highway ramp. At that moment, my wife was screaming and my child was crying in fear. Although I took over the control and no accident happened, I canceled my FSD subscription that night. I cannot afford the cost of that 1%."