BYD's AI team is unveiled to the public for the first time, with remarkable robotics-related technical genes from Harbin Institute of Technology, and its large model reaches SOTA level upon its debut.

Even BYD has started publishing AI papers!

And these are not low-quality perfunctory papers with mediocre outputs.

The latest newly published work HyWorldVLA comes from BYD New Automotive Technology Research Institute, targeting the most popular direction in autonomous driving at present — Vision-Language-Action (VLA) + World Model.

HyWorldVLA has achieved SOTA (state-of-the-art) results on the public autonomous driving benchmark NAVSIM v1, with a PDMS score reaching 90.59.

What a huge contrast!

In the past few years, when the public talked about BYD's intelligent capabilities, they mostly associated it with "God's Eye", supply chain integration, large-scale implementation, and of course the "full backing" commitment launched this year.

But if you are asked to name any specific AI technologies and academic achievements of BYD that are self-developed down to the code level... you will draw a complete blank.

There were even rumors that the PPT for BYD's intelligent product launch events was made by suppliers.

However, the release of this paper has shown everyone a BYD with a completely different technical gene:

It not only has autonomous driving achievements, but also has a R&D team that has long been committed to the research of foundational physical AI models.

What problems does HyWorldVLA aim to solve?

Let's first talk about what this paper is focusing on.

HyWorldVLA targets the most cutting-edge technical route in the current autonomous driving field:

Enabling AI to evolve from "recognizing roads" to "understanding the world".

Therefore, the industry has begun to explore end-to-end autonomous driving models in recent years. Such models directly take inputs from sensors and output vehicle control signals or trajectories, and the further development direction is to integrate the World Model.

To put it simply, the World Model is designed to endow AI with an "internal simulator". It not only knows what is happening now, but also can predict the future: how will the road change in a few seconds? What actions may other road users take? What strategy should the vehicle adopt?

This is also the direction that players including Tesla FSD, Waymo and NVIDIA are exploring.

However, there is still a core contradiction in the current autonomous driving world model:

We want the model to understand the details of the real world, and at the same time hope that the model can perform reasoning efficiently.

One approach is the Pixel-level World Model, which directly predicts future video frames. Its advantage is that it is intuitive, and the model can generate future changes of roads, vehicles and the environment. But its disadvantages are also obvious: high computing cost, and it is easily affected by visual noise.

The other approach is the Latent World Model, which predicts the future in the compressed latent space. This method is more efficient and more suitable for large-scale model training. But its problem is: the model may understand abstract relationships, but lose the details of the real world.

What HyWorldVLA proposes is a Hybrid World Modeling solution, which retains the visual world model's ability to understand environmental details, and uses the latent space model to improve reasoning efficiency.

On this basis, the paper further introduces the Vision-Language-Action (VLA) model, enabling the system to not only understand the visual environment, but also generate driving actions combined with semantic information.

It combines the advantages of the two routes and avoids their respective shortcomings, finally forming an end-to-end process of perceiving the environment, understanding the environment, predicting the future and taking actions.

What about its performance?

The test platform selected by HyWorldVLA is a well-recognized public benchmark in the current autonomous driving research field: NAVSIM v1.

Different from traditional visual task tests, it pays more attention to whether the vehicle can complete the driving task safely and efficiently if it actually drives according to the route planned by AI.

Therefore, it adopts PDMS (Predictive Driver Model Score), a comprehensive indicator that is closer to the actual driving quality, including whether there is a responsible collision, whether the vehicle stays in the drivable area, whether a reasonable safe distance is maintained, whether the driving target is completed, and whether the vehicle movement is smooth and comfortable.

In other words, a model cannot get high scores by "driving slowly", nor can it only pursue speed while ignoring safety. It needs to achieve safety, efficiency and comfort at the same time.

HyWorldVLA achieved a PDMS score of 90.59 in the NAVSIM v1 test, reaching the leading level among all public results.

This shows that at least in public datasets and simulated driving environments, this foundational model from BYD has the cutting-edge capabilities of current autonomous driving research.

However, we need to view this objectively: leading in benchmarks does not mean that mass-produced autonomous driving has been completed.

Therefore, the more accurate significance of BYD's SOTA-level autonomous driving large model is:

BYD has proved that it has the research capability of multimodal and physical AI foundational models.

How did they achieve this?

The hybrid design of HyWorldVLA does not simply splice the two routes together, but makes the pixel-level and latent space representations play their respective roles in different training stages. The whole process is divided into three steps.

The first step: Train a video compressor.

First, use video VAE to compress consecutive video frames into compact latent features, so that subsequent models make predictions in this compressed space, instead of struggling with noise in the original pixel space. To improve compression quality, text information is introduced as semantic guidance in the encoding process — text descriptions help the model understand which are important structures in the picture and which are negligible details, which significantly improves the clarity of the reconstructed picture.

The second step: Pre-training. Convert all images, actions, languages and latent features into unified discrete tokens, splice them into a long sequence, and let the model learn to predict what the next token is in an autoregressive way.

The key design at this stage is "walking on two legs" — the model does two things at the same time: predict the tokens of future frames, and predict future latent features. The frame token prediction acts as a "supervisor", forcing the latent space to retain enough information to restore clear frames, preventing the representations from becoming empty and useless during the learning process.

The third step: Joint fine-tuning. At this stage, the model no longer predicts frames, and only outputs latent features. The action generation module fuses these latent features with historical information to generate the final trajectory.

The paper verified two action modules — one selects the optimal result by scoring from candidate trajectories, and the other directly outputs continuous trajectories using a generative model. Both of them have performance improvements, which shows that the gain of this method does not depend on a specific action design.

What exactly makes this SOTA performance possible? The ablation experiment gives the answer.

The biggest contribution comes from the "hybrid" design itself. If frame prediction is removed and only the latent space is retained, the PDMS score drops from 90.59 to 87.50 — this is the largest drop among all ablation groups, indicating that pixel-level fine supervision is crucial to the overall performance. Conversely, if the latent space is removed and only the pixel-level model is retained, the PDMS score also drops to 89.91. Neither of the two routes can be omitted, and it is impossible to reach 90.59 with only one single route.

The weight of latent feature supervision is also very important. During joint fine-tuning, if there is no constraint on latent features at all (λ₃=0), the PDMS score is only 90.17; when a moderate weight is set (λ₃=0.1), the optimal score of 90.59 is achieved; if the weight continues to increase, the score drops to 89.75. This shows that moderate constraints can prevent the semantics learned in pre-training from being washed away during fine-tuning, but too strong constraints will restrict the model from discovering the most useful representations for driving.

Noise robustness is an "extra bonus" brought by the latent space. The paper specially collected 655 rain and fog scenes for testing. The pure pixel method WoTE only got 60.65 points, and DriveVLA-W0 only got 61.18 points; HyWorldVLA reached 86.87 points. This gap shows that in bad weather, reasoning in the compressed space is far more reliable than struggling in the pixel space.

In short: This solution wins by "working in stages" — during pre-training, the latent space is fully enriched through pixel reconstruction to retain sufficient environmental information; during fine-tuning, it switches to pure latent space reasoning, and automatically gains immunity to rain, fog and light interference.

The two stages have clear divisions of labor and do not interfere with each other, instead of making one model undertake two tasks at the same time.

HyWorldVLA represents the exploration direction of autonomous driving foundational models, which is in the same technical trend as routes such as Tesla FSD, Waymo, and NVIDIA Cosmos, but it chooses the specific technical path of combining VLA with hybrid world model, with its own differentiated focus.

However, to achieve large-scale mass production deployment, it still needs to face a series of engineering challenges such as real data closed loop, vehicle-side computing power constraints, and long-tail scenario coverage.

Re-recognize BYD AI: What kind of team is behind the paper?

This HyWorldVLA paper is completed entirely independently by BYD New Automotive Technology Research Institute (the former Planning Institute, led by Yang Dongsheng), without co-authorship from external universities or research institutions, which is not common in AI papers published by automobile enterprises.

Following the clue of "New Automotive Technology Research Institute" and expanding the search scope to all related information of BYD, we can find more complete information.

HyWorldVLA is BYD's first multimodal foundational model paper, but not its first AI paper.

At least there is another paper EMoE-Planner (autonomous driving planning model based on Mixture of Experts) published in 2025.

Other papers include racing trajectory optimization (Global minimum time trajectory planning considering curvature and distance for track racing) and terrain recognition through tire noise (Road Terrain Recognition Based on Tire Noise for Autonomous Vehicle).

But these fields are not deeply related to autonomous driving and more cutting-edge foundational physical AI research.

Going further back, there are almost no public AI academic papers of BYD available on public channels.

This proves to a certain extent that BYD's research on core AI foundational models has only started in recent years.

Several frequently appearing names also confirm this judgment.

Liulong Ma, there is not much public information about him, he is very low-key on GitHub, with no personal homepage or Google Scholar homepage, but he is a co-author on several public AI papers of BYD New Automotive Technology Research Institute, and is the corresponding author.

He first appeared in public related records in 2025.

Further back, Liulong Ma's academic experience covers multiple directions such as autonomous driving planning, robot navigation, multimodal perception, and world model, which is highly consistent with the current Physical AI technology trend —

He graduated from Harbin Institute of Technology, and is from the famous State Key Laboratory of Robotics and System of HIT, as well as the School of Mechatronics Engineering of HIT.

Judging from his research direction, his possible supervisor is Professor Zhao Jie from HIT, who is the director of the HIT Robotics Institute.

Starting from the clue of HIT, we can also find that BYD's AI team has a distinct HIT gene.

Hongbiao Zhu, the first author of the aforementioned EMoE-Planner, was also affiliated to BYD New Automotive Technology Research Institute when the paper was published in 2025, and his past resume is also closely related to HIT, as well as CMU.

This shows that the AI team of BYD New Automotive Technology Research Institute