ByteDance is exploring autonomous driving, and the Seed World Model Team is responsible for the related work | 36Kr Exclusive

36Kr has learned from a number of industry insiders that ByteDance is exploring entry into the autonomous driving field. This project is currently led by the world model team of Zhou Chang under Seed. It is reported that Seed not only has teams for Zhou Chang's multimodal model, world model and other related areas, but also conducts research in the direction of large language models.

The technical routes of autonomous driving and world models overlap in many aspects.

Another source told 36Kr that in terms of business layout, the autonomous driving scenarios ByteDance plans to develop include unmanned logistics, which belongs to the automotive industry line under ByteDance's Volcano Engine.

Several insiders close to ByteDance said that ByteDance's team has previously carried out business discussions and exchanges with some leading autonomous driving teams, and ByteDance's autonomous driving project is now in the early stage of preparation. ByteDance has extended offers to a large number of top talents in assisted driving or autonomous driving fields, and some technical talents who are optimistic about ByteDance's layout also plan to join the team.

In response to the above news, 36Kr contacted ByteDance for verification, and ByteDance responded: "We have a lot of early research and exploration in the cutting-edge exploration field of large AI models, including the field of physical AI, but we have no plan to develop intelligent driving business."

Seed is the basic large model research team under ByteDance, founded in 2023, and it is a first-level strategic department of ByteDance.

Zhou Chang is one of the core R&D leaders of ByteDance Seed AI. He joined ByteDance in 2024, responsible for the R&D of multimodal large models and other related work, and the scope of business under his management has continued to expand since then. Last year, the visual generation business was incorporated into his scope of responsibility, and this year the Seed Robotics team also reports to Zhou Chang. This means that Zhou Chang's current management scope covers multiple fields including multimodal large models, world models, visual generation, and embodied intelligence.

Volcano Engine is ByteDance's cloud service brand. As early as 2020, Volcano Engine established the automotive industry line. Tan Dai, President of Volcano Engine, once publicly stated: "We are the first cloud vendor to set up the automotive business as a first-level department."

Since the beginning of this year, the wave of physical AI has risen, and autonomous driving and embodied intelligence are regarded as the two earliest scenarios for the implementation of physical AI. Among them, autonomous driving is recognized as one of the earliest embodied intelligence scenarios to be implemented.

Therefore, when all major technology companies are chasing the physical AI trend, autonomous driving may become a springboard for ByteDance to move towards embodied intelligence.

For the autonomous driving industry, at a time when the world model has become the technical consensus of the autonomous driving community, ByteDance's cross-border entry with its advantages in resources, talents and technology is very likely to shake the existing pattern of the industry.

World Model Becomes Technical Consensus, ByteDance Seizes Cross-border Opportunity

In fact, ByteDance has long shown its ambition for automotive business.

Before deploying autonomous driving, ByteDance has already entered the automotive cockpit field through its Doubao large model, aiming to redefine the interactive experience of the cockpit. Now ByteDance's further layout in assisted driving can form a closed loop of the intelligent experience of the entire vehicle.

36Kr has previously reported that Volcano Engine under ByteDance is one of the teams that has invested the most in automotive cockpit agents, and SERES is the automaker that has the most in-depth cooperation with it at present. Volcano Engine also customized the Thor z chip for NVIDIA, and deployed the cockpit model on the vehicle side in the form of an external "AI Box".

SERES has handed over the development leadership of the new cockpit interaction to Volcano Engine, and provided it with the full vehicle platform, implementation capabilities, and hundreds of millions of yuan in financial support.

In June this year, AIVA, an automotive brand cooperated by SERES and Volcano Engine, was officially released. It is expected that the AIVA ME7 equipped with the "Doubao Cockpit" will also be launched within this year.

However, it is worth noting that the assisted driving partner of the AIVA brand is DeepRoute. A person in the autonomous driving industry commented: "If the intelligent cockpit and assisted driving are not integrated, it is difficult to form a closed loop of the intelligent experience of the entire vehicle."

The technical consensus on world models in the autonomous driving industry has just given ByteDance the opportunity to cross-border into this field.

Last year, there was a debate in the industry about the technical routes of VLA and world models. This year, more and more companies including Geely, Momenta, Xiaomi, Qcraft have gathered under the banner of world models, including some former VLA advocates.

For example, XPeng. This year, it not only released its self-developed X-World world model, but also removed the "L module" in VLA, that is, the language module, and launched VLA 2.0. This means that XPeng no longer converts visual content into language for reasoning, but takes vision as the "cornerstone of intelligence". Such a model architecture is very close to the world model.

The MindVLA O1 released by Li Auto at this year's GTC conference has a built-in 3D ViT spatial encoder and a predictive latent world model, which not only emphasizes the understanding of language instructions, but also the structured understanding of three-dimensional space and the prediction of the evolution of future road conditions. The latter two are exactly the essence of world models.

Therefore, after ByteDance launches the R&D of its world model, it is only natural for it to cross-border into the autonomous driving field.

Cross-border Entry with Advantages in Resources and Talents, ByteDance Will Launch an Impact on the Autonomous Driving Industry

For ByteDance, the layout of autonomous driving may bring it to the other side of embodied intelligence faster.

This year, Jensen Huang, founder of NVIDIA, Elon Musk, CEO of Tesla, and others all predicted that the ChatGPT moment of physical AI has arrived. Autonomous driving and embodied intelligence are exactly the two earliest scenarios for the implementation of physical AI.

Their capability stacks are highly universal. Among them, the road condition standards and hardware of autonomous driving have tended to be unified, and the industrial chain has been relatively mature after years of development. Therefore, the industry generally believes that it is difficult to achieve more general embodied intelligence without well-developed autonomous driving. Therefore, autonomous driving for ByteDance is no less than a training ground for embodied intelligence.

Another important significance of autonomous driving for ByteDance's embodied intelligence may lie in data.

The lack of real data from the physical world is a factor that plagues many embodied intelligence enterprises. Autonomous driving has natural advantages in data collection.

Therefore, after ByteDance deploys autonomous driving, it will be easier to obtain a large amount of data from the physical world to iterate its world model, and then reuse the model capabilities for embodied intelligence.

Public information once showed that Tesla has billions of miles of real road data from FSD, and all capabilities including perception, world modeling and motion planning can be migrated to the Optimus robot.

The help that FSD emphasized by the industry brings to the Optimus robot is reflected in the "vehicle data pipeline", which allows the robot to skip many trial and error cycles.

Therefore, ByteDance's layout of autonomous driving, to some extent, reveals its ambition for embodied intelligence.

For the autonomous driving industry, ByteDance's cross-border entry into the field may have a considerable impact on the pattern of the autonomous driving industry.

The shift of autonomous driving technology to the AI paradigm has, to a certain extent, rewritten the underlying logic of industry competition. Autonomous driving technology based on the AI paradigm is rapidly aligning the product experience of all players. In addition, ByteDance also has advantages in talents and resources.

A person from the autonomous driving industry told 36Kr that ByteDance's advantage lies in that "it has enough capital to invest at present, and it has enough talents", and "ByteDance also has enough computing power chips, which is fully capable of supporting it to achieve later-mover advantage".

He explained: "The general world model that ByteDance is developing now is not exactly the same as the world model dedicated to autonomous driving, and there are quite big gaps in many technical details. Moreover, the world model dedicated to intelligent driving has a deeper understanding of traffic physics.

But ByteDance has sufficient computing power resources. As long as it can obtain enough traffic data, it is no problem to retrain a world model dedicated to intelligent driving.

As long as the cloud model is strong enough, the autonomous driving engineering capabilities that ByteDance lacks can be quickly supplemented by recruiting relevant talents."

After nearly 10 years of fierce competition in the autonomous driving industry, ByteDance's entry may bring considerable variables to the industry.