ByteDance explores autonomous driving, led by the Seed World Model Team | 36Kr Exclusive

36Kr has learned from multiple industry insiders that ByteDance is exploring entry into the autonomous driving sector. This project is currently overseen by Zhou Chang's world model team under the Seed organization. It is understood that Seed not only houses teams focused on Zhou Chang's multimodal models and world models, but also operates in the large language model direction.

The technical pathways of autonomous driving and world models share overlapping elements.

Another source told 36Kr that in terms of business direction, ByteDance intends to develop autonomous driving scenarios for unmanned logistics, which falls under the automotive industry line of ByteDance's Volcano Engine.

Several insiders close to ByteDance stated that the company's teams previously conducted business discussions and exchanges with leading autonomous driving teams, and ByteDance's autonomous driving project has now entered the early preparation phase. ByteDance has extended offers to many top talents in assisted driving or autonomous driving, and some technical professionals who are optimistic about this strategic move by ByteDance plan to join the team.

In response to these reports, 36Kr sought verification from ByteDance, which responded: "ByteDance conducts extensive early research and exploration in cutting-edge fields of large AI models, including physical AI, but has no plans to develop intelligent driving businesses."

Seed is a large-model fundamental research team under ByteDance, established in 2023 as a first-tier strategic department of the company.

Zhou Chang is one of the core leaders of R&D for ByteDance Seed AI. He joined ByteDance in 2024, responsible for the R&D of multimodal large models, and his scope of managed businesses has continuously expanded since then. Last year, visual generation businesses were incorporated into his purview, and this year the Seed Robotics team also reports directly to him. This means that Zhou Chang's current management scope covers multiple fields including multimodal large models, world models, visual generation, and embodied intelligence.

Volcano Engine is ByteDance's cloud service brand. As early as 2020, Volcano Engine established its automotive industry line, and Tan Dai, President of Volcano Engine, publicly stated: "We are the first cloud vendor to establish the automotive sector as a first-tier business line."

Since the beginning of this year, the wave of physical AI has risen, with autonomous driving and embodied intelligence regarded as the two primary scenarios for the first implementation of physical AI. Among them, autonomous driving is widely recognized as one of the earliest embodied intelligence scenarios to be deployed.

Therefore, as major tech companies are chasing the physical AI trend, autonomous driving may become a stepping stone for ByteDance to reach embodied intelligence.

For the autonomous driving industry, at a time when world models have become the technical consensus in the sector, ByteDance's cross-industry entry with its advantages in resources, talent, and technology is highly likely to disrupt the existing industry landscape.

World Models Become Technical Consensus, Creating Cross-Industry Opportunity for ByteDance

In fact, ByteDance has long demonstrated its interest in automotive businesses.

Before deploying autonomous driving, ByteDance had already entered the automotive cockpit field through its Doubao large model, aiming to redefine the interactive experience of vehicle cabins. Now, ByteDance's further deployment in assisted driving can close the loop on the intelligent experience of the entire vehicle.

36Kr previously reported that Volcano Engine under ByteDance is one of the teams that has invested the most heavily in automotive cockpit agents, with Seres being the automaker that has the deepest collaboration with it so far. Volcano Engine also customized the Thor Z chip for NVIDIA, deploying the cockpit model in the vehicle as an "external attachment" via the Al Box.

Seres handed over the development leadership of new cockpit interactions to Volcano Engine, providing it with the entire vehicle platform, implementation capabilities, and hundreds of millions of yuan in funding support.

In June this year, AIVA, the automotive brand jointly developed by Seres and Volcano Engine, was officially launched, and the AIVA ME7 equipped with the "Doubao Cockpit" is expected to hit the market within this year.

However, it is worth noting that the assisted driving partner of the AIVA brand is Horizon Robotics. An industry insider in autonomous driving commented: "Without integrating intelligent cockpits and assisted driving, it is difficult to achieve a closed-loop intelligent experience for the entire vehicle."

The technical consensus on world models in the autonomous driving industry has precisely provided ByteDance with the opportunity to cross into this sector.

Last year, the industry witnessed a debate over the technical pathways of VLA and world models. This year, an increasing number of companies including Geely, Momenta, Xiaomi, and Zhixing have rallied under the banner of world models, including some former proponents of the VLA approach.

For example, Xpeng not only launched its self-developed X-World world model this year, but also removed the "L module" — the language module — from VLA, introducing VLA 2.0. This means Xpeng no longer converts visual content into language for reasoning, but instead treats vision as the "foundation of intelligence". Such a model architecture is already very close to a world model.

The MindVLA O1 released by Li Auto at this year's GTC Conference features a built-in 3D ViT spatial encoder and a predictive latent world model, which not only emphasizes understanding of language instructions, but also structured understanding of three-dimensional space and prediction of future road condition changes — the latter two being the core essence of world models.

Therefore, after launching R&D on world models, ByteDance's cross-entry into the autonomous driving sector naturally follows a logical progression.

Cross-Industry Entry with Advantages in Resources and Talent, ByteDance Will Challenge the Autonomous Driving Industry

For ByteDance, its deployment in autonomous driving may accelerate its progress toward the goal of embodied intelligence.

This year, NVIDIA founder Jensen Huang, Tesla CEO Elon Musk, and others have predicted that the ChatGPT moment for physical AI has arrived. Autonomous driving and embodied intelligence are exactly the two primary scenarios for the initial implementation of physical AI.

Their capability stacks are highly generalizable. Among them, road condition standards and hardware for autonomous driving have tended to unify, and the industrial chain has become relatively mature after years of development. Therefore, the industry generally believes that it is difficult to achieve more general embodied intelligence without mastering autonomous driving. Thus, for ByteDance, autonomous driving serves as a training ground for embodied intelligence.

Another significant value of autonomous driving for ByteDance's embodied intelligence efforts may lie in data.

The lack of real-world data from the physical world is a challenge that plagues many embodied intelligence enterprises. Autonomous driving inherently has advantages in data collection.

Therefore, after deploying autonomous driving, ByteDance will more easily access massive amounts of physical world data to iterate its world models, and subsequently reuse the model capabilities for embodied intelligence applications.

Public information once showed that Tesla has billions of miles of real road data from FSD, with perception, world modeling, and motion planning all transferable to the Optimus robot.

The industry-emphasized support that FSD provides to the Optimus robot is embodied in the "vehicle data pipeline", which allows the robot to skip many trial-and-error cycles.

Therefore, ByteDance's deployment in autonomous driving, to some extent, reveals its ambition in the field of embodied intelligence.

For the autonomous driving industry, ByteDance's cross-entry into the sector may pose a considerable challenge to the existing industry landscape.

The shift of autonomous driving technology to the AI paradigm has, to some extent, rewritten the underlying logic of industry competition. Autonomous driving technologies based on the AI paradigm are rapidly aligning the product experiences across different players. Moreover, ByteDance has advantages in talent and resources.

An autonomous driving industry insider told 36Kr that ByteDance's strengths lie in the fact that "it currently has sufficient capital to invest, and a large pool of talent", and "ByteDance also has abundant GPU resources, which fully enable it to achieve latecomer advantage".

He explained: "The general world model ByteDance is developing now is not completely identical to the autonomous driving world model, with significant differences in many technical details. The dedicated world model for intelligent driving has a deeper understanding of traffic physics.

However, ByteDance has sufficient computing power resources. As long as it can access enough traffic data, retraining a world model specifically for intelligent driving will not be a problem.

With a powerful enough cloud model, the lack of autonomous driving engineering capabilities that ByteDance currently has can be quickly compensated by recruiting talents."

After nearly a decade of fierce competition in the autonomous driving industry, ByteDance's entry may introduce significant variables to the sector.

Author's WeChat: luckg17305264638