HomeArticle

Zhou Chang has left ByteDance Seed, behind which lies the dispute over the unified multi-modal development path.

白鲸实验室2026-11-03 08:17
Why is it so difficult to unify multimodal understanding and generation?

The rumors about Zhou Chang's resignation are not new.

After joining ByteDance's Seed team in August 2024, he has been rumored to leave at least twice. The first rumor emerged in December 2025. At that time, Zhou Chang posted on his WeChat Moments to deny the news, stating that the relevant rumors had already affected his work. This October, he was once again rumored to have submitted his resignation.

In fact, during his tenure at ByteDance, Zhou Chang had thought about leaving more than once or twice. An insider from the Seed team told us: "Zhou Chang always pulls stunts. Every time he wants to leave, he ends up staying after two weeks."

But this time the outcome is different. According to Z Finance, Zhou Chang has left ByteDance to start a business in the Physical AI track and has received investment from a top-tier US dollar fund. However, people close to ByteDance denied the resignation news.

It is rumored that before his resignation, Zhou Chang no longer controlled half of the multimodal business. We learned that Zhou Chang's core base at ByteDance was the Vision-Language Model (VLM), but VLM has never delivered obvious AGI-related effects. In an organizational adjustment of Seed in August this year, the VLM business that Zhou Chang built up from scratch was merged into the Large Language Model (LLM) system, which is equivalent to becoming a sub-department responsible for data under the LLM business line.

"He lost his core base, and basically had no core territory left," the above-mentioned Seed insider said. This may be the direct trigger for his decision to resign.

The biggest public debate over Zhou Chang is how much he actually contributed to Seedance. In the view of this Seed insider, Zeng Yan is the actual leader of Seedance, and all technical work and decision-making are made at her level.

Putting aside personnel relations, Zhou Chang's resignation also marks the end of ByteDance's narrative of "unified multimodal understanding and generation".

01

Losing the "Core Base"

In 2024, Zhou Chang left Alibaba's Tongyi Qianwen to join ByteDance's Seed team, with a job level of 4-2, an annual salary package reportedly reaching eight figures, and he brought more than ten former Alibaba colleagues with him. This talent migration once aroused dissatisfaction from Alibaba, and the two sides were even involved in a labor arbitration over non-compete agreements.

Zhou Chang is not just a pure business manager, he also has a strong academic background. He graduated from Fudan University with a bachelor's degree in Computer Science and Technology, and then went to Peking University to pursue a doctorate in Computer Software and Theory, with research directions covering deep learning, graph mining and distributed computing.

In July 2017, he joined Alibaba, and worked his way from e-commerce recommendation algorithms and graph neural networks to multimodal pre-training. A few years later, he led the team to develop the M6 multimodal pre-training model.

When the large model wave rose in 2023, he switched to Tongyi Qianwen as the core technical person in charge, promoting Qwen to evolve from early models to serialized R&D and open source, and gradually became a key figure in Alibaba's large model system.

After joining ByteDance, Zhou Chang was arranged to be in charge of the multimodal interaction and world model direction, which is exactly the extension of his technical accumulation over the past years. Whether it is the Vision-Language Model or a more advanced world model, the model needs to have the ability to understand, reason and represent visual information, and multimodal understanding is exactly Zhou Chang's advantage.

The turning point came in 2025. In February, Wu Yonghui, who had worked at Google for 17 years and participated in the core R&D of Gemini, joined ByteDance to take charge of Seed and restructured the team's reporting lines. At that time, Yang Jianchao was the head of Doubao visual multimodal generation, responsible for image generation, video generation and basic research on visual models. In July, Seed carried out an organizational adjustment, and Yang Jianchao suddenly announced that he would "take a temporary break".

The public version is that Yang Jianchao took a temporary break for family reasons, and Zhou Chang took over his work. In fact, this organizational adjustment shook his position. The above-mentioned Seed insider revealed that the organization required Yang Jianchao to report to Shiguang (Zhou Chang's nickname), Yang Jianchao was dissatisfied with this, and left soon after to start a business in the video model track.

It was recently reported that Yang Jianchao has joined creative software giant Adobe as Vice President of Research and AI Technology.

Taking over Yang Jianchao's business means that Zhou Chang was not only responsible for the multimodal understanding business, but also took over the visual generation business. After that, Zhou Chang began to promote a new technical narrative inside ByteDance — the unification of multimodal understanding and generation.

But this narrative is not easy to promote. ByteDance had previously open sourced BAGEL, a model that explores the route of "unified understanding and generation", and this technical route itself is still in the early exploration stage.

After Zhou Chang took over the relevant business, there were not many publicly released achievements that clearly reflect the integration of understanding and generation architecture. The most representative one is the Doubao image creation model Seedream 5.0 Lite, released in February 2026 at the same time as Seedance 2.0.

The performance of this model in evaluation results is not particularly outstanding. ByteDance itself described this model as "a relatively small model", which "still has room for improvement" in structural stability, realism and aesthetics.

In the industry's view, Zhou Chang's technical accumulation is more concentrated on multimodal understanding, and he is not very familiar with multimodal generation.

A more important change took place at the technical route level. Since the beginning of this year, the focus of the industry narrative is shifting: compared with integrating multimodal understanding and generation into the same model, more and more manufacturers have begun to give priority to promoting the integration of Vision-Language Model (VLM) and Large Language Model (LLM).

In August this year, Seed launched a new round of organizational adjustment, and the VLM business originally under Zhou Chang's charge was merged into the LLM system. This means that the part of the business that he is most familiar with and has the most technical accumulation has also been split out from the original organizational structure.

02

Why It Is Difficult to Unify Multimodal Understanding and Generation

Zhou Chang nominally took over visual generation businesses such as Seedream and Seedance, but the real leader of Seedance is Zeng Yan. She joined ByteDance through campus recruitment in 2021, reached the 4-2 job level in five years, and is the technical person in charge of Seedance 2.0.

In fact, the technical route of Seedance has no direct relationship with the "unified multimodal understanding and generation" that Zhou Chang emphasized. It has its own R&D logic and technical accumulation, and is not the product of this unified narrative.

ByteDance's most successful and profitable model Seedance 2.0, its core technology is the "Dual Branch Diffusion Transformer (DB-DiT)" architecture, which generates video and audio in parallel in one model, shares the same understanding encoder, and realizes native audio-video collaboration.

This is a unification within the generative modality, which connects audio and video, not understanding and generation. The model can understand multimodal reference materials and generate content based on them, but this is still two different things from truly integrating visual understanding and visual generation into the same basic model.

In other words, the success of Seedance 2.0 relies on Zeng Yan and her team integrating video and audio into the same DiT, and has little connection with "unified understanding and generation".

Multimodal unification includes two different levels. The first level is the unification of LLM and VLM, which integrates visual encoding capabilities into the large language model, so that AI can not only process text, but also understand pictures and videos, and perceive objects, scenes and actions. This solves the problem of how the model perceives and understands the world.

The second level is the unification of VLM and DiT, which integrates the originally relatively independent visual understanding and diffusion generation capabilities into the same set of model architecture or parameter system, so that the model can not only understand visual content, but also generate pictures and videos. This solves the problem of whether the model can further create the world on the basis of understanding the world.

GPT4o released by OpenAI in 2025 aroused the industry's interest in the unified framework of multimodal understanding and generation. Whether it is Google, or domestic giants such as Alibaba and ByteDance, they are generally trying to achieve the second level of unification.

But after more than half a year of exploration, they have generally encountered practical difficulties in the unification of VLM and DiT, such as the high-dimensional computing bottleneck of multimodal data and the problems in existing evaluation systems. This unification path is still in its early stage, and the route is far from converged.

A multimodal researcher from Tencent Hunyuan told us, "Multimodal understanding and generation are not that unified. Understanding is generally done by the LLM team, and generation is relatively independent." A researcher from Alibaba Qwen also said bluntly that there is basically no known very effective solution for "unified understanding and generation", and no one can achieve it at present.

Generally speaking, people engaged in multimodal research are more convinced of the unification of LLM and VLM, and generally have doubts about the unification of VLM and DiT.

The technical logic behind this is not complicated. Understanding models usually adopt an autoregressive architecture to complete understanding and reasoning by predicting the next token. Image and video generation, on the other hand, relies heavily on diffusion models to gradually denoise in the continuous latent space. The training objectives, data representations and computing mechanisms of the two systems are different. Stuffing them into the same model, how to avoid capability interference between different tasks is a difficult problem in itself.

Large manufacturers at home and abroad have all experienced a period of wavering on technical routes. Google did not do the "VLM + DiT" style unification in the early stage. It took Gemini as the core of native multimodal understanding and reasoning, and retained professional generation models such as Imagen and Veo for a long time.

Later, Gemini 2.0 began to try native image generation, and Gemini 2.5 Flash Image further integrated understanding, generation and editing.

But this does not mean that Google has used one set of parameters and one architecture to solve all understanding and generation problems. But when it came to Gemini 3.0, Google mapped multimodal data to a unified representation space again, which means it returned to the unification of multimodal and large language models.

Since last year, Alibaba has been conducting research on the unification of multimodal understanding and generation from papers to practice. But in February this year, starting from Qwen3.5, visual capabilities were integrated into the main model through early fusion, and Qwen no longer released new general VL understanding models.

In July this year, Tencent also began to lead the narrative of integrating multimodal capabilities into the large language basic model. However, news came out today that Pang Tianyu, Principal Research Scientist of Tencent Hunyuan and technical person in charge of Multimodal Reinforcement Learning (Multimodal RL), has resigned. He joined Tencent from Singapore Sea AI Lab at the end of January this year, and has been in the company for less than a year.

Just as Yang Jianchao's departure may be due to the change of technical narrative route, Zhou Chang's departure is also related to the new route transformation.

He left ByteDance with the narrative of "unification", and turned to start a business in Physical AI. Physical AI exactly requires world models, as well as multimodal understanding and generation capabilities. The model needs to not only understand the physical world, perceive objects, space and motion, but also predict how the world will change, and finally generate decisions and trajectories that can guide actions.

In any case, the technical ideal that Zhou Chang tried to realize inside a large manufacturer has now started again in a new place.

This article is from the WeChat official account "Beluga Lab", author: Wu Xun, editor: Liu Pei, published with authorization from 36Kr.