HomeArticle

Zeng Yan's new paper is ByteDance's "100 Trillion" roadmap

字母AI2026-11-18 19:48
Zhou Chang has resigned and Zeng Yan has risen to the leading position: in this strategic route dispute at ByteDance, the trusted internal faction of long-tenured core employees defeated the externally parachuted senior executives.

Zeng Yan, a member of the post-1995 generation, is getting promoted at a strikingly fast pace.

Born in 1997, Zeng Yan joined ByteDance via campus recruitment in 2021, and later became the head of pre-training for Seedance. AI Letter has previously covered her in detail in the article "ByteDance's 'Luo Fuli' Who Supports Half of Seedance's Business".

There are now rumors that Zeng Yan's title has been promoted to core lead of Seedance. Back in March this year, Zeng Yan's job rank was 3-2, but by November, her rank had reached 4-2, achieving two consecutive rank jumps within one year. In addition, Zeng Yan has taken over more and more work from Zhou Chang, the former core lead of Seed.

Such a promotion speed is rare in ByteDance's Seed team, where talent density is extremely high.

The reason behind this may be that Zeng Yan's technical path is more aligned with Zhang Yiming's ideas. The papers published by Zeng Yan as the project lead with her team are the best proof.

This paper is titled "Cola2: Applying Diffusion Models and Continuous Latent Space Methods from the Video Generation Field to Language Modeling", and its weights were released on Hugging Face in the first half of November.

The paper proposes a method that allows the model to process text by generating videos. According to the disclosure in the paper, this method not only saves more computing power, but also delivers better generation effects.

This paper also serves as a technical roadmap at the same time.

Zeng Yan starts from diffusion models and believes in the path of scale effect, which is consistent with the concept of super-large parameter models led by Zhang Yiming himself as revealed by foreign media, and also fits the long-termism that Zhang Yiming proposed in internal ByteDance meetings.

At a time when computing power is in extremely high demand, it is difficult for ByteDance to support a multi-line parallel strategy. Therefore, ByteDance merged its departments, shifting from the multi-line horse-racing mechanism to concentrated betting. It aims to win on one modality first, then let the winning team bring their methods to expand to other modalities.

Zeng Yan's technical path is exactly this bet.

What Does Cola2 Talk About

Before explaining the Cola2 paper, we need to first popularize a basic fact.

Currently mainstream large language models, such as DeepSeek, Qwen, and Doubao, all use the same method called "autoregression". Just like the Chinese idiom solitaire game, based on the content that has already been written, the model guesses what the next token is, guesses one after another, and outputs tokens one by one from left to right.

This method is very successful, but it has an inherent flaw: it cannot draw up an outline in advance the way we do when writing articles.

When people write articles, they usually first figure out what this paragraph is about, and then organize the language. The autoregressive model thinks while writing, stops wherever it reaches, and must work in queue: as long as the previous character is not written, it can never write the next character.

The idea of Cola2 is to split the text generation process into two layers.

The first layer is to figure out "what to say" first in a place called "continuous latent space"; the second layer is to convert the confirmed meaning into specific characters one by one.

It is equivalent to that the first layer is drafting the outline, and the second layer is writing the full text according to the outline.

Although the term "latent space" sounds a bit sci-fi, it is actually used every day by people who generate AI videos.

The raw data of a video is very large. If you train the model directly with it, you will face insufficient computing power. Therefore, the common practice of video generation models is to first compress the frames into a much smaller "compressed package", let the model work in this compressed package, and finally decompress it into frame-by-frame images.

The work done in the compressed package is called "diffusion". It is similar to the old process of developing photos: at the beginning, there is a negative full of snow noise, and the model wipes off the noise step by step, so that the picture gradually emerges.

The logic of Cola2 is to treat "text generation" the same way as "video generation".

This logic already existed in the first-generation Cola DLM, but Cola DLM had shortcomings. The Cola2 paper states that Cola DLM performs poorly on generation tasks, and the problem lies in the decoder responsible for "writing the full text". Therefore, the main function of Cola2 is to make up for this shortcoming, and it has done three more things than the previous generation.

The first thing is to learn to split content by semantics.

If you want to compress text the same way you compress video, you must first decide how to split the content blocks.

The old method rigidly splits content by fixed length, for example, every two tokens form a block. As a result, "Peking duck" may be split into "Peking" and "duck", or even nonsensical fragments like "king duck".

Cola2 changes to dynamic splitting by semantics, compressing an average of two tokens into one "outline point", and the specific number of tokens grouped together is determined by the content itself.

The compression degree is controlled by a negative feedback mechanism similar to the thermostat of an air conditioner. If the compression is too tight, for example, "Peking", "duck", "delicious" will automatically loosen up a bit. If the compression is too loose, for example, "Peking duck is very delicious", it will automatically tighten up a bit, and always stay stable near the target value.

In the experiment, splitting by semantics has an 8.2% higher accuracy rate than splitting by fixed length. More interestingly, under appropriate settings, the score after compression (29.4%) is slightly better than that without any compression at all (28.7%), which indicates that what is compressed out is mostly redundant "moisture".

The second thing is to make the decoder that writes the full text able to both follow the outline and generate content independently.

If the decoder can only read strictly according to the outline, once the outline is wrong, the whole article will naturally be completely wrong. Therefore, when training the decoder, Cola2 deliberately provides it with three types of outlines: one is completely clear, one is added with noise and is hard to recognize, and the third type has no outline at all.

It is like training an actor: the teleprompter is sometimes clear, sometimes blurry, and sometimes completely blacked out, but he must be able to continue performing.

The final selected proportion is that clear outlines are only provided in 20% of cases, and the remaining 80% are blurry outlines or no outlines. Even so, the accuracy rate of the decoder restoring the original text according to the clear outline remains above 99.5%.

When actually writing content, use the CFG knob to adjust the proportion of "following the outline" and "playing independently".

The third thing is to train the outline and the full text together. This is exactly what the subtitle "Co-Adaptive" refers to.

The previous practice was usually to train the "compressor" first, freeze it, and then train the module responsible for writing the full text.

If the outline template is fixed, when writing the article, even if the problem of the outline is found, the outline side will not make any adjustment at all. Cola2 simply lets both sides train together: if the module writing the full text finds something insufficient, the signal will be transmitted back, forcing the outline side to adjust.

In the experimental results, if the compressor is completely fixed and not allowed to be modified, the score is 47.6%. When both sides train together, the score rises to 53.5%. Adding a training target that allows the compressor to learn to "predict the subsequent content", the score will increase to 58.2%.

There is also a counter-intuitive design here. The DiT responsible for drafting the outline is only allowed to learn, and is not allowed to modify the compressor in reverse.

Cola2 used about 346 billion tokens in training, the model has not converged yet, and the average score on 8 tasks is 60.10 points, surpassing Llama-3-8B on the HellaSwag and StoryCloze tasks. The publicly announced pre-training volume of Llama-3 is more than 15 trillion tokens, which is more than 43 times that of Cola2.

In terms of computing power, according to the paper, since an average of two tokens are compressed into one outline point, the sequence to be processed by DiT is halved. Theoretically, its main calculation amount will be reduced to half of the original, and the attention part will be reduced to a quarter.

The last part of the paper writes that the model is "far from converging", especially DiT. The training scale is "limited by available resources". To reach the upper limit of this path, larger models, longer training, more data and more computing power are needed.

In addition, the paper also states that the continuous latent space may become a shared interface between text, image and video, leading to the "unification of understanding and generation".

The "unification of understanding and generation" is exactly the story that Zhou Chang has been talking about at ByteDance for more than a year but failed to deliver.

Zeng Yan's Path Is More Aligned With Zhang Yiming's Ideas

On November 12, The Paper reported that Zhang Yiming recently said in an all-hands meeting for the AI team that ByteDance has the ability to compete with OpenAI and Anthropic, but to truly catch up with the cutting-edge level, "the earliest time will be 2029 or 2030".

Zhang Yiming said that compared with top overseas companies, the pre-training data scale of the Seed model is "still insufficient", which is the most urgent problem to be solved at present. At the same time, foreign media reported that Seed is pre-training a model with a maximum parameter count of up to 10 trillion.

Insufficient data, building larger models, and adhering to long-termism are obviously more inclined to Zeng Yan's path. Zhou Chang's path is not quite the same as this direction.

In July 2025, Yang Jianchao, who was originally in charge of visual generation, left the company, and the image generation and video generation lines were assigned to Zhou Chang. Since then, he has started to promote a new statement inside ByteDance called "unification of multi-modal understanding and generation".

Zhou Chang's idea is to start from the understanding end, bring generation into the system, and make one model that can both perceive and generate content.

But this story was not successfully delivered.

After Zhou Chang took over, the most representative achievement that embodies this unified idea is the image model Seedream 5.0 Lite, released in February this year at the same time as Seedance 2.0.

The internal evaluation of 5.0 Lite inside ByteDance is "a relatively small model", and it "still has room for improvement" in structural stability, realism and aesthetics. The general view of the community is that Zhou Chang understands understanding tasks well, but is not so familiar with generation tasks.

A researcher from Alibaba's Qwen team said that there is basically no known very effective solution for the "unification of understanding and generation", and no one can achieve it at present. Piling up parameters in this scenario may not lead to valid results.

In August this year, Seed carried out an organizational adjustment: the VLM team founded by Zhou Chang was merged into the large language model system, changing from an independent department to a subordinate data department. An internal insider of Seed said: "He (Zhou Chang) lost his base where he started his career, and basically no longer has core territory."

In fact, Zeng Yan also started her career in the understanding field. Just two months after joining ByteDance in 2021, she published X-VLM as the first author, focusing on how to make AI match the objects in images with corresponding text.

For example, when you say "a little girl in red clothes is riding a bicycle", the model must accurately find that little girl in the picture.

After transferring to Seed in 2023, she began to move towards the generation field. In 2024, Zeng Yan participated in the development of PixelDance, which solves the problem of character deformation when characters move in videos. It is the core technical base of ByteDance's current video generation business, and also the origin of the name Seedance later.

The success of Seedance 2.0 is not directly related to the "unification of understanding and generation". It generates video frames and audio together in the same DiT, so the picture and sound are naturally aligned. The capability of Seedance 2.0 mainly comes from the "continuous latent space" technology that Zeng Yan is good at.

Therefore, some Seed insiders said that Zeng Yan is the actual leader of Seedance, and all technical and decision-making power lies at her level.

The training objectives of understanding and generation are different, the data forms are different, and the calculation methods are also different. If you force them into one model, the two sides will easily drag each other down.

It is just like the "left-right mutual skill" of Zhou Botong in *The Legend of the Condor Heroes*: it is very simple for the left hand to draw a circle, and it is also very simple for the right hand to draw a square, but doing both at the same time is extremely difficult.

On this issue, Zhou Chang and Zeng Yan are moving in opposite directions.

Zhou Chang starts from the language model, while Zeng Yan starts from the diffusion model.

More critically, Zeng Yan believes in the scale effect more than Zhou Chang.

At the end of 2025, the Seedance team once had a dinner to discuss the training plan for the next-generation model. Zeng Yan proposed to train a larger model with at least 200B parameters, that is, 200 billion parameters. Some members of the team thought this was too aggressive, and it was more prudent to train a 100B model first, but Zeng Yan insisted on achieving the goal in one step. Later, in the formal review meeting of Seed, Wu Yonghui and Zhou Chang both supported her plan.

Previously, overseas media broke the news that Zhang Yiming is leading ByteDance to develop a large model with 10 trillion parameters, and as the richest man in China, he personally participates in the project. Obviously, Zeng Yan's path is more aligned with what Zhang Yiming wants to do.

There is also the last layer of reason, which is their origin.

Zhou Chang was recruited by ByteDance with an eight-figure annual salary. When he joined, he brought more than a dozen former Alibaba colleagues with him, which even led to a non-compete arbitration case from Alibaba. During his two years at ByteDance, there were at least two rumors that he would leave the company.

An internal insider of Seed said: "Zhou Chang always makes jokes about resignation. Every time he says he wants to leave, he stays two weeks later."

Zeng Yan is ByteDance's own campus recruit, who was promoted to rank 4-2 in five years, and has been working on the Seedance line all the way. For ByteDance, there is a clear difference in closeness between Zhou Chang and Zeng Yan.

ByteDance Is Shifting From Multi-Line Horse Racing to Concentrated Betting

Whenever ByteDance steps into a new field, it usually sets up more than one team. Zhang Yiming prefers to arrange multiple teams to explore in parallel at the same time, and bets on whoever can achieve results. Douyin was developed in this way.

The same is true for developing large models.

In the past two years, Seed has recruited the largest number of top researchers in China, distributed in various directions such as text, code, vision, speech, and reasoning. According to LatePost's report, some people once promised them that they could lead the "tide" with their own innovation here.

A former researcher at Seed said: "Everyone wants to achieve great things."

But two years later, there are too many "tides", which have instead become a resistance for ByteDance to move forward in the AI field.

Computing power is the biggest problem.

Large-parameter models consume the most computing power, and computing power is also the scarcest and most precious resource.

According to LatePost, although ByteDance used to be the most resolute company investing in AI, its current computing power reserves can no longer support the multi-line parallel R&D mode. Concentrating resources to promote a small number of core projects has become a more realistic choice.

A family with many children, and with a