Yao Shunyu overhauls Tencent's multimodal development roadmap: drawing closer to Liang Wenfeng and distancing himself from Li Feifei

Another personnel change has taken place in Tencent Hunyuan Multimodal Team. According to media reports, Lin Xudong, former head of multimodal understanding at xAI, has left xAI and joined Tencent Hunyuan to serve as the head of multimodal content generation algorithms.

This personnel news is noteworthy as it comes against the backdrop of ongoing adjustments to the Hunyuan multimodal team.

Over the past period, there have been successive internal news at Hunyuan about the departure of team leads, job transfers of researchers, and the onboarding of new members.

Hu Han, the former head of multimodal understanding, left to start his own business, Tian Yonglong and others joined Tencent, and the reporting lines of the original multimodal team have also changed following the integration of the large language model department and the multimodal model department.

However, it is not yet possible to draw a direct conclusion that Tencent's multimodal team is undergoing a complete reshuffle.

What can be confirmed from public information is that Hunyuan has indeed seen personnel mobility and organizational restructuring. The rumor of Lin Xudong's joining is more like a new signal in this round of adjustments: Tencent is recombining the two lines of multimodal understanding and content generation.

So the question arises, what is Lin Xudong's background, what capabilities can he bring to Tencent, and whether Tencent's multimodal route is shifting from "generating content" to Yao Shunyu's preferred direction of "understanding context and acting in the physical world"?

What is Lin Xudong's background and what can he do after joining Tencent?

Public information shows that Lin Xudong graduated from Tsinghua University in 2018, and then went to Columbia University to pursue his doctoral degree.

During his PhD studies, his research directions included embedding learning, video analysis and generative models, and he also participated in the Vx2Text project jointly developed by Columbia University and Facebook AI.

"V" refers to video, "x" refers to unknown variables which can be sound, speech or even ambient noise, "2" stands for "to", and "Text" refers to subtitles. Its logic is to first convert different modalities such as video and sound into vectors similar to "language tokens", then feed them into the language model for unified fusion, and finally generate open-ended text through an autoregressive decoder.

Transformers can only understand tokens, so essentially AI cannot understand video and audio as file formats, let alone convert them into text.

For example, if a dog jumps into the water next to a swimming pool, the video recognizer (V) of Vx2Text will output keywords: dog, jump, swimming pool; the sound recognizer (x) will output: water sound, splash.

Although the product function of Vx2Text is "generation", the core difficulty of the product lies in "understanding".

Of course, Vx2Text does not simply "translate" the picture into a few sentences. The model needs to identify characters, objects, actions and events from the video, understand the temporal changes of these elements, and finally organize the visual information into language.

After graduating with a PhD, Lin Xudong joined DeepMind to participate in Gemini-related multimodal pre-training and post-training work. In 2025, he joined xAI. Public information shows that he was responsible for the multimodal understanding direction, and participated in the training of multimodal content understanding and generation models.

Now he joins Tencent Hunyuan and will be in charge of Hunyuan's multimodal content generation algorithms.

Lin Xudong's joining is not so much about improving the performance of Hunyuan's multimodal generation, as it is about solving a problem that plagues all multimodal systems — understanding.

In the past, generative models were more like image creators. Given a prompt, they could generate an image or a video clip. But once users put forward more complex requirements, the models would struggle to keep up. For example, characters change in long videos, the shape of objects is inconsistent before and after, and camera movement does not conform to spatial relationships, etc.

This is not because the model cannot generate, but because it does not stably remember and understand the physical world.

Therefore, placing Lin Xudong in charge of multimodal content generation is likely precisely because he is capable of "translating" multimodal content into forms that AI can understand.

Lin Xudong's joining can only be clearly seen against a larger background: Tencent Hunyuan is currently reorganizing its own multimodal route.

In January 2025, Hu Han, Tencent Distinguished Scientist, succeeded Liu Wei who had left the company earlier to take full charge of the R&D of Hunyuan multimodal large model, and also served as the Tech Lead of Tencent Hunyuan large model.

In the second half of 2025, following Tencent's internal organizational adjustment, he was transferred from the Multimodal Model Department to the "Frontier" advanced technology research group under the Large Language Model Department, with the title of head of the multimodal understanding direction, and his reporting line was changed to report to Yao Shunyu.

His actual status changed from "leader of an independent department" to "head of a specific direction in a research group under the Large Language Model Department". His scope of responsibility shrank from the entire multimodal large model to only multimodal understanding, while directions such as multimodal generation were separated out.

In March 2026, Tencent shut down its AI Lab which had been established for nearly a decade, and some of its personnel were merged into the Hunyuan Large Language Model Department to report to Yao Shunyu. After this adjustment, Jiang Jie, the former actual head of Hunyuan and Vice President of Tencent, officially completed the work handover with Yao Shunyu, and no longer concurrently managed AI Lab and Hunyuan.

In early July 2026, Tian Yonglong, former OpenAI researcher and Yao Shunyu's Tsinghua undergraduate alumnus, joined Tencent to be in charge of the vision-language model direction.

Shortly afterwards, Tencent TEG officially issued a document to shut down the Large Language Model Department and the Multimodal Model Department, and merged them to establish the "Foundation Model Department", with Yao Shunyu as the head reporting to Lu Shan, President of TEG. Tencent stated that this move aims to improve the efficiency of model R&D and collaboration, and explore the intelligence upper limit of full-modal models.

Subsequently, Hu Han was exposed to have left the company to start his own business.

Linus, former Amazon chief scientist, former chief scientist of JD Digits, former head of the applied vision team of Alibaba Tongyi Lab, and former head of Tencent Multimodal Model Department, changed from his original position parallel to Yao Shunyu to reporting to Yao Shunyu after this adjustment.

That's not all. Zhong Zhao, head of Tencent Hunyuan multimodal foundation models, who is in charge of the two generation lines of HunyuanVideo and HunyuanImage, is the actual leader of Hunyuan's multimodal generation direction.

On August 14, he posted a moment on WeChat, saying that "the upcoming version may be my final version in the HunyuanVideo and HunyuanImage projects", which is full of farewell in every line.

Lin Xudong may just be one of the new members in the Hunyuan multimodal reshuffle, but his joining sends a clear signal that Tencent's entire R&D direction for multimodal has changed.

What did Tencent do in the field of multimodal before this?

Previously, Tencent's language model and multimodal were two completely separate lines, and the two core products of Hunyuan multimodal are exactly the aforementioned HunyuanVideo and HunyuanImage.

HunyuanVideo is a text-to-video model. When it was first launched in December 2024, it became the world's largest open-source video generation model at that time with 13 billion parameters, supporting 5-second 720p output. It expanded its capabilities intensively in 2025: it launched image-to-video generation in March, launched customized generation HunyuanCustom and digital human driving Avatar in May, and released the 1.5 version in November, which compressed the parameters to 8.3 billion, supporting native generation of 5 to 10 seconds of 480p or 720p videos, and can output 1080p through super-resolution.

HunyuanImage started even earlier. It was the first Chinese native DiT architecture image model launched by Hunyuan in May 2024. In 2025, it iterated to the 2.0 version, the industry's first industrial-grade real-time image generation version. In September of the same year, it released the 2.1 version (17 billion parameters, native 2K resolution) and the 3.0 version (80 billion parameters, the first industrial-grade open-source native multimodal image generation model). In January 2026, it further launched the Instruct version with reasoning capabilities, supporting prompt self-rewriting and image-to-image editing.

There is another line called Hy 3D, which is a model dedicated to generating single 3D models. It open-sourced the 1.0 version together with Hunyuan-Large in November 2024, upgraded to 2.0 in January 2025, which is split into a two-stage pipeline of DiT shape generation and Paint texture synthesis. In June 2025, it fully open-sourced the 2.1 training code, and released 3.0 in September, which increased the modeling accuracy by 3 times with a geometric resolution of 1536³. It has now been iterated to version 3.1, and is widely used in e-commerce modeling, product design and 3D printing.

Behind these three pillar products, it may be that Li Feifei's spatial intelligence theory has influenced Tencent, or that the relevant knowledge Tencent mastered in digital twins many years ago can be reused with the support of AI.

Just like Zhao Yun following the trio of Liu Bei, Guan Yu and Zhang Fei, Hunyuan multimodal also welcomed its fourth product, Hy World 1.0, in late July 2025.

The so-called spatial intelligence, according to the statement of World Labs under Li Feifei, refers to enabling AI to perceive, generate, reason and interact with the world in three-dimensional space.

Language models learn the statistical structure of text, while world models need to learn the structure of space and time, such as how light falls on the surface of objects, what a garden would look like from a never-before-seen shooting angle, and how objects move after being subjected to external forces.

Tencent stated that Hy World 1.0 is the world's first open-source immersive 3D world generation model that supports simulation. Its technical route is to first generate a 360-degree panorama as a "world proxy", then decompose it into semantic layers for layered 3D reconstruction, and finally output layered Meshes that can be imported into game engines.

Since Tencent itself is very good at developing game engines and game content, Hy World has amazing expressive power, but it has also been ridiculed by netizens as "Tencent is unwilling to leave its comfort zone".

More than a month after the release of Hy World 1.0, Tencent released Hy World-Voyager, positioned as an ultra-long roaming world model based on camera trajectory control.

Hy World-Voyager solves a pain point left by version 1.0. Although the quality of the 3D world generated by version 1.0 is good, the roaming range is quite limited, and geometric drift and model penetration will occur after walking a short distance.

The idea of Voyager is to adopt a different technical route: instead of directly generating 3D assets, it first generates RGB-D dual-modal videos. Each frame outputs a depth map synchronously while outputting colors, and then uses the depth map to reconstruct 3D point clouds in real time.

All in all, Hy World almost perfectly restores Li Feifei's vision of spatial intelligence by leveraging Tencent's advantages in engine technology and art resources.

It is worth noting that the full name of the technical report of Hy World 2.0 is "HY-World 2.0: A Multi-Modal World Model for Reconstructing, Generating, and Simulating 3D Worlds", in which "Reconstructing, Generating, and Simulating 3D Worlds" is exactly the aforementioned description of spatial intelligence by World Labs.

However, not everyone agrees with Li Feifei, including Yao Shunyu. On the multimodal issue, Yao Shunyu seems to agree more with Liang Wenfeng's point of view.

Yao Shunyu endorses Liang Wenfeng's view

Li Feifei's view is that the world model, as part of spatial intelligence, is an important route to AGI, while Liang Wenfeng holds the opposite opinion.

An investor once asked Liang Wenfeng about the multimodal issue, and Liang Wenfeng replied: "We only focus on the main line of AGI — GPT, CoT, Agent. The AI field is very broad, but 3D, video generation and world models have little to do with the main line of intelligence."

DeepSeek has a product called DeepSeek OCR, which is almost the technical embodiment of Liang Wenfeng's multimodal concept. It treats the visual modality as a tool serving the language model, rather than an independent intelligence direction.

The core of DeepSeek OCR is called "Contexts Optical Compression". Its core idea is to render long text into images, compress the text into compact visual tokens through a visual encoder, and then hand them over to the language model for decoding.

Doesn't this look familiar? Looking back at Lin Xudong's Vx2Text mentioned earlier, Yao Shunyu may be learning from Liang Wenfeng, converting Hunyuan's existing multimodal foundation into such an understanding method, so as to better integrate it into the main line of the Hy model.

Many large AI companies including OpenAI actually adopt similar practices. Yao Shunyu once worked at OpenAI, and it is very likely that he established this direction during that period.

Yao Shunyu's first flagship product after joining Tencent is Hy3, on which he focused on reasoning, agents, long context and productivity tasks.

The official introduction states that Hy3 is optimized for tasks such as software development, office production, financial modeling, front-end design and game production, and continues to improve in tool calling stability, complex context processing and multi-turn intent retention.

The goal of Hy3 is not to "top all benchmarks", but to make the model make fewer mistakes in real products, understand context, and be able to continue completing tasks.

This product itself is very "Yao Shunyu-style".

On June 5, 2026, in a dialogue with Dowson Tong at the Tencent Cloud AI Industry Application Conference, Yao Shunyu made the judgment that the competitive barrier in the second half of AI development does not lie in model parameters, but in Context.

"We now seem to have a universal hammer that can hit any nail. The methodology has become very mature. On the contrary, finding truly valuable problems is becoming more and more difficult," Yao Shunyu said.

Also at that conference, Tencent announced the three-layer architecture for long-term AGI construction: foundation base (pre-training + post-training + infrastructure), product implementation, and cutting-edge exploration. Reasoning capability is a problem to be solved by the foundation base, and context capability is the intersection of product implementation and cutting-edge exploration.

Previously, in order to test whether the model can find answers only from the context without relying on existing knowledge, Yao Shunyu specially developed the context testing benchmarks CL-bench and CL-bench life.

It can be seen that Yao Shunyu, like Liang Wenfeng, believes that the main line is Agent