A math genius from Tsinghua University has jumped ship to OpenAI. He once led the development of SAM and Llama. The person in charge of Sora said: Welcome aboard!

Another top talent from Tsinghua University joins OpenAI!

Just recently, researcher Pengchuan Zhang, who graduated from the Department of Mathematics at Tsinghua University, worked at Meta FAIR for 3.75 years, and led multiple core projects on SAM and Llama, announced his departure.

His next stop is OpenAI, where he will engage in research in the field of World Simulation and Robotics.

He wrote on Twitter that he is very much looking forward to exploring how visual perception, world models, and robotics can be integrated to build true "physical intelligence".

Aditya Ramesh, one of the leaders in the world simulation direction at OpenAI and a senior leader of the Sora project, also left a message to welcome him immediately.

The shift from visual understanding to underlying world models is quite significant in itself.

So, who exactly is this researcher who has been at the core of SAM and Llama?

Who is Pengchuan Zhang?

In 2007, Pengchuan Zhang graduated from Nankai High School in Chongqing and was admitted to Tsinghua University, majoring in mathematics.

After graduating from his undergraduate program, he went to the California Institute of Technology (Caltech) in the United States to pursue a doctoral degree and obtained a Doctor of Applied and Computational Mathematics in 2017.

During his doctoral studies, he focused on the theory and application of machine learning and began to explore the potential of deep learning in the field of vision.

After graduating with his doctorate, he joined Microsoft Research and served as a principal researcher at Microsoft Research Redmond.

During this period, he led the research on computer vision and multimodal intelligence at the Redmond Laboratory.

He also promoted computer vision and multimodal intelligence projects across different organizations within Microsoft, including directions such as Alexandar Multi-Modal and Florence.

It is worth mentioning that he also joined the University of Washington in 2021 and has been serving as an adjunct assistant professor in the Department of Electrical and Computer Engineering ever since.

Since 2022, he has transferred to Meta FAIR (now Meta Super Intelligence Laboratory) to conduct research on computer vision and multimodal intelligence, with a total working experience of nearly four years.

During this period, he led multiple groundbreaking projects, including:

Serving as the project leader of Segment Anything 3 (SAM 3): SAM 3 is a unified framework that enables object detection, segmentation, and tracking in images and videos.

As the latest iteration of the SAM series (November 2025), this model introduces a unified framework that supports object detection, segmentation, and tracking in images and videos, achieving zero-shot generalization to arbitrary objects and scenarios.

Serving as the project leader of Llama 3 Visual Grounding, he led the development of the visual grounding capabilities on the input side, making Llama 3 the first open-source model to reach human-level performance on the Visual Commonsense Reasoning benchmark.

Serving as the project leader of Llama 4 Visual Grounding, building on the advantages of Llama 3, he further strengthened the expert-level image grounding capabilities (such as pixel-level positioning and complex scene understanding), which is regarded as a key differentiating feature of Llama 4 compared to GPT-4o.

These efforts not only enhanced Meta's competitiveness in generative AI but also contributed high-impact tools to the open-source community.

As of now, Pengchuan Zhang's Google Scholar citations have reached as high as 34,659!

One more thing

Below Pengchuan Zhang's Twitter post, a netizen couldn't help but ask:

Why is everyone suddenly joining OpenAI? I'm quite excited, but why?

This question is not out of place.

Since the end of 2025, many well-known figures in the industry have successively joined OpenAI -

including the top talent Lijie Chen from the Yao Class, Arvind KC, an executive at Roblox, Brendan Gregg, the author of "Systems Performance: Enterprise and the Cloud", and Barret Zoph, Luke Metz, Sam Schoenholz and others who have returned from the Thinking Machines Lab.

Meanwhile, the recent wave of departures from xAI has also made many people wonder -

Will OpenAI become the next stop for these top researchers?

In response, a netizen in the comment section replied quite straightforwardly:

Because (OpenAI) has computing power and world modeling infrastructure at the Sora level. Without these two, it will be almost impossible to build a truly high-level robotic system by 2026.

If this judgment holds true, then Pengchuan Zhang's choice may not just be a personal career shift.

It is also OpenAI's bet on the "world model + physical intelligence" approach this year.

Let's look forward to Pengchuan's work at OpenAI!

Reference links:

[1]https://pzzhang.github.io/pzzhang/

[2]https://scholar.google.com/citations?hl=zh-CN&user=3VZ_E64AAAAJ&view_op=list_works

[3]https://x.com/PengchuanZ/status/2026189659228012558?s=20

This article is from the WeChat official account “QbitAI”. The author focuses on cutting-edge technology and this article is published by 36Kr with authorization.