66-year-old Yann LeCun released a new paper challenging the Scaling Law: the model with 109 million parameters outperforms the 72-billion-parameter one, and a larger model does not necessarily understand the world better.

66-year-old Yann LeCun has returned to the front line of AI research with a new paper.

This time, he has targeted a deep-rooted concept in the large model industry: to make AI understand the world better, people have to continuously increase parameters and expand training data.

On October 29, LeCun and Randall Balestriero, Assistant Professor of Computer Science at Brown University, jointly released a 37-page new paper titled Proving and Improving the Platonic Representation Hypothesis with JEPAs. The paper proposes that under specific conditions, AI models of different modalities can learn mutually alignable world representations even if they are trained independently.

What is even more remarkable is a set of figures. A text JEPA with only 109 million parameters trained by the research team outperforms large language models including Qwen2.5-72B and Llama-3-70B in cross-modal representation alignment tests.

The parameter scale differs by about 660 times.

In another experiment, the model adopting the new method only needs 21.6 million training samples to reach the representation alignment level that the next-token prediction model can only obtain with 900 million samples. The number of samples differs by about 42 times.

These results easily lead people to a question: In the past few years, the AI industry has invested huge sums of money to train larger and larger models. Could it be that it has taken the wrong direction?

But this is not the conclusion that the paper really intends to present.

LeCun's New Paper Challenges "Scale-Only Theory": Why Can a 109 Million Parameter Model Compete Against a 72 Billion Parameter One?

In this paper, the authors LeCun and Balestriero aim to answer a more fundamental question than model size: when AI observes the world through different means such as text and images, what exactly determines whether they can form similar internal understandings?

For LeCun, who has long advocated the world model approach, this is also an important theoretical exploration.

The starting point of this research comes from a philosophically intriguing hypothesis proposed by the MIT research team two years ago.

In 2024, Minyoung Huh, Brian Cheung, Tongzhou Wang and Phillip Isola published The Platonic Representation Hypothesis at ICML.

The researchers observed an interesting phenomenon: as the capabilities of different models increase, the internal representation structures of image models and language models seem to be becoming more and more similar.

For example, people can use a photo to describe a scene where a cat is lying on a sofa, and they can also use a line of text to express the same meaning. Pictures are made up of pixels, and text is made up of tokens. The two look completely different, but they point to the same real-world object. If AI has truly learned some shared structure behind these contents, the internal representations of the relationships between things in different models may gradually converge.

This is exactly the core of "The Platonic Representation Hypothesis": different AI models may be converging toward a common statistical representation of the real world.

The 2024 study observed that the increase in model scale is often accompanied by the convergence of cross-modal representations, but it did not prove that this phenomenon will definitely occur, nor did it explain what training methods can achieve it more efficiently.

LeCun and Balestriero tried to take a step forward. They established a mathematical framework in the paper, treating different modalities as observations of the same underlying world state, and studied under what conditions models can learn mutually aligned representations.

The answer is related to the positive sample pairs constructed during the training process.

The researchers refer to a data relationship as Good Views, which can be understood as valid observations of the same scene.

For example, two photos of the same scene, or two paragraphs of text describing the same scene respectively, although the specific contents are not completely the same, they both point to a certain state in the real world. In contrast, the paper uses Bad Views to describe another type of training sample construction method: for example, the prefix and subsequent content in the same text, or different fragments obtained by occluding the same input.

The difference between the two is that the former is more conducive for the model to extract the shared information behind different observations; the latter may make the model spend more representation capacity to process the unique details in a single modality.

The paper proposes that when different JEPA models are trained through positive sample relationships that meet specific conditions, even if they process different modalities, they can learn representation spaces that differ only by a rotation transformation.

The so-called "rotation" does not mean changing the direction of the image, but performing mathematical transformation on the high-dimensional vectors inside the model, so that the representations of different models can correspond to each other. This is a mathematical conclusion with clear prerequisites, rather than proving that all existing AI models already have exactly the same understanding of the world.

The research team then verified this theory through experiments. The paper compares text models and visual models that adopt different training strategies, focusing on observing how close their internal representation structures are. The results show that the text JEPA trained with Good Views performs outstandingly in cross-modal representation alignment.

The specific cross-modal representation alignment index reported in the paper, the higher the score, the closer the model representation structure is, and it is not the comprehensive capability score of the language model.

Among them, the 109 million-parameter text JEPA scored 0.184, higher than Qwen2.5-72B's 0.154. This means that in this test, a small model with only about one six-hundredth of the parameters of a large language model has instead learned a representation structure closer to that of the visual model.

However, it should be noted that the "surpass" here has strict boundaries. It does not mean that JEPA surpasses Qwen2.5-72B in mathematical reasoning, code generation or knowledge question answering capabilities. The paper compares cross-modal representation alignment, not the general large model capabilities familiar to developers.

Another important experiment involves training efficiency.

The researchers found that under the same corpus and comparison settings, the Good Views training scheme can reach the representation alignment level of next-token prediction using 900 million samples with about 21.6 million samples.

If calculated according to the number of samples, the efficiency is improved by about 42 times; but if calculated according to the actual number of text tokens read, the gap is about 14 times.

This shows that letting the model learn what kind of information relationships during training may be more important than simply letting it process more text. What is more interesting is that the research team also tried to convert this representation alignment capability into practical applications.

They connected the separately trained visual encoder and text model through a linear transformation to build an image captioning system. On the COCO dataset, this system achieved a CIDEr score of 0.838. In another experiment that does not require paired image-text data to fit the alignment transformation, the model obtained 64% on the Top-5 classification metric of ImageNet-100, surpassing some language model schemes compared in the paper. This makes the research results no longer limited to mathematical proofs, and also demonstrates the practical application space that cross-modal representation alignment may bring.

Leaving Meta to Start a Business at 66, Why Does LeCun Still Stick to JEPA?

Behind this paper is actually a continuation of LeCun's own research experience.

LeCun was born in France on July 8, 1960, and is already 66 years old this year. He once served as Meta's Chief AI Scientist, and is also one of the pioneers in convolutional neural network research. He jointly won the 2018 ACM Turing Award with Geoffrey Hinton and Yoshua Bengio.

In today's large model industry, LeCun is a somewhat special figure. He is not only one of the important founders of the deep learning wave, but also a researcher who has long held reservations about the current mainstream large language model route.

As early as the 1980s, LeCun began to explore the application of neural networks in computer vision. He promoted the development of convolutional neural networks, enabling machines to recognize handwritten characters and image content more effectively. Later, these studies became an important foundation for modern computer vision and deep learning.

In 2013, LeCun joined Facebook and participated in the founding of FAIR (Facebook AI Research). After that, he worked at Meta for about 12 years, successively serving as the founding head of FAIR and Chief AI Scientist. But with the large language model boom triggered by ChatGPT, the divergence between LeCun and the industry's mainstream technical route has become more and more obvious.

He does not deny the practical value of LLMs, but repeatedly emphasizes: relying only on predicting the next token is not enough to build an intelligent system that can understand the physical world, remember environmental states, reason and plan complex actions. In his vision, future AI should not only learn language and knowledge through massive texts, but also learn the abstract structure of the world by observing the environment, just like humans and animals.

This is also the reason why he has promoted JEPA (Joint-Embedding Predictive Architecture) for many years. Different from the training method that requires the model to reconstruct input details, JEPA focuses more on learning abstract representations and making predictions in the representation space.

In November 2025, LeCun publicly announced that he was preparing to leave Meta to establish a new AI research company and continue to promote the Advanced Machine Intelligence (AMI) research program. He clearly stated at the time that the goal of the new company is to promote the next round of AI breakthroughs: to build systems that can understand the physical world, have continuous memory, and have the capabilities of reasoning and complex action planning.

In March 2026, AMI Labs completed a financing of about 1 billion US dollars. Unlike a large number of startups that focus on product development around existing LLMs, AMI Labs takes the world model as its core research direction.

This background adds an extra layer of significance to this paper.

After winning the Turing Award and going through more than ten years of research career at large technology companies, he did not choose to step back behind the scenes, but left the familiar research platform to participate in the establishment of a new company, and continued to promote a technical route that has not yet been proven to be able to replace mainstream LLMs.

There is a noteworthy persistence in scientific research: in an industry that pays more and more attention to model scale, capital investment and product release speed, LeCun is still trying to find the next breakthrough from the learning mechanism itself.

Although this paper has gained attention because of the name of Yann LeCun, the other author Randall Balestriero is also worth introducing.

Balestriero is an Assistant Professor of Computer Science at Brown University, whose research directions include self-supervised learning, deep neural network theory and representation learning. His research experience spans signal processing, deep network theory and self-supervised learning.

He received his doctorate from Rice University in 2021, and then worked as a postdoctoral researcher at Meta FAIR in collaboration with LeCun.

In 2023, Balestriero joined Citadel to work on financial time series prediction and representation learning. Later, he returned to academia to conduct research at Brown University.

Industry Comments: Some Marvel, While Others Question

After the paper was released, the most widely spread statement is undoubtedly "109 million parameters beat 72 billion parameters".

On Hugging Face, the paper entered the list of popular papers of the day. Co-author Balestriero also personally appeared in the comment section to introduce the research results.

He particularly emphasized that the 109 million-parameter text JEPA trained by the research team using Good Views achieved a score of 0.184 on the cross-modal representation alignment index, surpassing Qwen2.5-72B's 0.154 and Llama-3-70B's 0.151.

When Yann LeCun replied to comments related to this paper on X, he said that the reason why today's large language models can answer a large number of questions is largely because they have accumulated massive amounts of declarative knowledge during the training process. He wrote:

"Large language models have accumulated a lot of declarative knowledge. That is why they can answer so many questions, and that is why they need so many parameters, which in turn take up a lot of memory."

LeCun's views have also sparked discussions among netizens. Some people take a wait-and-see attitude towards this technical route. Netizen Sage Williams commented: "If your point of view