Feng Shuo: The Scaling Law of world models lies not in parameters, but in data.

To scale the world model, solving the data problem must come first, and the parameter scale is the subsequent consideration.

On September 17, Feng Shuo, Associate Professor, Doctoral Supervisor and Deputy Director of the Institute of Systems Engineering under the Department of Automation of Tsinghua University, shared a judgment different from the mainstream narrative in his speech themed *Scaling Law of the Physical World Model*:

"The starting point of the world model is not the ontology, but must be data."

In the past few years, discussions on embodied intelligence always start with robot ontology, VLA, end-to-end models and generalization capabilities.

But Feng Shuo raised an earlier question: Where does the data of the physical world come from? If data is always expensive and scarce, and robots cannot enter large-scale real scenarios, what can the world model rely on for continuous training?

We need to clarify the accounts first: where the data comes from, and who will bear the cost.

If this problem is not solved, it will be very difficult for the physical world model to truly scale.

01

Robots cannot "grow up for 20 years" in the real world like humans

Go is the most intuitive example to illustrate the problem.

The chessboard is limited, the rules are clear, and the state transition of what will happen next is very clear. AlphaGo can conduct a large number of internal searches and simulations before actually placing a piece on the board.

No matter how complex the game is, the cost of restarting it is not high. Characters can restart after death, time can be accelerated, and the environment can be reset repeatedly.

The physical world removes this layer of protection.

Time cannot flow back, equipment has costs, and action errors may even bring safety risks. It is also impossible for robots to collect data in real environments to be infinitely replicated with just one click.

Feng Shuo mentioned that humans can spend 20 years growing up, making mistakes and accumulating experience before stepping into a complex society. Robots cannot wait that long. We cannot let them "grow up for 20 years" in the real world before sending them to work in factories, homes or public environments.

Robots need to have a basic understanding of the world before leaving the factory.

When a human walks to a river, he will not solve fluid equations in his mind first. We observe the water flow, depth, riverbed and foothold, and judge whether we can cross it based on experience. After taking the first step, the feedback from our body will make us adjust our judgment.

This is how humans deal with the physical world.

In Feng Shuo's view, machines also need to first know what may happen and which step has higher risks, then make corrections based on feedback after entering real scenarios.

02

If large language models continue to grow larger, can they bypass the world model?

Someone at the scene put forward the question clearly.

Today's large language models and multimodal models have shown certain physical understanding capabilities. In the future, can we let larger models understand problems first, call codes and simulations, and then solve most tasks in the physical world?

Feng Shuo did not give a completely negative answer.

This path is possible to work.

The model can convert a physical task into a data problem, program problem or simulation problem. When encountering difficulties, it can allocate more computing power for deduction. Video models may also learn part of the physical laws from a large amount of visual data.

What he cares more about is how many detours this path will take and how much cost it will incur.

If the model has to go through layers of text, images, videos, symbols and programs to understand the physical world, the answer may be calculated, but the path is far too long.

Feng Shuo used Plato's "Allegory of the Cave" to describe this difference. What we see may only be the shadow cast by the physical world, not the law itself.

Humans have a more direct perception of the physical world.

When we pick up a cup, step on a step, or push open a door, we will not make a rigorous calculation every time. Many judgments come from experience and form an internal model; after taking action, we use feedback to correct it.

In Feng Shuo's vision, the language model and the world model are responsible for their respective fields: the former handles logic, symbols and reasoning, while the latter handles physical intuition, action prediction and real feedback.

Only when these two capabilities are connected can machines truly understand the world and take actions.

Thus the problem falls back to data: what can this model use for training?

03

What is really scarce is not robots, but scalable physical data

Computing power, algorithm and data are the three most commonly used elements to explain the growth of AI.

In the past, computing power continued to improve along Moore's Law; algorithms relied on long-term exploration from academia and industry; content such as text, code and images on the Internet provided massive training materials for large models.

The most troublesome part of the physical world is precisely that data cannot accumulate on its own.

Internet data is produced by humans over a long period of time and naturally precipitated. Physical data often needs to be collected specially: purchasing equipment, building scenarios, arranging personnel, designing tasks, and bearing the cost of wear and failure.

Feng Shuo believes that the core bottleneck of current embodied intelligence and physical world models is insufficient data scale.

The problem lies here: it is very difficult for robots to replicate the data flywheel of autonomous driving.

Feng Shuo compared the routes of Waymo and Tesla.

One side uses high-cost dedicated vehicles, sensors and test systems. Every additional piece of data collected requires continuous investment of manpower, equipment and time.

The other side relies on mass-produced vehicles. Even without high-level intelligent driving functions, the vehicles can still be purchased and used by users. The data generated by users in their daily driving can naturally flow back to the system.

The key for the latter path to achieve scale is that the marginal cost of data generation is lower.

Things are more difficult for robots.

A car without intelligent driving capability is still a car. A robot without sufficient intelligence often cannot even perform stable tasks, let alone be sold to real users on a large scale.

Without user scale, it is difficult for data to be generated naturally. Without sufficient data, it is difficult for the model to improve.

It is impossible to break this cycle just by buying a few more robots.

04

The path needs to be reversed: make data first, then build the model, and finally empower the ontology

In the past, people used to follow this order:

Ontology → Data → Model

Build robots first, then let the robots enter scenarios to collect data, and finally train the model with the data.

If the ontology is not good enough, it cannot enter enough real scenarios; if the data scale cannot be expanded, the model capability is difficult to improve; if the model is not mature, the ontology can hardly generate stable value.

Feng Shuo proposed a reversed order:

"Make data first, then build the model, and finally empower the ontology."

The ontology is of course important, but in the early stage, we should prioritize solving the data source problem.

What kind of data is worthy of being used to train the world model?

First, the marginal cost should be close to zero.

Feng Shuo raised a key question: If we do not build the world model, will this data still be generated naturally?

If yes, the data cost does not need to be fully borne by the world model project. Vehicles naturally generate driving data when driving in the real world. A truly sustainable data source should also be embedded in existing real scenarios and real businesses.

Second, the data must contain basic knowledge of the physical world.

It does not necessarily cover all tactile mechanics and complex interactions, but at least it must help the model understand spatial relationships, state changes, object movements, and the possible consequences of actions.

Feng Shuo specially mentioned that even if a complete dynamic model cannot be established temporarily, as long as the data can make the model form basic intuition about motion laws, this kind of data is valuable.

Third, the data source must be scalable.

At the beginning of the project, we need to consider whether this path can be scaled up 10,000 times or 100,000 times.

Immaturity in technical links can be improved gradually. But if the data source itself is stuck at an insurmountable physical cost, no matter how large the subsequent model is or how much financing is obtained, it will be difficult to achieve real large-scale application.

Therefore, for this Scaling Law, the first thing to look at is not the parameters, but the data.

Look at its marginal cost, whether it contains physical knowledge, and whether it can be continuously expanded.

05

Algorithms cannot be discussed separately from data

In the era of large models, many discussions start from model structure, parameter scale and training methods.

Feng Shuo also brought the algorithm problem back to data: algorithms should be designed around data.

If the data source is naturally expensive, narrowly distributed and difficult to expand, no matter how sophisticated the algorithm is, it will be difficult to support the model scaling for a long time.

Only after finding data with low marginal cost, sufficiently high density of physical knowledge and continuous scalability can the algorithm have space to build an effective training and iteration method around it.

Data determines whether this path can continue to move forward.

After the model is improved, it can help robots obtain better perception, prediction and action capabilities; when more mature robots enter more real scenarios, they may also bring back higher-value data.

To start this loop, the first step is not to blindly increase the number of ontologies.

First find a real entry point that can make data flow continuously.

This is also the starting point for the establishment of DenseAI: to bring the world model into real industrial scenarios, so that scenarios, data and models can truly form a closed loop.

06

From being able to disassemble an engine to being able to build a new one

Feng Shuo finally used a very specific metaphor: disassembling an engine.

The first stage is to understand a specific system.

After disassembling an engine, you can reassemble it completely.

The second stage is generalization capability.

After disassembling engines from different manufacturers and different models, you can understand the common physical laws among them and deal with systems you have never seen before.

The third stage is the creation capability.

You can not only understand and reproduce existing systems, but also design new systems that did not exist before and have better performance.

Robots today have not completed these three steps yet.

To get things done in the physical world, robots need to be able to speak, reason, and understand the environment, objects and consequences of actions.

Whether machines can achieve all these ultimately depends on whether there is sufficient data.

For the world model to move out of demos and into the real world, three questions must be answered first: Where does the data come from, who bears the cost, and how to expand the scale.

This is the Scaling Law mentioned by Feng Shuo: first find a path that can continuously generate real physical data, then talk about larger models and more robots.

DenseAI also hopes to follow this path, bring nearly ten years of scientific research accumulation of Feng Shuo's team to the front line of the industry, and make the world model truly become the capability base for physical AI to move towards the real world.