World Model, go all out to target Chinese people who are passionate about creating high-quality shareable photos and videos in their daily lives.
To fill the 9-photo grid on their WeChat Moments, Chinese people who have always been obsessed with taking great photos keep buying new equipment, studying shooting positions, and sometimes even ruin their good mood during trips.
Meanwhile, top scientists and capital are pouring billions of dollars into using "world models" to teach AI to understand the physical world.
These two seemingly unrelated things may soon collide head-on within the same viewfinder frame.
#01
World Models Start to Work on Generating Great Photos
Can you change the shooting position after taking a photo? WorldLabs, founded by Li Fei-Fei, demonstrates this possibility in its world model Atlas.
By inputting one or more reference images and specifying the position and angle of the camera, the model can generate a brand new picture. Users can raise the virtual camera, move it to the side, and try out perspectives that they did not capture during the actual shooting. The areas not recorded in the original materials will be inferred and completed by the model.
WorldLabs has made Atlas available through web applications and developer interfaces, and lists film and television production as well as visual effects as one of its application directions. A photo is no longer just the end point of a single shooting, but can also become the starting point for exploring infinite possibilities.
Consumer imaging manufacturer Insta360 has also put forward the long-term vision of Cameraman, which enables devices to automatically compose frames and capture moments just like a photographer.
In this vision, AI is responsible for understanding the scene and user's intentions and generating shooting strategies, the lens captures environmental information, and robots, drones and gimbals complete movement and lens control. Users put forward their demands, and the device takes over more work of finding shooting angles and executing the shooting.
The world model is one of the technical directions that Insta360 has laid out for this purpose. DiT360, PanoWorld and PanoLOG respectively focus on panoramic image generation, frame change prediction under camera movement and 3D spatial representation of large scenes, helping devices understand the real world and paving the way for trajectory planning and active creation.
One adjusts the photo perspective in the virtual space, and the other hopes to make real devices shoot autonomously. The two companies are exploring the use of world models in "generating great photos" from different directions.
The demand for taking photos is very common, but the judgment before pressing the shutter is not simple. It involves spatial geometry, light and shadow changes, people's movement paths and the choice of shooting positions. The seemingly trivial daily problem of "taking good photos" essentially tests the machine's spatial understanding and prediction ability of the real world.
#02
World Models Each Have Their Own "World"
Different from large language models that predict "the next word" through probability, world models try to build an internal representation of the environment and predict how it will change over time or with actions.
Antoine Moyroud, Partner at Lightspeed Venture Partners, once said: "Everyone agrees that world models are very important, but almost no one can reach a consensus on what they actually are."
Under the name of world model, several distinct "worlds" have already diverged.
Google's Genie series is biased towards generating interactive virtual environments. When users change their actions or observation directions, the model continuously generates corresponding images; WorldLabs founded by Li Fei-Fei puts more emphasis on the generation and reconstruction of 3D spaces.
AMI founded by Yann LeCun advocates predicting changes in abstract representations, capturing information that is useful for actions without having to generate every pixel in the future. From this line of thinking, rendering the world realistically and understanding how the world works are two completely different things.
Models and tools such as NVIDIA's Cosmos are oriented towards robots and autonomous driving, providing capabilities for world generation, prediction and synthetic data, directly serving industrial training and testing.
Although there is no unified solution yet, capital has poured real money into different technical routes. This year, AMI secured a $1.03 billion seed round financing; in November, AMD announced the completion of the approximately $8.2 billion acquisition of WorldLabs.
The goals corresponding to these sky-high investments are usually very grand: teaching robots to assemble in factories, enabling cars to cope with complex traffic, or building virtual worlds that can be continuously explored.
However, at present, the technology itself is still in the stage where the technical routes have not converged and commercialization is in the early stage of tackling key problems. Problems such as spatial consistency, physical causality prediction, and the transition from simulation to reality still need to be solved one by one.
Compared with these high-risk, long-cycle hard-core scenarios, helping ordinary people take a good photo seems extremely trivial in front of grand narratives. But precisely because the mass consumer-level "photo generation" scenario has a high fault tolerance rate, short feedback cycle, and clear payment demand, it may instead become a pragmatic entry point where world models can take the lead in achieving commercial verification.
#03
How Do World Models Transform Photography?
From travel photography and accompanying photography services, to compact point-and-shoot cameras, panoramic cameras and pocket gimbals, consumers have been spending real money to "lower the shooting threshold and increase the rate of getting satisfying photos".
What world models try to expand is exactly the adjustment space left by existing imaging tools.
The common "shoot first, frame later" feature can adjust the composition by cropping, or select images of different orientations from panoramic photos, but the shooting position is still fixed. To get around the occlusion of foreground objects, or change the relative position of the person and the distant mountain, the device usually needs to move on site.
World models can generate images from another shooting position based on existing photos, so that composition adjustment is no longer limited by the original shooting position. For autonomous photography devices, this kind of prediction can also compare different perspectives in advance, then plan the movement and complete the shooting.
However, there are still several thresholds before achieving "effortless great photos" for ordinary people.
The first is cost and efficiency. The computing resources, waiting time and usage fees required to generate new perspectives will affect whether it can change from a novel experience to a daily tool. For autonomous photography hardware, factors such as battery life, obstacle avoidance, and the burden of carrying and setting up also need to be considered.
If you have to wait for a long time and pay an extra generation fee just to change the shooting position, consumers may find that asking people around you to "move two steps to the left" is still a very cost-effective technique.
More importantly, taking photos is never just 3D geometric calculation. High-performing sample photos can teach AI common composition rules, but may not tell it which angle the person in front of it prefers and what expression they care about. The judgment accumulated by photographers through communication and trial shooting is probably much harder to directly "distill" than a set of "internet-famous check-in formulas".
Technological evolution often has unexpected landing points.
When NASA's Jet Propulsion Laboratory invented the CMOS image sensor, its target was planetary exploration in the deep space; decades later, its largest application scenario is not in outer space, but in the phone lenses that ordinary people hold up every day.
While world models are still making long-term efforts for general-purpose robots and autonomous driving, the seemingly trivial task of helping people "take great photos" may instead become the first window through which the technology is put into real use.
This article is from the WeChat Official Account "Zhengu Research Lab" (ID: zhengulab), written by Lu Jianguo, edited by Zhang Duo, and authorized for release by 36Kr.