Just now, Li Feifei has made a disruptive breakthrough! The world's first multimodal world model has been released, which can save hundreds of camera positions with only a few photos.

Not long ago, peer competitors were burning themselves out competing to generate short "electronic snack" videos that are only a few seconds long; now, World Labs founded by Li Feifei has completely overturned the entire industry landscape — 

Just now, Atlas, the world's first multimodal world model, was officially unveiled!

Official blog 🔗 https://www.worldlabs.ai/blog/atlas 

According to the official definition, Atlas is an omni world model oriented towards spatial intelligence, pre-trained from scratch. It can natively process text, images, videos, camera poses and 3D depth information, and complete world generation, spatial reconstruction and spatiotemporal simulation within the same model.

Let's start with the most eye-catching breakthrough operation: 

Camera Controlled Generation. 

In the past, when people used video generation models, the results were more or less a lucky draw. We humbly typed in the Prompt box: "Slowly pan the camera left and lift it, rotate around the character at a slight upward angle", and after pressing enter, we could only pray for a decent result. 

In contrast, the approach of Atlas is to directly take "camera pose parameters" as its underlying native input.

What angle do you want? What trajectory? What moving speed? 

Just input the coordinates directly! 

As long as you feed in 1 to 6 ordinary photos and set the camera movement trajectory, Atlas can generate high-quality clips of up to 1 minute long with 1440p resolution.

The generated footage does not break down, and the spatial geometry is smooth and consistent, which can be called cheat-level camera movement. 

What is even more impressive is its spatial "imagination capability". 

Input a photo that only captures the front of a robot? Atlas can perfectly complete and generate its full back view; 

Give it a partial shot of the corner of a swimming pool? Relying on its built-in "world common sense", it will silently complete the unshot lawn and distant mountains next to the pool for you. 

Judging from the official released Demo, Atlas has indeed made a big leap forward over ordinary video generation models in camera movement and spatial consistency. 

However, once the camera enters the area not captured by the original image, Atlas still actually relies on the model prior to guess. The larger the viewing angle span and the longer the generation path, the more likely local geometric drift, object shape change and texture flicker will be exposed.

In other words, what it generates is more like a visually reasonable 3D world, not necessarily an exact digital replica of the original world. 

Behind all these operations, there is an industry term called Spatial Context that cannot be avoided. 

Large language models understand the preceding text to generate the following content, while Atlas binds 3D coordinates and depth to each image, building a room with an axis grid in its "mind". 

To put it more bluntly, the underlying logic of many of Atlas's capabilities is inseparable from multi-view synthesis. 

It will put images and videos from different angles and different camera positions into the same 3D space together, first judge their relative positions to each other, then fill in the unshot areas, and continue to generate new perspectives. 

When a single image is input, it relies more on the model prior to "imagine" the world; once the number of input perspectives increases, it is more like performing a multi-view fusion with spatial constraints, gradually stitching scattered frames into a continuous, roamable 3D scene. 

Even if you casually throw two completely unrelated photos into the space, Atlas can forcibly imagine corridors, gates and rooms to seamlessly connect the two scenes together. 

Students from the directing department will fall into tactical silence after seeing this, and slowly type a question mark: So in the future, we don't need to shoot camera movements on site, we can just "boot up virtually" directly in the computer to get the footage done? 

In addition, Atlas has a second ace card: Spatial Reconstruction.

Traditional 3D Gaussian Splatting (3D GS) or point cloud scanning requires carrying professional equipment to circle around the target for dozens of times, shooting hundreds or thousands of photos, which is costly and labor-intensive. 

But Atlas shows its mysterious smile again: Come on, there are no so many rigid rules at all.

The official directly released a Stanford University Main Quad case: just input 2 to 25 ground-level flat photos taken by tourists, and it can easily restore all details of the lawn, arcade and the exterior wall of the Memorial Church. 

Then the camera rises directly from the ground, presenting an aerial roaming shot from a God's-eye perspective. 

On a number of public datasets such as DTU, ETH3D, and ScanNet, the sparse view reconstruction error (AbsRel ×10⁻³) is specifically tested: Atlas scored 25.3 (the lower the score, the better). 

In comparison, Pi3X scored 28.7, Depth Anything 3 scored 39.3, and MapAnything even scored as high as 47.7. 

Moreover, the output is directly connected to point cloud and 3D Gaussian Splatting, which can be seamlessly integrated into World Labs' own Marble platform for high frame rate real-time rendering. 

What is even more amazing is the "bullet time" style Reframing Video feature.

Without the Hollywood's ring array of hundreds of cameras, researchers can set up 3 to 5 ordinary mobile phones or action cameras casually, put them in their backpacks and leave; 

Shoot a daily playful scene, and Atlas can freely switch perspectives in the virtual space to reproduce the effect of *The Matrix* on the spot. 

Of course, the ultimate goal of Li Feifei and her company World Labs for this whole series of operations is definitely not to compete with Hollywood for the special effects industry. Her ambitious vision points to the biggest pain point of this generation of AI: 

Embodied intelligence and robotics.

People who are currently working on robot training all know that the biggest pain point of embodied intelligence is actually "insufficient virtual training grounds". 

Letting robots repeatedly make mistakes in real factories, warehouses and homes leads to slow data collection speed and high cost; if we switch to training in a simulation environment, we need to spend a lot of manpower to build 3D scenes, materials, lighting and objects in advance. 

The industry even has an estimation that if we completely rely on traditional methods to collect robot training data of sufficient scale, the final cost may reach the order of 100 trillion US dollars.

The Real to Sim route proposed by Atlas directly eliminates a large amount of manual work of building simulation worlds in the middle process. 

Shoot two 24fps videos of a factory or a room with a mobile phone, feed them to the model; Atlas will immediately output a high-precision 3D physical space, so that the robot can enter this twin space to frantically "explore the map" and make trial and error. 

Even when the robot moves to any position, Atlas can render the RGB image and depth image that the onboard camera should see on site. 

When the robotic arm touches a rigid box, pulls a hinged cabinet door, or squeezes a soft sponge, Atlas can simulate the force feedback for all these actions. 

As long as you record a real scene, you can derive millions of different lighting, placement and obstacle conditions. 

On the technical base, World Labs has launched an extremely hardcore combination of technologies this time: 

Multimodal Autoregressive Diffusion Transformer, Multimodal Autoregressive Diffusion Transformer. 

Breaking it down, it combines the strengths of the two current mainstream paradigms: 

Multimodal: Natively integrates text, 2D images, camera poses and 3D depth; 

Autoregressive: Just like language models predicting the next word, it predicts new spatial states step by step in the spatiotemporal sequence; 

Diffusion: Gradually denoises based on Rectified Flow to ensure the image quality and geometric accuracy of continuous high-dimensional signals; 

Transformer: Based on the most mature matrix multiplication structure as the base, it seamlessly adapts to existing large-scale computing power clusters. 

This architecture allows Atlas to simultaneously benefit from KV Cache and distributed inference optimization in the LLM field, as well as sampling distillation and prior guidance of diffusion models. 

In the human Blind Test for camera motion control, the result is a completely one-sided overwhelming advantage: 

Against MiniMax H3, Atlas has a 75% win rate; against Gemini Omni Flash, the win rate is 81%; against FLUX 3, the win rate directly rises to 93%; against Seedance 2.5, the win rate