Fei-Fei Li has brought the new Atlas, but she is still following the old path from 2024.

On September 1 local time in the United States, World Labs, the company founded by "Godmother of AI" Li Fei-Fei, released Atlas, its new-generation world model.

Atlas can process text, images, videos and 3D data at the same time. Different from models that mainly generate a single image or video based on prompts, Atlas puts the input content into a context with spatial positions, and then continues to generate new content according to the information it has already obtained.

This means that after a photo is input into Atlas, the camera can move to positions that were not originally captured. Several photos from different angles can be organized into a complete scene, and a video of a real space can be converted into new perspectives and 3D outputs.

World Labs divides Atlas's capabilities into four categories: camera-controlled generation, spatial reconstruction, spatio-temporal simulation and image generation. Atlas is currently in the early access phase and will be applied to World Labs' products such as Marble in the future.

01 New perspectives can be generated even from one single photo

The first change Atlas brings is the generation mode of images and videos.

Traditional generative models usually determine the content of a picture according to text, while Atlas can directly receive the position and angle of the camera. After users provide one or more reference pictures, they can specify a new camera position, and Atlas will generate the corresponding perspective based on the input content.

For example, if a photo only shows a copper-colored toy robot standing in a vegetable field, users can move the camera behind the robot. Atlas will generate the back of the robot based on the information in the original picture, as well as the surrounding areas that are not captured in the original picture.

This means that the model not only needs to process the pixels in the picture, but also judge the spatial relationship between objects, and infer what should be seen after the camera moves based on the existing information.

The camera path can also be directly added to the generation process. In the test demonstrated by World Labs, users can design a specific movement path for the camera, instead of only using text descriptions such as "move left" and "shoot down from a height". Atlas generates continuous pictures according to these paths.

This kind of control can be further extended to multiple reference pictures.

If two pictures correspond to different positions in the space respectively, Atlas can put them into the same spatial context, and then generate the missing content between the two pictures.

In the case demonstrated by World Labs, two scene pictures with no direct connection are placed at different positions, and Atlas generates doors, corridors and other transition spaces connecting the two scenes.

Therefore, the reference picture is no longer only used to determine the visual style of the final picture, but can also become part of the generated space.

In terms of video generation, Atlas can use 1 to 6 input pictures, together with manually designed camera paths, to generate videos with a maximum length of 1 minute and a resolution of 1440p.

Users first determine the scene and how the camera moves, and Atlas generates the whole process according to these conditions.

The same spatial control can also be used to generate multiple different camera paths. In the case demonstrated by World Labs, after inputting a small number of pictures, different routes can be designed in the same scene to make the camera pass through different areas, or change the movement speed, length and complexity.

No matter how the camera changes, the spatial relationship in the scene needs to remain consistent.

Image generation is another capability of Atlas, but it is not the main direction that World Labs emphasizes.

Atlas can generate pictures and 360-degree panoramas based on text, process complex prompts, generate text in pictures, and support multiple visual styles. The cases demonstrated by World Labs include movie frames, interior design, game scenes, and panoramic views of fairy tale villages generated from text.

The real difference of Atlas lies in that all these generation results are built on spatial relationships.

02 Reconstruct from real scenes, and then let robots enter the scenes

If the previous part solves the problem of "generating new pictures from existing information", spatial reconstruction solves the problem of "restoring the real world".

Atlas can reconstruct a scene from one or more real-world pictures. The fewer input pictures there are, the more areas the model needs to supplement by itself. After the number of input pictures increases, the model obtains more real spatial information, and the parts that need to be inferred will decrease accordingly.

World Labs demonstrates this process with gardens, houses and cabins.

When there is only one photo of the garden, Atlas can reconstruct the garden in the photo, but other areas still need to be inferred by the model. After adding the second photo of the cabin, both the garden and the cabin correspond to the input pictures, but inferences still exist in other parts. After adding the third photo of the house, the scene can be restored more completely.

Atlas can adjust the reconstruction result according to the number of inputs. World Labs says that in some cases, two or three pictures can get a relatively faithful reconstruction result, and the model can also process more than 100 input pictures.

The reconstruction result is not just a set of new pictures. Atlas can output point clouds and 3D Gaussian Splatting. When inputting one picture, the model can generate new perspectives and the corresponding 3D geometric structure at the same time. When inputting a real space video, it can predict the depth of each frame, and then combine this information into a 3D reconstruction result.

For areas that are not captured by the camera, Atlas can still complete the missing parts.

World Labs says that the Gaussian Splatting generated by Atlas can be rendered at a high resolution and frame rate on the device side, and this representation is consistent with the company's Marble product.

With the 3D space, Atlas further processes the temporal changes.

World Labs uses ordinary mobile phones and action cameras to shoot real scenes, and then uses 3 to 5 camera positions to reconstruct the space. In this way, even if there is no camera shooting from a certain angle originally, this angle can be regenerated in post-production, so as to achieve an effect similar to "bullet time".

This process does not require professional photographic equipment. Engineers and researchers from World Labs use ordinary mobile phones, tripods and small clamps that can be put into backpacks to complete the shooting.

The same capability can be applied to robot simulation.

In the Real-to-Sim process, Atlas first reconstructs the space from the video of the real environment. Then, when the simulated robot moves in this environment, Atlas can generate the RGB images and depth data that the camera on the robot body may capture.

The two large environments demonstrated by World Labs are each reconstructed using only 24 frames from the mobile phone video. After the reconstruction is completed, different robots can be simulated to move along different paths, and the pictures from the robot's perspective can be generated.

Robot operation tasks also require simulating the changes between objects.

Atlas can assist in building robot operation simulation from a small number of real environment records, and process rigid objects, movable parts and deformable objects. After the simulation environment is built, you can change the position of objects, robot movement, lighting and background, and generate different conditions from the same real scene for robot training and testing.

In this way, the processing object of Atlas expands from a single picture to a space that can be observed, moved and modified.

03 Why can one model handle all these tasks at the same time

The core reason why Atlas can integrate generation, reconstruction and simulation into one model is that it does not process images, videos and 3D information completely separately.

World Labs says that Atlas is a multimodal autoregressive diffusion Transformer trained from scratch. It can currently process text, images, videos, camera poses and 3D depth maps.

Among them, spatial context is the core of the whole architecture. Atlas encodes the input content into context, but images do not exist in isolation. Each image corresponds to a 3D position, and images and depth maps can also be associated with clear camera poses.

When the model generates new content, it can refer to both the input content and the position of these contents in the space.

Videos can be represented as continuous image sequences in Atlas. Different tasks can be organized into different data sequences: the input is followed by the output, and the model generates new elements one by one based on the previous information. This is the "autoregressive" part of Atlas.

The "diffusion" part adopts the rectified flow model to generate high-dimensional continuous data such as images and videos through gradual denoising. World Labs says that by adjusting the denoising steps, a trade-off can be made between generation speed and quality.

Transformer serves as the basic architecture of the model. World Labs believes that this architecture can make full use of the technologies in large language models, image models and video models at the same time. The autoregressive Transformer can use technologies such as KV caching, cache-aware routing and disaggregated serving; the diffusion part can use methods such as diffusion distillation, classifier-free guidance, shifted noise schedules and VAE.

World Labs has released two quantitative tests.

The first test is camera-controlled generation. The test gives the model one input picture, and then asks it to complete 1 to 3 cinematic camera movements, including pan, truck and crane. Atlas directly receives the camera path, while other models describe the camera movement through text. Third-party human evaluators then judge whether the generated result is more consistent with the specified camera path.

Atlas won a 75% vote rate when compared with MiniMax H3, 81% when compared with Gemini Omni Flash, 86% when compared with Happy Horse 1.1, 93% when compared with FLUX 3, and 94% when compared with Seedance 2.5.

World Labs says that the more complex the camera trajectory is, the more obvious Atlas's advantages will be.

The second test is 3D reconstruction. The test provides the model with a set of pictures with camera poses, and requires the model to predict the 3D point corresponding to each input pixel. The comparison objects include Pi3X, π³, VGGT-Ω 1B, Depth Anything 3 and MapAnything.

According to the unified test protocol, the average point cloud reconstruction error of Atlas is 25.3‰, which is the lowest in this group of tests. A lower value of this indicator means better performance.

World Labs does not use a single indicator to evaluate all capabilities of Atlas. The company says that the world model involves generation, reconstruction and simulation at the same time, and it is difficult to measure it completely with one test. Therefore, it currently focuses on releasing the two results of camera-controlled generation and 3D reconstruction.

In addition to the test results, World Labs also emphasizes the scalability of Atlas.

During the development process, World Labs trained Atlas models of different scales and with different training computation volumes, and observed that as the training computation increases, the model gains new capabilities. World Labs predicts that after continuing to expand the training scale, the future world model will still improve along this direction.

04 A different path from Seedance

Although Atlas uses the Transformer architecture, its mode is completely different from that of generative models such as Seedance, and its capabilities are almost vertically complementary.

Seedance acts as a director, focusing on how a segment of picture evolves. Atlas is more like a set builder: it first creates a space that makes sense when viewed from any angle.

Video generation models of the Seedance type learn from a large number