Beyond Google and Fei-Fei Li, the third path of the world model

Many years from now, looking back on humanity's journey toward AGI, the year 2026 may well be marked as the "Year Zero of the World Model."

Recently, on the core Navi sub-leaderboard of WBench, the evaluation benchmark for interactive video world models, an "unfamiliar" name has quietly taken the top spot — HiDream-O1-World. With an average score of 80.9, it has drawn widespread attention from the industry. Developed by Zhixiang Future, its outstanding performance has once again put the world model track in the spotlight.

Overseas, Yann LeCun, a staunch believer in world models, founded AMI Labs after leaving Meta, and secured over $1 billion in financing relying solely on his industry influence and full commitment to world models. Later, Genie 3, the general-purpose world model pre-released last August, was officially unveiled. On January 30, Project Genie was opened to the public, allowing users to generate interactive 3D virtual worlds via text or images. World Labs, founded by "Godmother of AI" Li Fei-Fei, launched Marble, its commercial product focused on 3D world generation, and the company also completed over $1 billion in financing with a valuation exceeding $5 billion.

In China, on April 16, Tencent and Alibaba launched their respective world model products on the same day. Tencent released the open-source Hunyuan 3D World Model 2.0 (HY-World 2.0), while Alibaba rolled out HappyOyster, which features real-time interaction. A week later, ByteDance officially released its new-generation 3D generation large model — Seed3D 2.0.

For a moment, numerous players in the world model track are vying for dominance and making breakthroughs, yet each of them still has their own limitations.

Among them, Google and Li Fei-Fei's World Labs represent two major technical schools. Google focuses on real-time interactive world modeling, where users only need to press one key and the screen will respond accordingly. However, challenges still remain in the unified representation of multi-modal inputs and the joint generation of audio and video. In June this year, Li Fei-Fei systematically elaborated on the three roadmap directions of the world model: renderer, simulator, and planner. Marble from World Labs is exactly the first product in the simulator direction, following the route of structured 3D space reconstruction — generating structurally accurate and editable 3D spaces starting from 3D geometry. It pursues not photorealistic visuals, but structural accuracy and operability. This route has obvious advantages in the structural fidelity of static scenes, but it is still in the early exploration stage in terms of long-sequence dynamic generation and physical interaction.

Divergences in technical routes and the lack of industry consensus precisely mean that no player has locked in the final outcome yet, and every enterprise has a chance to take the lead.

On August 17, Zhixiang Future officially released HiDream-O1-World, a native full-modal interactive world model. The model took the top spot on the core Navi sub-leaderboard of WBench upon its debut. Notably, it ranked first with a score of 73.3 in the Physical dimension, and was among the top performers with a score of 88.0 in the Consistency dimension. This means that HiDream-O1-World has made key breakthroughs in the core technical challenges widely recognized in the field of interactive world models — spatio-temporal consistency and physical consistency.

Breaking Through in "Unbounded Roaming" and "Long-Duration Interaction"

This means that HiDream-O1-World is on a par with the current mainstream interactive world models.

The model has two core functions: roaming and editing, and supports multi-modal inputs including text, image, and control commands. In roaming mode, users can freely explore from first-person or third-person perspectives, with stable and coherent scenes free of drift and discontinuity. In editing mode, users can edit the screen in real time, use commands to adjust character actions or environmental changes in real time, with the screen responding synchronously and audio and video generated simultaneously. More importantly, the model supports multiple subjects such as humans, animals, and fictional characters, covers diverse styles from realistic urban street views to two-dimensional fantasy worlds, and boasts extremely strong generalization capabilities.

△ Model performance in online real-time editing mode, the video is generated by HiDream-O1-World

These features make HiDream-O1-World no longer just a video generator, but a dynamic world that users can enter and control.

But what truly makes HiDream-O1-World stand out from other world models is its groundbreaking capabilities on two widely recognized industry challenges.

The First Challenge: Spatio-Temporal Consistency, a "Master of Spatial Memory" for Long-Duration Sequences

Interactive world models have long faced a fatal shortcoming: after perspective movement and scene roaming, the position, structure, and occlusion relationship of objects become disordered, the screen "collapses more and more as you interact", and a continuous world cannot be formed. For example, when a person walks through a corridor with murals and then turns back, the murals may disappear or deform. This is because models that generate images through "pixel prediction" have to redraw the entire picture for each interaction, with geometry and appearance deeply coupled. Any tiny deviation will be continuously amplified in long sequences.

After conducting a "CT-like" scan of more than 20 interactive models, WBench found that the performance of all models deteriorates after continuous interaction, with the average navigation score dropping by a full 33 points. The cumulative pose error in each round is a structural flaw of the current iterative generation paradigm.

One of the core breakthroughs of HiDream-O1-World is that it can remember the structure of previously explored scenes during long-sequence interactions. Even after multiple rounds of perspective switching and navigation movement, it can maintain the geometric consistency of object size, spatial position, and occlusion relationship, fundamentally avoiding perspective drift and scene "reset" issues. This enables the model to have real spatial navigation capabilities for the first time. When users turn their heads, look around, look up, or look down from a first-person perspective, the response is sensitive and smooth, the picture is stable without drift, achieving the effect of "the world does not refresh when the person moves, and the scene does not reconstruct when the person turns back".

The Second Challenge: Physical Consistency, From "Guessing Pixels" to "Understanding Physical Laws"

Many current models are still competing on rendering quality and picture sophistication. But no matter how exquisite the picture is, if objects appear to float, pass through walls, or have distorted collision feedback, users will immediately be pulled out of the immersive experience. This is a common flaw of almost all current video generation models — they fit pixel distributions rather than physical laws.

In the WBench evaluation, the physical dimension is one of the dimensions with the lowest scores across all models. Different from ordinary rendering models, HiDream-O1-World has made substantial breakthroughs in the realism of physical simulation: in the visual rationality evaluation that assesses whether motion in the picture conforms to common-sense physics, the model's performance is 13.6% higher than the industry average; in the more challenging causal fidelity evaluation, which comprehensively measures physical dimensions such as fluid, collision, and deformation, the model has also achieved a significant increase of 12.7%. HiDream-O1-World took the first place in the industry in the physical dimension with a score of 73.3, proving that the model has real physical inductive bias and can independently deduce objective laws such as gravity, collision, fluid, and deformation.

Judging only from the demonstration videos, HiDream-O1-World still has shortcomings. But it has taken the lead in making a breakthrough on the two extremely tricky problems that have troubled the industry. The reason it can achieve all this lies in the fact that it has chosen a technical route different from those of Google and Li Fei-Fei's World Labs.

A New Solution: Separate "Geometry" and "Appearance"

The reason why HiDream-O1-World can achieve breakthroughs in spatio-temporal consistency and physical consistency lies in two layers of design: the unified cognitive framework laid by the native full-modal UiT architecture, and the paradigm innovation of geometry-appearance decoupled generation.

Traditional multi-modal models follow the "building blocks" route — an independent text encoder understands text, VAE processes images, and a diffusion model generates pictures, with each module operating independently. The UiT (Unified Transformer) architecture independently developed by Zhixiang and carried by HiDream-O1-World maps all signals including image pixels, text tokens, video voxels, and spatial relationships into the same shared token space, and interacts directly with the same set of Transformers. This means that when a user issues commands such as "turn left" or "walk forward", the model understands the spatial meaning of the commands, predicts the world state, and generates visual results in the same cognitive framework — "understanding" and "generation" are no longer separated at the underlying level.

If the UiT architecture solves the problem of "what framework to use to understand the world", then the generation paradigm integrating geometry-appearance decoupling solves the problem of "how to generate a 3D-consistent world". Most of the current mainstream 3D interactive generation models rely on implicit spatio-temporal representations to infer scene structures. After the model receives an image, a command, and a camera trajectory, it fits the pixel distribution and directly generates the video in a "one-step" manner. This geometry-independent modeling method, on the one hand, is extremely prone to problems such as unreasonable geometric structures, object deformation, and cross-perspective spatial inconsistency under large perspective changes. On the other hand, it directly fits the pixel distribution without understanding physical laws, so the model will expose its flaws once it encounters a physical scene that has never been seen in the training data.

The solution of HiDream-O1-World is that instead of letting the model implicitly "guess" all spatial reasoning, it is better to split "geometric structure generation" and "appearance rendering" into two independent stages, that is, the two-stage framework of "geometry-appearance decoupling".

• Stage 1 — Complete geometry generation first: HiDream-O1-World first pre-trains a 3D foundation model. After the user inputs a command, the 3D foundation model first extracts multi-scale geometric features, and compresses them into a representation space suitable for the video diffusion model through a lightweight projection module. Subsequently, these geometric features are used as explicit structural conditions to guide the subsequent video generation.

• Stage 2: Video diffusion model generation: Use the geometric representation generated in the first stage as the structural condition to perform conditional denoising in the VAE latent space, and finally decode it into video frames.

△ HiDream-O1-World combines the strong spatial structure prior of the 3D foundation model with the high-fidelity generation capability of the video diffusion model, so as to realize 3D consistent world modeling

The core of this design is the decoupling of geometry and appearance outputs. The geometry stage mainly focuses on structural completeness, cross-view consistency, and compliance with camera trajectories. The appearance stage mainly focuses on texture, visual details, and photorealistic rendering, without the burden of structural reasoning at the same time.

HiDream-O1-World introduces the spatial structure prior of the 3D foundation model into the generation process, coupled with the ability to dynamically adapt geometric constraints to unknown scenes during the inference phase (Test-Time Training online inference adaptation). On the one hand, it enables the model to maintain 3D geometric consistency throughout long-sequence interactions. On the other hand, the constraints of geometric structure provide underlying support for physical consistency — an object with a clear position, volume, and occlusion relationship in 3D space naturally follows physical laws such as gravity and collision. It tells the model that the world has geometric constraints, and objects cannot appear out of thin air, penetrate each other, or violate spatial logic.

The new geometry-appearance decoupled generation paradigm allows the model to "think through" the 3D structure first and then "draw" the picture when generating new perspectives, fundamentally solving the structural flaws of the traditional "one-step" generation method. This set of combined measures is the technical foundation that distinguishes it from Google's "renderer" route and Li Fei-Fei's third route.

It is reported that the paper of HiDream-O1-World focusing on spatial consistency research has been officially accepted by the European Conference on Computer Vision (ECCV 2026), one of the three top academic conferences in the global field of computer vision and artificial intelligence, opening up a technically feasible path with both theoretical originality and