Creating a world with AI, the real challenge lies in the second step.
Generating a world is only the beginning,
making it withstand continuous modifications is the real hard battle.
Generative AI is expanding from text, images and videos into software, games, 3D spaces and simulation environments. The tasks faced by models are also changing accordingly: objects need to maintain their identities, rules must withstand operations, and the world must be continuously updated based on action outcomes.
Liu Zidong, founder of Habitat Tech, believes that AI "world creation" is entering a more difficult second stage: shifting from generating viewable worlds to maintaining states, simulating changes and planning actions. World Labs summarizes the relevant capabilities as Renderer, Simulator and Planner, and the state layer that connects simulation and planning determines whether a world can operate stably for a long time.〔1〕
Looking further, large-scale real interaction also provides a practical path for Recursive Self-Improvement (RSI): the system continuously accumulates "state-action-feedback" trajectories, identifies capability gaps and generates new training tasks. Spatial worlds with clear objects, executable actions and verifiable results provide a natural engineering foundation for this kind of self-evolution.〔2〕〔3〕
1. AI deliverables are evolving from content to worlds
3D scenes not only test generation effects, but also require the world to maintain consistency of objects, relationships, rules and states in subsequent operations.
1.1 · 3D scenes become a comprehensive stress test
Create a runnable and playable 3D world with one sentence
The procedural open world of Kimi K3, the 3D games and Unity scenes built by GPT-5.6 and Codex, as well as experiments such as Fable 5, Factorio, and browser CAD, are all pushing the frontier of "generating a runnable world with one sentence". 3D scenes simultaneously test language understanding, spatial reasoning, long-term programming, tool invocation, asset organization and physical execution; failure in any link may result in empty scenes, missing objects, model penetration or state loss.〔4〕〔5〕〔6〕
More importantly, models are beginning to transition from "one-time generation" to continuous operation. Works such as MiniTown, Material Lab and Voxel Velocity have shown that with the help of code and tool interfaces, models can continuously organize objects, rules and feedback; in PlayCo's Unity experiments, structured invocation also significantly reduces Token consumption and iteration times.〔7〕〔5〕〔6〕
When the 3D world enters the time dimension, the core of the problem also changes accordingly: whether the identity can be maintained after the object moves, whether the passage is still available, and whether a local modification will break other relationships. Once the scene starts running continuously, state maintenance rises from a back-end engineering problem to a core intelligent problem.
This shows the current capability boundaries of models. Different cases use different calibers, and no horizontal performance ranking is made.〔12〕〔17〕
1.2 · From viewable to runnable
The perceptual and symbolic routes are converging in the action closed loop
World models are generally developed along two routes: one generates continuous worlds from visual representations, and the other builds executable worlds from objects, codes and rules. As systems enter continuous interaction, the two are converging rapidly.〔29〕〔30〕〔31〕〔32〕〔33〕〔34〕
- Perceptual route
Genie 3, World Labs Marble, Odyssey: Generate subsequent observations from pixels, videos or visual Latent, emphasizing realism, continuity and roamable experience.
- Symbolic route
Kimi K3, GPT-5.6/Codex, Fable 5: Generate objects, codes, rules and engine states, then execute, render and continuously modify.
The focus of the industry is also shifting from "how the world is represented" to "whether the world can run continuously". Future systems need to form a closed loop of observation, state, action and feedback.〔1〕
Observation is responsible for presentation, and state is responsible for calculation.
The same table, two types of compression. Perceptual representation preserves continuous appearance; structured state representation maintains components, connections and task structures.
1.3 · The engineering chain behind one-sentence generation
A world is maintained by a long-running engineering chain
"One-sentence generation" is only the entry point. The system needs to continuously maintain objects, relationships, geometry, rules and operating states inside: LLM understands goals and constraints, the spatial model completes solving, the engine executes results, and the state layer writes changes back to the world. Each modification inherits the existing state instead of regenerating everything.
This mechanism can be simplified as R–T–V:
- R · Representation:
Describe the objects, relationships and states of the current world.
- T · Transition:
Describe how the world changes after an action occurs.
- V · Verification / Value:
Judge whether the new state is valid and can continue to run.
LLM judges "what to want", the dedicated spatial model calculates "where to place it and how to make it valid", and the state infrastructure is responsible for storage, solving and verification.
Whether world generation can move from demonstration to product largely depends on whether this engineering chain can run stably for a long time.
Language, state, solving, execution and verification collaborate around the same world.
After code generation, rules and VLM continue to verify the scene.
2. Technical bottleneck: After world generation, both speed and state are difficult to maintain
Current world models can already generate amazing 3D results, but productization is still stuck in two more basic problems: how long it takes for the result to be truly usable for the first time, and whether the world can maintain the same set of objects, relationships and rules after continuous modifications.
2.1 · Common state gaps for the two routes
The result can be valid, but the world may not remain valid
Two types of world models will expose different forms of state drift after entering continuous operation.
- Perceptual route
The picture can remain realistic and continuous, but there are often no stable constraints on real scale, object identity and occluded areas. When the camera rotates, returns after leaving or continues to expand the scene, the scale may drift, and the space outside the line of sight will gradually blur and distort.
Typical errors of pixel world models: scale drift, the space outside the line of sight will become blurred and unstable.
- Symbolic route
Although objects, coordinates and codes exist explicitly, collisions, boundaries, orientations, topologies and cross-object relationships are often scattered in different scripts and tools. After local modification, the object still exists, but it may have crossed the boundary, overlapped or broken the original relationship.〔34〕〔36〕〔37〕
Typical error cases of layoutGPT: successful object generation does not mean that the spatial relationship is valid.
Both types of faults ultimately point to the same gap: the system lacks an independent, verifiable and incrementally updatable world state. Evaluations such as WorldCoder-Bench, GameCraft-Bench, FloorplanQA and GEST also repeatedly show that the code can run and objects can be generated, which does not mean that the whole world is still valid after multiple rounds of operations.〔41〕〔42〕〔43〕
The real challenge is therefore no longer just "generating correctly", but ensuring that the world remains consistent after modification and operation.
2.2 · Speed bottleneck of the first valid result
Generation is only the starting point, inspection and rework eat up response speed
Product experience really depends on the first valid result, not the first return of the model. If the time consumed for one "generation-inspection-repair" is (T_{round}) and the result acceptance rate is (p_{accept}), then:
T_valid ≈ T_round / p_accept
Public cases have shown this gap: World Labs Draft can get a draft in about 20 seconds, a complete World takes about 5 minutes, and high-quality Mesh is close to 1 hour; other complete scene processes generally take several minutes to tens of minutes.〔39〕〔34〕 The real speed bottleneck is not only generation, but also inspection, repair and rework. The more you rely on recalculating the whole process, the harder it is to enter high-frequency interaction.
Public cases show that obtaining a usable scene still requires significant generation and repair costs.
3. Technical trend: From one-time generation to state-action closed loop
Speed and maintenance bottlenecks are driving world models to restructure the technology stack: Renderer, Simulator and Planner are connected through an independent state layer, the scene becomes a compilable program, and generation and editing become spatial actions that can be learned, verified and replayed.
3.1 · The scene middle layer is taking shape
World generation is shifting from "direct output" to "compilation and execution"
An increasingly clear technical trend is that models no longer directly convert natural language into final 3D results, but add a layer of computable, modifiable and verifiable intermediate scene representation between the model and the engine.
The model understands the intention, the scene middle layer organizes objects, geometry, topology and constraints, and the engine is responsible for execution, rendering and simulation. This division of labor allows local modifications to not regenerate the whole world, but only update the affected parts, just like modifying a program.