GTA 6 hasn't been released yet, but overseas users have already driven cars using Google's new model.

Hey folks, video generation models are no longer impressive enough these days.

Imagine you're riding on the back of a black dragon, sprinting through a valley at 140 km/h. You can hear the flapping of its wings and the howling wind crystal clearly.

But this is not a fixed unchangeable footage. You can switch perspectives or input commands midway to alter the scene.

If you get bored just watching the scenery, type "the castle fires fireballs and arrows" on your keyboard.

Just a few seconds after you press the Enter key, giant fireballs come flying out, and arrows shoot outward with whooshing sounds.

The two-minute footage has no cuts at all. It is fully generated by AI in real time while a human is operating it.

This is the capability of Genie 4.

In the wee hours of yesterday, Google DeepMind released its new-generation world model Genie 4. After watching several official demos, I can hardly believe it has only been three years since the famous "Smith eating noodles" AI-generated video went viral.

The entire overseas internet has been blown up now. Google CEO Sundar Pichai immediately shared the news, calling Genie 4 "the most realistic and interactive world model to date".

Many people share the same thought as Shi Chao: every time you open your eyes, AI has evolved once again. Some netizens even stated bluntly that "game engines are dead, long live world models".

However, Genie 4 is currently only open to a small group of users for testing. Apart from the official demos, almost all real-world test results are shared on X.

Why are people reacting so strongly this time?

If you compare it with the previous generation Genie 3 released last August, you will probably understand.

The continuous coherent generation duration of a single world in Genie 4 has been extended from 1 minute in the last generation to 2 minutes, but that is not the most critical improvement.

Did you guys notice that the earlier black dragon video has audio? It contains not only ambient sounds like wind and explosions, but also human voices.

This is another obvious difference from the previous generation: Genie 4 is Google's first real-time world model with native support for synchronized audio and video generation, all sounds are calculated in real time by the model based on the visuals.

Another important upgrade is that users can modify the generated world in real time.

Some attentive readers may ask, didn't the last generation Genie 3 already support "promptable world events"?

Indeed, in the official demo of Genie 3, when you wander on a prairie, you can add a horse rider or a hot air balloon to the scene via text, but the effect back then was relatively simple.

Genie 4 is far more responsive to commands, you can change the weather, move the camera, and all the newly generated elements integrate perfectly with the surrounding scene.

Take the official demo of a couple riding a motorcycle as an example: the tester typed "explosion happens in the background" and "giant alien tentacles that can destroy the world" in the input box one after another.

The next second the camera cuts, an explosion suddenly goes off in the distance, and alien tentacles start wreaking havoc everywhere.

Isn't this far more complex than watching a few hot air balloons on a prairie?

However, to handle more complex changes, Genie 4 alone is not enough. This time it gets great support from Gemini as a partner.

According to DeepMind, this system is called Hybrid World Models.

Simply put, Genie 4 is responsible for generating images and audio in real time, while Gemini works in the background to ensure the world develops following certain logical rules.

For example, if you suddenly send a command, Gemini will judge how to make adjustments based on the current scene.

Gemini Omni is mainly responsible for making the generated world more consistent with real life, including what the scene looks like and how the characters and objects inside should move.

Gemini 3.8 Flash takes on more complex tasks. It observes the real-time footage generated by Genie 4, breaks down the task into a series of steps, checks whether the generated content meets expectations, and makes timely adjustments if any changes happen midway.

Apart from handling more complex tasks, Genie 4 has also made considerable progress in physical simulation this time.

For common physical scenarios in the real world such as aerodynamics and object collision, it can simulate far more realistic effects.

To prove that Genie 4 truly understands physics, the official specially released a slow-motion test: a marble bust smashes down on a watermelon on the table. With a loud bang, the watermelon is instantly shattered, with pulp and juice splattering everywhere.

To sum it up in one sentence: the newly launched Genie 4 not only generates more realistic visuals, but also produces ambient sounds and character dialogues in real time, allowing you to control the changes of the scene.

However, according to our observation of the official demos, the AI-generated character speech still sounds quite unnatural, and the facial expressions are not very smooth either.

It's not that Shi Chao is deliberately picking faults. Overseas users have long been nitpicking the issues with a magnifying glass.

For the watermelon-smashing demo, some people pointed out that according to normal physical laws, the bust should tip over after smashing down, but that does not happen in the video. As for the demo set at a rural Japanese train station, the train conductor is speaking English.

It can only be said that the visuals look pretty convincing at first glance, but some details still cannot stand close scrutiny.

The official has also admitted that this is still a very early stage product. If the model runs for too long, or you frequently rotate the perspective and modify commands, objects and details in the scene may drift.

But we cannot deny that Genie 4 has very broad application prospects.

Take game development as an example. On X, some users who got the internal test qualification generated an open world similar to GTA, and even named their work "GTA Waymo".

It can also be applied in film and TV series production: you can first adjust the camera angles and character movements in the virtual world before actual shooting.

However, compared with game development and film & television production, the potential of Genie 4 in robot training may deserve more attention.

Why can't current robots even do simple tasks like folding clothes or carrying a plate properly?

Apart from the complexity of the real physical environment, there is another practical problem: it is very difficult to obtain high-quality training data.

To let a robot learn how to carry a plate, in most cases human demonstrators have to show the operation over and over again, and record all data including every movement and the force applied. This is not only costly and time-consuming, but the collected data cannot necessarily cover all possible scenarios in real life.

But Genie 4 provides a new solution: simulate various scenarios in the virtual world first.

This time Google teamed up with robotics company Apptronik for a trial, letting Gemini 4 Argon cooperate with Genie 4 to simulate a robot playing checkers.

They only gave a very general instruction: "Play checkers with the boy opposite, move one red piece."

As a result, Argon broke this single sentence into 22 action steps for Genie 4 to render, and commanded the robot to lower its head, stretch out its hand, pick up the red piece, lift its arm, move and place the piece.

If this path works out, household helper robots will finally become a realistic expectation.

Anyway, Google is not the only player targeting world models.

In the past two months alone, the world model sector in Silicon Valley has been extremely active: Runway launched GWM Worlds 2 that supports multi-player online interaction, AMD spent 8.2 billion dollars to acquire Li Fei-Fei's World Labs, and Amazon is also reported to be preparing to invest 7 billion dollars to acquire world model startup Decart.

The actual large-scale implementation of these models is still not on the horizon, but at this development speed, I won't be surprised no matter what new technologies come out in the next couple of years.

Shi Chao can increasingly understand Emperor Qin Shi Huang now. With so many fun new technologies coming out, I really wish I could live to be 100 years old...

This article is from WeChat official account "X.PIN", author: Xixi, editor: Jiang Jiang & Mian Xian, published with authorization from 36Kr.