More than a year later, Google DeepMind's Genie World series of models has finally entered its fourth generation.
Recently, Google DeepMind officially announced the latest generation of the Genie 4 world model, calling it "the most realistic and interactive world model to date".
Unlike previously only viewable AI videos, Genie 4 is able to generate dynamic environments in real time, where users can not only control the role or move through the lens on the keyboard, but also enter natural language at any time, change the weather, add objects, introduce new characters, and see how the whole scene changes with it.
More importantly, the world now has a voice.
From winds blowing through the woods, waves hitting the sand, to conversations between characters, Genie 4 is able to generate corresponding audio in real time while generating images, allowing virtual characters to not only move, but also react to their surroundings and user input.
Meanwhile, for the first time, Google is introducing the Gemini series models more deeply into the world simulation process: Big Language models are responsible for understanding tasks, planning events and maintaining scenic logic, and Genie 4 is responsible for presenting dynamic worlds in real time.
This means that Google is trying to solve a more complex problem than producing realistic video: how to make AI a world that can continue to function, respond to external interference, and follow causal logic as much as possible.
It's not only realistic, but the whole world can change in real time.
According to the official announcement, Genie 4 primarily enhances its capabilities in four areas: more realistic visual and physical simulation, more natural human action, real-time audio, and complex event reasoning supported by Gemini.
The most intuitive change is that the user not only enters a generated world, but can change it in real time as he explores it.
In the official presentation, a man stands on a suspended platform in the center of the lake playing saxophone. The surrounding forest is covered in fog and the lake reflects faint light.
The user can move the camera independently through the keyboard and observe the scene from different angles. The characters continue to play, and the environment around them continues to change.
If you want to change the world, you can also directly input natural language instructions, such as adjusting the weather, moving lenses, or adding new people and objects. Gemini combines these requirements to understand the current scenario and guide the simulation to continue.
In other words, users do not need to recreate the entire scene in order to continuously modify content in a continuously running virtual world.
In addition to visual effects, Genie 4 further enhances its ability to simulate physical phenomena.
For example, an oak seed that falls from the air and is affected by air resistance spins and falls in the air.
Google said that Genie 4 improved simulations of physical phenomena such as aerodynamics, collisions, and in particular improved the natural level of character movement, allowing characters to respond accordingly to their surroundings.
And this world is not just a picture, it's also a sound.
Genie 4 is Google's first model of the world to be able to natively generate real-time audio, supporting ambient sound, music, and personality conversations that can be adjusted in real time based on changing scenarios.
In a first-person perspective demonstration, the user rides a giant blackbird that travels through the mountains at 140 kilometers per hour. Accompanied by high-speed flight, the ears transmit wings that tap, high winds and distant rumblings.
These sounds are not later added sound effects, but are generated by Genie 4 alongside visual scenes.
Furthermore, the characters in the scene can also respond to real-time conversations based on user input or ongoing events.
Gemini is responsible for thinking, and Genie is responsible for generating:
World models are moving towards mixed architecture
However, if improvements in images, physics, and audio bring the world of Genie 4 closer to reality, then the addition of Gemini will allow the world to begin to have a more complex event development logic.
According to Google, the Genie 4 incorporates the capabilities of the Gemini series models, where the Gemini Omni simulates visual and behavioral knowledge of the real world, while the Gemini 3.8 Flash takes the lead in real-time generation.
Specifically, as Genie 4 continues to generate virtual worlds, Gemini 3.8 Flash continues to observe images, breaking targets down into multiple steps, checking if each step has been completed, and adjusting follow-up plans as the environment changes.
This allows the model not only to continuously generate the next image, but also to organize complex events around a specific target with a continuity.
Google also showed a robot simulation experiment.
In this presentation, Gemini 4 Argon, in collaboration with Genie 4, simulates the process of jumping international chess under the Apptronik Apollo humanoid robot.
Gemini 4 Argon is responsible for breaking down user requests into a series of steps, then guiding Genie 4 through real-time text instructions to generate the corresponding scenarios, allowing the entire simulation process to change as the mission progresses.
Google refers to this technology as Hybrid World Models.
Simply put, Gemini is responsible for logical reasoning, mission planning and event organization, and Genie is responsible for generating world vision, sound, and dynamic feedback. Combining the two, virtual environments are expected to be more coherent and controllable.
Of course, the value of this technology is not just in creating an interesting virtual world.
Google says that early testers have begun exploring Genie 4 for applications such as video previews, reconstruction of historical scenes, game prototyping, and industrial simulation.
For example, filmmakers can directly access generated scenes, adjusting shots, lights, lyrics, and character movements in real time; game developers have the opportunity to quickly experience a prototype that can be explored through natural language before writing game engine code.
The long-term goal is to use these controllable virtual environments to provide training and test scenarios for robots and agents to try and perform complex tasks before entering the real world.
Currently, Genie 4 is open to some trusted testers for research previews, which will be made available to Google AI Ultra users through Project Genie and to early partners at Gemini API, Google AI Studio.
How far is it from the real world of infinity?
Although Genie 4's presentation has been very surprising, it is still significantly distant from a virtual world that can run infinitely, fully in accordance with real-world physical laws.
Google DeepMind also explicitly acknowledges in its official blog that Genie 4 is still an early work.
The first is the long-term consistency problem, which is that while Genie 4 is able to maintain the consistency of the environment over a short period of interaction, the persistence of visual details and objects can still change or shift over time. These problems are especially noticeable when the simulation runs longer, after multiple complicated word-of-mouth modifications, or users continue to explore motion in the scene.
Second, the simulation of complex physical interactions, the accuracy of real-time audio, clear text rendering, spatial accuracy, and so on are areas of continuous research and improvement.
It's worth noting that the official announcement said that the video and audio generated by Genie 4 will contain Google's SynthID digital watermark to help identify AI-generated content.
However, there is no denying that Genie 4 brings new possibilities.
The Hollow Deck is coming?
Genie 4 has been widely discussed on social media platforms such as X, Reddit.
Some people have begun to wonder how games and virtual reality will change as a result.
On Reddit, one netizen imagined: "If we use this technology in augmented reality (AR), we basically have a prototype of a holodeck".
The Holodeck is a technology that can create highly realistic, interactive virtual environments in interstellar spaceflight.
Some netizens believe that if Genie 4 is combined with future VR, AR devices, it could really bring people into a world created by AI in real time.
Some even associate it with David Foster Wallace's novel Infinite Jest.
In the novel, there is an extremely attractive entertainment that makes viewers so absorbed in it that they do not even want to return to normal life.
One of X's netizens wrote: "We're only a few years away from creating a world full of endless jokes. A world that's optimized for you, realistic enough to capture your attention completely and make you never want to leave the digital world.
In addition to gaming and entertainment, some netizens are interested in the deeper technical meaning of the world model.
There are some netizens on X who think that many people may not fully understand the use of world models yet, but Genie 4 has made it clearer how it can help AI move towards AGI.
He used a very pictorial metaphor: humans can first imagine a real-life scenario in their minds before they take action, speculate on the possible outcome, and then decide what to do next.
Although large language models can describe physical phenomena in text, analyze mathematical problems and predict the outcome of actions, they also require an environment that interacts with them if they are to be directly tested.
The world model is expected to provide such a testing ground.
When the reasoning power of a large language model is combined with the physical dynamics learned from the world model, AI may first attempt different actions, observe results, and then decide how to deal with problems in the real world.
This is also one of the important aspects of Genie 4's connection to Gemini.
Genie 4's current performance has also received mixed reviews on Reddit.
Some netizens say they prefer to see an ordinary person live on YouTube using Genie 4 instead of Google's carefully crafted demo footage, accepting random requests from viewers to see how the model performs without prior screening.
Some netizens have noted that while the image quality of Genie 4 has improved significantly, the keyboard direction control does not appear to be as direct in some of the presentations.
One user pointed out that Genie 3 has demonstrated the ability to directly control characters, while parts of the Genie 4 video look more like a model-independent character movement, with user input mainly used to guide the direction of the lens.
Genie 4's voice performance has also sparked some discussion.
According to Reddit's Background-Wafer-548, voice is still the weakest link in Genie 4, and it sounds like an old-fashioned text-translated voice (TTS).
He pointed out that even on older laptops from a decade ago, small voice models running on a CPU can now read text quite naturally. In contrast, the voice of the characters in the Genie 4 demonstration does not seem natural enough.
He summed up the phenomenon as: "It's relatively easy to get good results, but it's very difficult to get to a really great level".
This also indicates that the ability to produce real-time audio in native language is still two things, as opposed to allowing virtual characters to express themselves in a natural, credible language.
So what do you think of the latest generation of Google's world model, Genie 4, welcome to comment and share in the comments section!
The link is as follows:
© THE