This year, the interactive video world model has become very popular.
Ali HappyOyster, LingBot, Vidu S2 and PixVerse R2 are all coming together.
While the technology routes are focused, the goals are highly consistent, and they want AI video to respond in real-time to user operations, support continuous interaction, and create a virtual world that can be freely explored.
Of course, we have always had a conservative expectation of a truly open, long-running world of production, after all, there are a lot of technical and engineering difficulties in the real world.
But progress in reality is often faster than expected.
Recently, when we evaluated WBench's top spot in the interactive video world model, we saw a new name: Murea-Pro.
It took first place in the multi-round interaction video rating WBench Full-289, with 81.1 points, breaking the gap between maximum interaction capability and physical rationality, achieving a high score of 82.6 and 74.4 respectively.
Wbench links to:#leaderboard
The RTX PRO 6000 produces 3.6s of video length, RTF < 1 (Real-Time Factor, real-time factor), processing speed faster than the speed of audio playback, and real-time video.
We found a lightweight open-source version of the model on the official website: Murea-Fast.
This "lightweight model" is not inferior, ranking first in the OpenS2V-Eval open-source model with 64.98 points in the reference live video rating, with its NexusScore reaching 51.66, also ranking first in the reference recovery capability.
The two achievements that brought the model into our field of view, after thoroughly dismantling its technical reports and testing, revealed a lot of interesting things: the team's ambition is not to be the next world model, but a more fun, more imaginative space, faster landing, fast-paced, interactive content model.
To move away from the logic of "watching video" is to actually "play" in an AI-created world.
Technical explanation: How does Murea-InWorld sustain a continuously evolving world?
Murea-InWorld is a two-tier collaborative architecture.
At the start of each round of interaction, the upper-level Murea-Mind as the model brain reads the current state of the world, user instructions and the end images of the previous video, plans the next event, acts as a writer and director and the core role; the lower-level Murea-Pro and Murea-Fast are responsible for generating the corresponding images and sounds; the results are then passed on to the brain model for examination, confirming the changes that occur and rewriting the state of the world as the basis for the next round of interaction.
This process can be summarized as "state, action, generation, observation".
This two-tier collaboration allows models to avoid rapidly generating, but meaningless, content, and actually allows users to continue to change a virtual world with memory, voice, archival branches, and support for multi-player participation in natural language.
Technical report link:
This is the first time I've seen this.
What is the meaning of hugging face?
The official link:
The key to the stable operation of this closed-loop system is the continuously updated state of the world. The system maintains a set of "books" outside of the video model, where reference pictures, sounds, and fixed descriptions of characters and scenes are kept as assets; information such as character location, clothing, emotions, objects, and weather, lighting is continuously updated. Fixed identity is managed separately from dynamic state, which can reduce the probability that character appearances and scenario changes interfere with each other.
When the user withdraws the operation, the system restores the pre-generated state snapshot; re-opting creates a parallel branch, retaining the original video and plot. In multiplayer mode, each player's input is first converted into a structured action, and then organized into the same image by the system uniquely.
World Agent overview. The upper layer preserves the world's memory; the middle layer converts user instructions into generating conditions and saves snippets; the lower layer reviews the generated images to provide the basis for subsequent planning and partial status updates.
After determining the state of the world, Murea-Pro and Murea-Fast are responsible for generating images and sounds, the core mechanism of which is Omni-Reference Generation. Persons, animals, objects and scenarios references are coded separately as clean, latent conditions, and are independently numbered.
The left-hand side is the input reference, the right-hand side is the continuous sampling of a lifetime of videos. The focus is on the combination of characters, cats, fabrics and courtyards, as well as the correspondence of entities in multiple frames, writing and styling scenes.
Video and audio are co-generated in the same DiT. The controlled attention mechanism allows reference information to act on their respective formats, and then connects through the video and audio tokens in the image. The trained data uses structured descriptions to clarify who each reference corresponds to, by whom the script is spoken, and the timing of the action and footage.
Overview of the audio video generation framework. The three-tier structure of model capabilities. Above (a) explains how reference charts, selectable audio references and text together control audio video generation; Below (b) explains the two-phase distillation of MeanFlow → DMD; Below (c) explains streamlined rewriting with limited history.
Continuous generation uses a limited history window strategy. The video side mainly retains an earlier segment, while the audio side preserves a longer context, using unspoken sentences and character tone. During training, the system adds noise or latent disruption to the historical content, allowing the model to adapt to the preceding segment in advance in a situation where there is a gap.
After 50 steps to 4 steps of distillation and system optimization, Murea-Pro shortened the production time of 5-second audio video on 8 H20s from 231.6 seconds to 6.96 seconds, accelerating end-to-end cumulatively by 33.3 times.
Instead of the RTX PRO 6000 Blackwell, the same set of pipes generated a 5-second audio video for only 3.6 seconds. The real-time factor RTF is about 0.71, according to the "production time/finishing time" calculation.
The RTF is less than 1, indicating that the model generates faster than the speed of video playback. In other words, under the hardware and test conditions, the Murea-Pro has reached the threshold for real-time generation.
From left-to-right, up-to-down, cumulative optimization. On eight H20s, the 5 second audio video generation delay decreased from 231.6 seconds to 6.96 seconds, accumulating 33.3 times the acceleration; 50 → 4 model evaluations were delayed to 23.2 seconds, with subsequent optimization continuing to reduce costs. Each line retained its previous optimization.
The results of the evaluation confirmed the focus on the respective capabilities of the two models.
In WBench Full-289, Murea-Pro was ranked first with a score of 81.13 with an interactivity of 82.60 points and a physical rationality of 74.42 points, leading second with 9.52 points and 5.23 points respectively.
Specifically, Murea-Pro excelled in event editing, role-playing, and perspective switching. The event editing score was 90.92, and the role-playing score was 87.04. Navigation score was 71.79, one of the weakest of the four types of interaction tasks.
In terms of image quality, the Murea-Pro scores 79.97, which is higher overall.
Murea-Fast's advantage focused on multi-reference generation, earning 64.98 points in OpenS2V-Eval and ranking first in the open-source model with a NexusScore of 51.66, the highest for a live video reference system tested. In the OS2V-Pro expansion rating, Murea-Fast's reference-by-reference accuracy was 46.0, ranking first; in-screen identity consistency was 89.0, respectively.
This shows that Murea-Pro excels at executing multiple rounds of instructions and physical factors, and Murea-Fast stands out in maintaining multiple reference identities.
One-Hand Test: Can AI keep a story going?
Of course, in addition to these achievements and technological principles, we want to know if interaction, which is still commonly a prefabricated phase, can actually allow users to "play" in the AI world.
We first experienced a Murea-InWorld.
The system randomly generates a science fiction world that automatically sets characters, scenarios, and storylines.
The story is simple, but there are a lot of details worth paying attention to.
First of all, in terms of continuity and consistency, the board props in the main character's hands have been recorded on the board after the first video, so that in the following ten rounds of interaction, the main character has always held and has always held this board.
Regular video re-writing relies heavily on the continuity of previous images, what the characters have taken, and who the items now belong to. There is not necessarily a clear record.
However, Murea-InWorld can generate dozens of videos in succession, with the appearance of the characters, accompanying props, and background still stable.
Second, natural language is instantly generated, and after each round of videos, Murea-InWorld recommends follow-up actions based on the current scenario, selecting a system that automatically advances the scenario and footage without the need to write detailed hints.
We can also temporarily change the scenario and enter it directly: "Suddenly a kitten appears in the picture".
The system immediately created an independent asset for it, and in the next video, a cat appears in the control room. After the camera switches, its appearance remains consistent and does not suddenly disappear due to changes in the scene.
And in the process of the appearance of the kitten, the plot and lyrics give a natural twist based on the scene (the space station) and the actions of the protagonist, ensuring the consistency of the experience.
Another instruction for the kitten to spin a circle around the protagonist, the system correctly identifies the subject and the target of the action, and the interaction between the two characters is not confused.
In this 15-round interaction, the most intuitive feeling Murea-InWorld gives is that the story doesn't "get forgotten" as the video clip increases. The props in the main character's hands will not disappear for no reason, and new characters temporarily added can also enter the sequel, and players can intervene and change direction at any time.
Currently, Murea-InWorld has created a sense of near real-time interaction in continuous generation.
How many markets can a productive, interactive world open up?
The most direct value of a generated interactive world lies in the process of generating content and putting it into user engagement.
Traditional interactive content typically requires pre-planned storytelling, pre-production of material and pre-production of a large number of branches, rather than a "pre-production". Murea-InWorld systems use the brain power of Murea-Mind to allow players to change the world at any time using natural language, a new real-time interactive content production paradigm.
This paradigm is actually "Give a thousand players a thousand hamstrings", and it can be extended to a wide range of scenarios, including games, interactive narratives, virtual characters, and online entertainment.
Multiple agencies estimate that the combined value of the relevant market has reached $1 trillion, and that the acceleration effect of adjacent circuits such as AI escorting is expected to expand to $50 billion by 2030. If high-quality interactive content penetration reaches 10% of the global content and entertainment market, the corresponding long-term scale will exceed $50 billion.
In terms of model capabilities and system completeness demonstrated by Murea-InWorld, the Murea Labs team has taken a noteworthy step in the direction of building real-time interactive content models.
The next step is to see if they can continue to transform their technological potential into real experiences in scenes such as gaming, film and interactive entertainment, and to find opportunities to scale down. The future of a generative world is not far away.
The following is a link to the article:
This is the first time I've seen this.
WBench links to:#leaderboard
Technical report link:
© THE