For the first time in two years, interactive entertainment has gone online.
For example, when you go to Henan to open the Yu-Gi-Oh! city, many tourists are not just running around and performing. Some people specialize in looking for NPCs to guess, take missions, exchange "silver tickets", or after browsing a short video of an NPC, run directly to the attraction in search of real interaction.
This kind of experience is not difficult to understand. Normal performances, at home brushing the phone to know the main plot, and viewing on the spot to draw a visual effect. But it's not the same as playing NPC, each interaction is personally involved, some intelligent NPCs can still hang up, randomly draw gold, go for a long time to remember.
These projects also show that the market for interactive entertainment is booming.
In January, the company launched the industry's first real-time video world model, the PixVerse R1. The gameplay is relatively new: you enter a sentence, and the content in the video changes in real time according to your instructions, allowing you to experience the joy of the creator.
At the time, the game was circumscribed on social platforms such as X, and some even said that it was the beginning of a whole new "war of attention". It was similar to what happened in the Old Man and the Sea: an interactive game that excited a group of players who had already been static and passive enough to get their attention redistributed. Online interactive entertainment is waiting to explode.
Sources are as follows:
Sources are as follows:
PixVerse, which had long anticipated this momentum, also continued to gain momentum in this race. Today, the real-time generation behind this interactive gameplay has evolved into the R2 version.
In this version, users begin to truly interact with a world, and the world can continue to run steadily, becoming a system that can truly "play". It both answers the question of how the real-time video world model continues to scale, and it finally gives the world model a way to get out of the lab and face the user in a landform.
So what kind of experience did R2 bring? Let's take a look at some of the officially released intuition cases.
From R1 to R2,
The world model is starting to think, "How do you play?"
When R1 was released, the most eye-catching feature was "real-time", which gives the user a magical feel through its real-time response. But if a world is really ready to "play", the answer to a Prompt is far from enough, because this game can only arouse momentary curiosity, and it's easy to play.
Getting into a world and interacting with it is a very complex thing, and you have to solve a lot of problems, such as how users interact with the world, how multi-modal inputs can't really work; latency can't hold up; how users add new scenarios, how systems connect; how to keep the world running steadily, don't drive for long periods of time, don't gang up...
From a few instances of internal testing cases currently released by PixVerse, you can see that these problems seem to have already been initially solved.
The first case is similar to a real-time generated immersion game. Characters are dodging the chase, while the player controls the action with WASD, while directly rewriting the situation in writing: input "adding obstacles to stop the chase", roll out a barrel of oil immediately along the way; ask to open a secret route, and a stone door appears near the walls of the character; enter the tunnel and encounter a patrol soldier, the player changes the character into a patrol uniform, mixing the crowd and continuing on.
A more intuitive point here is that different types of controls begin to be embedded in the same set of experiences. The direction keys are responsible for "where I'm going now", and natural language is responsible for "what I want in this world to happen next", and both inputs are simultaneously controlled and the picture can respond in real time.
The second case further illustrates R2's balance in terms of gameplay freedom and overall control. The player is moving into an ancient world where he suddenly enters a phrase "calling a tiger to ride" and a giant tiger in line with the scene's style appears immediately; the next instruction is "Ride on, and the characters ride the tiger to keep going". More significantly, this temporary twist does not interrupt the original gameplay: the player continues to come across the bridge, enters the scenario selection naturally, and then rides the tiger across the bridge, selects weapons, and the story continues.
This is actually a critical step in real-time generation: players can run out of script, but the world still knows how to get you back. The freedom in past games often comes from developers preparing more branches beforehand; here comes another possibility. The main line still exists, but the process between the main lines can be generated.
Other cases show another interaction: with the characters directly "chat". A Samsung stack of masks can read the player's questions and answer them in real-time using the dialect, while the face changes when speaking; in a digital live stream, viewers can directly input text in the course of the broadcast talk, the broadcaster quickly picks up the questions and transfers the topic naturally.
In addition, digital people also support direct speech insertion in the speech process, i.e. the simultaneous output and input of speech, and the response of the characters in the picture is very natural, with almost no apparent waiting in the process.
This kind of case doesn't seem so dramatic, but it's just as important for interactive entertainment. Because real-life interaction doesn't have to go through the obvious iterative process of "input, wait, create, play". Only respond quickly enough that users can forget that there's a model behind it that really focuses on the characters and the world itself.
So in these cases, R2 is no longer just letting the video be "listened to". The player can control the action with a button, but also insert new ideas with text or voice at any time; the world needs to make an immediate feedback while continuing to run along the original logic.
To put these seemingly natural experiences in the same system requires strong technical support.
R2 is a real change.
It's happening at the bottom.
After looking at these cases, the question naturally arises: what would a model do in a world like this? It should both have the capacity to generate models and process inputs and updates like a continuously running system, but it should also push the delay to the extent to which humans can interact naturally.
In the past, methods within the industry have been to break down the links more thoroughly: first converting the two-way model into a return model, then training the teacher model individually for each submission, then distilling a few steps, and finally having to reverse-correct the historical bias generated by the model itself.
R2 in turn solves the problem by converting the entire flow stream into two things: first, continuous pre-training of a unified Omni Causal AR, bringing generating quality, long-range consistency, and multi-modal controls together into the same model; and second, distilling a real-time model directly from this already-stable model, Real-Time Acceleration, to accelerate the original capability as steadily as possible to an interactive real-time operating area.
Omni Causal AR: To maximize the world's capabilities
First of all, Omni Causal AR's baseline. It's all about putting all the inputs -- text, reference, voice, WASD -- into the same model, putting out synchronous audio and video streams, and receiving new inputs alongside the runway. It sounds great, but to keep this model stable over an increasingly long period of time, there are several craters that can't be bypassed during pre-training.
The first crater is that the time scales of the various control signals are fundamentally different. You press the direction button for instant feedback; you enter a scenario instruction that describes an entire event that may have lasted several seconds. The model can only choose between the response speed and the expression of completeness if it cuts the audio video using a fixed-length time unit.
The solution to R2 is Dynamic Chunk, where the semantic boundaries of the control signal are self-adjusted: operations are cut short to ensure response accuracy; events are cut long to maintain structural integrity. This has a side benefit: different types of tasks and data can go through the same set of cause-generating interfaces, entering the same training process, which leaves continuous scaling behind.
The second crater is more hidden. When training, the history that the model reads is the true value of clean. But it really runs out, it reads the history that it generated in its previous round, which must be with noise and small errors. A model that has only seen clean history, single-step quality can be beautiful, but unprepared for its own mistakes, a small deviation can be magnified along the way.
Thus, R2 trains the history of noise, i.e. Hybrid Teacher/Diffusion Forcing, which allows the model to get used to imperfect oneself in advance. With the Causal Attention Mask, it controls "what histories can be read in the current state", the relative time coding controls "the location of these histories in the window", the gap between training and reality is narrowed, and the model can be steadily pushed downwards when the history is already a bit distorted.
The third problem is memory. Long-term running can't be done by remembering the whole history, so computing can't get ahead. But if you cut too short, who are the characters, what the scene looks like, what happened before, you lose it.
R2's approach is to divide memory into three tiers: Sink Memory is responsible for roles, scenes, world rules, these long-term focal points, ensuring that the world is always the same; Rolling History focuses on recent movements, gestures and footage, so that the states in front of the eyes are naturally connected; Object KV Cache uses object-level historical compression to replicate, leaving only a portion of the sequence that really affects the sequence.
The last crater is the error itself. Serious drift of the AR world model, the source of which is often small and inconspicuous: a small error was written into history in the early days, after which each cluster inherited it, and the error rolled into drift.
R2 built an Error Bank specifically for this type of error, which can be understood as the model's erroneous textbook, which uses representative erroneous history samples, returns them proportionally when training, and allows the model to repeatedly practice how to recover from a problematic state. The effect of this mechanism is to extract numbers: the long-term brightness drift indicator decreased from 0.201 to 0.129, a decrease of about 35.8%; 20 of the 29 long sequence samples had improvements, 5 samples of explicit inaction, and the error movement was completely eliminated.
Real-Time Acceleration: The ability to accelerate in real time
The next question is how to keep the world stable in the long run. One of the most crucial decisions here is the unification of the starting point: the teacher model, the initialization of the student model, real self-return reasoning, all come from the same Omni Causal AR. Thus, the step-by-step model cannot be used to study the world from scratch, and the distillation of this thing from "re-learning" to "step-by-step acceleration" can be directly inherited from previously practiced production quality, control capacity and long-range state transfers.
Specifically, acceleration depends on three methods, each of which costs one piece.
DDMD with Adversarial Regularization controls the quality of distillation. In step-by-step generation, there are two natural goals: to respond accurately to various inputs, while maintaining the authenticity of the image, and to optimize with a set of signals.
Block-sparse Attention cuts the calculation quantity in the long context. Models estimate the relevance per block, leaving only a small portion of the highest scoring key connections to do the accurate calculation, while the rest skip directly. Attention scarcity reaches more than 90%, and the quality of the image is evaluated, but the conditions following these indicators do not significantly deteriorate, saving the budget into lower delays.
Pyramid Ultra-few-step Distillation deals with the cost of high-resolution. Rather than having all computation run in high-resolution space, it is better to set the scene layout, subject movement, and lens structure first in the low-resolution phase, and then to replace the texture and detail with very few high-resolution steps.
So to put these two layers together, what R2 actually does is one thing: the upper focus is on growing the world's capabilities, the lower focus is on accelerating this capability as harmlessly as possible to the real-time space, not allowing it to break down in the middle of the migration.
Technical report links:
The world model + interactive entertainment
What's the big deal?
"The power of interactive entertainment represented by video games lies in the fact that even ten or fifteen years later, games played will remain in memory. Although novels and movies can be touching, they are often only remembered when they were actually touched, not even the plot can be told. However, the game is in-person, a form of participatory entertainment, and the touching way is very unique, which is the strength of the game".
In the past, games and entertainment experiences have developed this mechanism very well, but the vast majority of the possibilities in the game need to be prepared beforehand. Interaction, and more in the design of branches in fixed content. Branches are more closely designed, players can do, but only choose a path in the written script.
In the future, a creator may first define worldviews, roles, rules, visual styles, and core scenarios, and then generate some specific content at the moment of interaction.
In this rapidly warming world model, PixVerse is an early-stage real-time video generator and attempts to make it one of the leading providers of interactive entertainment products for ordinary users.
The three directions that R2 presents today - world exploration, interactive storytelling, and real-time characters - correspond to three fundamental components of interactive entertainment: a space for action, a set of storylines that keep things going and people living in it. When these three things are increasingly driven by the same real-time world model, the boundaries that were previously clear between games, interactive video, and digital people, and even online travel, may become blurred. The life cycle and commercial space of entertainment products may also be redefined.
In the comment area, we also put a link to R2's experience reservations. Interested readers can make reservations in advance and receive invitations after the next experience opens. Developers can also follow the API's open information through the same input.