Large models can understand the complex structure of web pages and the rules of the game.
But if they were allowed to grab a hot ticket during a slightly longer window period, or in a Russian game of squares, to complete a spin and move before the squares land, would they be able to handle themselves?
In order to test the model's real-time capabilities, researchers at institutions such as INVEDA and UT-Austin have proposed the Realtime-Bench, which is used to test 12 front-end models. The best-performing model (GPT-6-Astra) also has a 44% success rate when implemented in real-time environments.
The paper is titled: Real-time Bench: Can Frontier Models Solve Real-Time Interactive Problems?
Link to the paper:https://arxiv.org/abs/2610.22155
The main page of the project:
Huggingface paper:
In real-time interaction, the world won't wait for models
When picking up tickets, models have to cope with the ever-changing interface, making decisions and completing operations in a limited amount of time. Such real-time interaction tasks require models to have three core capabilities:
RT-Perceive: Observing changes in the environment. Determine the current state, whether information is outdated, and whether opportunities for action still exist.
2. RT-Deliberate: Adjust reasoning according to time. Decide how long to think and whether it is worth continuing to think.
3. RT-Act: Capture the timing of action. Judge whether to wait for or immediately act, and select actions that are still effective when executed.
The model must first find out how to place tickets, then how much time is left to control the time to think and order in time, depending on the purchase window.
Around some core capabilities, Realtime-Bench places the model in a constantly changing environment to examine whether it can be observed, reasoned, and acted upon by time constraints.
Let early and late action affect the outcome
Realtime-Bench tasks have two characteristics: the environment changes over time and the chances of achieving a goal are limited in time.
Even if the operation is correct and executed too early or too late, it can lead to mission failure.
As shown in the figure below, in real-time auctions, the model must bid before the end of the countdown and must also deal with the price increase of the opponent at any time; in greedy snakes, the snake will continue to advance, and the model must turn in time to eat the apple.
Figure 1: Typical example of real-time Bench tasks
Realtime-Bench's 30 web-based tasks and 20 game tasks include these time requirements, covering travel, e-commerce, services, security, and streets, puzzle, strategy, and simulation.
Figure 2: Distribution of tasks by Realtime-Bench
How does the time of the environment count when the model is reasoned?
Realtime-Bench compares the performance of the model in the following three environments:
The Wall Clock is based on real-time progression of flowers, how long the model takes to generate a response, and how long the environment moves forward.
Paused Clock, a method of reasoning in which the state of the environment is frozen in the process of reasoning. This method of reasoning as an idealized counterpoint makes it easy to distinguish the failure of a model due to a lack of reasoning ability or due to time constraints.
Token Clock, which is the timing of the number of tokens generated by the model, means that the more tokens generated, the later the actions will be executed. This method eliminates the impact of API service delays and makes it easier to observe the real-time capabilities of the model itself.
Figure 3: Realtime-Bench assesses the model's real-time interaction capabilities under three clock protocols. Wall Clock advances the environment according to actual reasoning; Paused Clock suspends the environment during reasoning; Token Clock according to generated token numbers advances simulation time, excluding the impact of actual service speeds, and assesses the model's time perception ability.
Experimental results: In real time, the model is still difficult to complete real-time tasks
Realtime-Bench tested 12 front-end models covering three categories: standard asynchronous models, including GPT-6 Astra, GPT-5.6 Sol, Claude Fable 5.1, Claude Opus 5, Gemini 3.8 Flash, DeepSeek V4.1 Flash, and Kimi-K3; real-time stream models, a large range of language models dedicated to real-time interaction optimization delays, including GPT-Realtime-2, Gemini 3.1 Flash live and Gemini 3.8 Live; and decision models, which only output decision-making delays for free-text output, including OpenAI Decisions and Jev 1.13.
Because real-time interaction requires stability, which cannot be repeated after a failure (e.g. not trying again after a ticket is stolen), the Reatime-Bench uses Pass^3 as the main indicator. Pass^3 is the probability that the model will succeed in three repetitive tests of the same task, measuring the reliability of the model in real-time tasks.
The best performance of the GPT-6 Astra when the environment pushes the actual reasoning delay according to the Wall Clock is only 36% of the Pass^3 under the high reasoning intensity. Most models have a delay of more than 1 second, while the Jev has achieved a delay of less than 1 second, but the Pass^3 is only 8%.
These results suggest that low latency alone is not sufficient to guarantee reliable real-time performance. The model is both responsive in time and has the reasoning ability to solve tasks.
Figure 4:12 shows the performance of a front-end model on the Realtime-Bench
Increasing the intensity of reasoning, not necessarily improving the success rate of real-time tasks
Figure 5: Success rate of missions under three clock protocols.
Under Paused Clock, improvements in reasoning intensity have improved success rates for all six models. For example, the GPT-5.6 Sol went from 42% to 58%, and the Claude Opus 5 went from 62% to 70%.
When the environment continues to change during thinking, the effect of improving reasoning intensity varies depending on the model. Under the Token Clock, improving reasoning intensity drops the success rate of Claude Opus 5 from 44% to 30%, while Kimi-K3 rises from 24% to 36%. Similar differences also exist under the Wall Clock: GPT-5.6 Sol drops from 16% to 8% and Gemini 3.1 Flash Live from 6% to 8%.
This shows that reasoning in real-time tasks needs to be balanced: continuing to think can help the model choose better operations, but it can also cause it to miss the time of execution.
Reducing rational delays is helpful but not enough.
Under the Token Clock, we will gradually increase the number of generating tokens corresponding to simulation time per second from 32 to 64,128, eventually completely eliminating the time spent on the generating process. Thus, generating the same number of tokens, the simulation delay before the action will be gradually reduced.
Overall, the relaxation time limit helps to improve the success rate. From 32 tokens per second corresponding to the total removal of reasoning time expenditure, the success rate of the Claude Opus 5 increased from 30% to 62%, and the success rate of the Kimi-K3 from 22% to 60%.
Not all models have success rates that continue to rise as delays shorten. Even if reasoning no longer consumes environmental time, there are still many tasks that cannot be completed. Reducing delays is helpful but not enough to solve all the difficulties in real-time interaction.
Figure 6: Effects of time stress on mission success rate
Realtime-Bench tasks require models to recognize the changing state, decide how long to think and decide when to wait and when to act based on the rest of the time. Future models need to coordinate these links more flexibly: timely comparison of programs, prompt action when the window is about to close, and timely modification of outdated plans when new information arrives.
The author's description:
Realtime-Bench is a collaborative project created by top universities and technology companies. The benchmark test was developed by the University of Texas at Austin (UT Austin), Northwestern University (UT Austin), OSU (USO), University of Illinois at Urbana-Champaign (UIUC) and NVIDIA in partnership with Steadyworks. The core research team included: PhD student Jing Tangyu from UT Austin, M.S. student Xu Yang and assistant professor Jiaxin Pei; professor Manling Li from Northwestern University; OSU's Zhao Zhang; UIUC's Zheng Wang and Haozhe Zhao; and NVIDIA researchers Qi Cheng and Steadyworks Wenx Li and Xuan from industry.
© THE