The first author of this article is Dr. Ding Kechin, PhD student at the University of Hong Kong. Members of the team include Dr. Chen Liu, Dr. Xue Wei, Dr. Liu Liyang, Dr. Liu Yuxiang, Dr. Li Jun, Dr. Chen Zhuang, and Dr. Chen Zhuang, PhD student at the University of Hong Kong.
How should the world model be judged? The answer is: go and play with it like real users.
For example, in the virtual world, people walk around the house for a week and check if the scene is consistent after returning to the original location; go into the water and observe if the flowers have been stirred; or go back and forth between different rooms to see if new progress has been made by the person painting in the living room each time they return.
For this purpose, researchers from the University of Hong Kong, the Chinese University of Hong Kong, Zhejiang University and the QuickCoin team used Agent Player to operate a model of the world: only a given set of scenarios and long-range targets, and Agent Player observed the geometric consistency, correctness of interaction and rational evolution of the video generated during the interaction. They systematically assessed and analyzed multiple SOTA models, and have open-source assessment sets and code.
PlayWorld: Benchmarking World Models with Agent Players over Long-Horizon Objectives
Link to the paper:https://arxiv.org/abs/2608.13552
The main page of the project:https://kxding.github.io/project/PlayWorld/
The evaluation group:
Open source code:https://github.com/kxding/PlayWorld
Why is a fixed action trajectory not fair enough?
When humans test a world model, they usually don't care if the model has performed three right turns strictly, but whether the model has completed a week's circumference and returned to the original landmark. Both of these assumptions seem close, but the truth is that they answer different questions. For example, with the model sculpting a circle around the center of a circle, fixed trajectories can only ensure that all models receive the same keys, but not that they all reach the same angle. If the model does not actually reach the target angle, the subsequent geometric consistency score loses its comparability.
PlayWorld therefore upgraded the rating unit from a fixed action sequence array array to a long-term target array related to the arcade scene: all models face the same initial world, the same goal, but allow Agent Player to make the necessary adjustments based on real-time observations.
The diagram revolves around the center of the sculpture and observes the shape of the sculpture unchanged:
Key innovations:
Agent Player will watch, adjust, and call off.
The Agent Player consists of a multi-mode model and an interface converter. At each step, it looks at the generated array, the actions performed, the scenarios described and the long-term goals, and then selects one of five decisions:
Keep: continue the current action as planned, or move on to the next action;
Stop: the target visual state has appeared, ending the current action early;
Ext