Beyond State Consistency: Behavior Consistency in Text-Based World Models
This is the first edition of the EMNLP 2026 Main Conference.
Link to the paper:https://arxiv.org/abs/2604.13824
Open source code:https://github.com/Ricardo-H/behr-wm
The model is open source:https://huggingface.co/collections/Ricardo-H/behr-behavior-consistent-world-models
In a nutshell, text-world models are more consistent than "generating text that doesn't look like the real environment", but we found that predictions of text that are closer to the real environment may make the intelligent body more irrational. After changing the training goal from "state consistency" to "behavioral consistency", 16 groups of experiments configured a nearly complete improvement in the trajectory level consistency of the world models, and the false-positive rate of off-line assessments of weak intelligent agents decreased from 42.5% to 9.5%.[3]
What is the "critical" delusion when forecasting bias is inevitable?
An LLM agent progresses rapidly in tasks such as web navigation, tool calling, text adventure, and so on, but it costs a lot to keep a smart body running in a real environment: slow, expensive, and sometimes irreversible. One idea is to use another LLM to "play" the environment.
How to calculate a good world model? The existing methods give almost the same answer: make the text generated as close to the real environment as possible. In training, whether it is supervised learning (per token on the real trajectory as much as possible) or reinforcement learning (with similarity to text such as F1 score as a reward), optimization goals all point to the same thing. So, make predictive text closer to real observation; evaluation also uses metrics such as F1, BERTScore, ROUGE-L to measure text similarity.
If all illusions can be eradicated, this logic is valid, of course. But in long-term interactions, the decades since the return of predictive errors have accumulated, leaking commodities, incorrect attributes, and making UI elements difficult to avoid. Since illusions cannot be completely eliminated in the short term, a more realistic question is whether different types of illusions have the same impact on downstream intelligent decision-making.
The more the text looks, the better the smart body doesn't have to do.
Figure 1: The indicator reverses the intent. Two hallucinatory outputs of the world model: The output 1 of the cluster missed the target shirt, the BERTScore still reached 0.974, but the smart body could not find the target; the task failed; the output 2 of the indicator missed only two unrelated items: a tunic and a scarf; the BERTScore dropped to 0.859, and the smart body completed the purchase. BehR correctly identified the "key" hallucination.
The two outputs in the diagram are the illusion of "missing commodities", but the impact on the decision-making of the intelligent body is quite different: output 1 is less of the target commodity, the intelligent body cannot find anything to buy; output 2 is less of the two unrelated commodities, which does not affect the purchase at all. However, the mainstream text reliability indicators of the feedback on these two types of errors are precisely the opposite. More severely punishes the "good illusion" of removing unrelated commodities, instead of the "fatal illusion" of removing the target commodity.
We used controlled disruption experimental quantification to verify this phenomenon of "indicator reversal": F1 score, BERTScore, ROUGE-L and even GPT-4o when the referee, all reversed.
This shows that text reliability is not the same as functional correctness, and that the mainstream world model's training goals have not always been aligned with what the intelligent body really wants.
From "State consistent" to "Behavior consistent"
Since the similarity of the text is unreliable, we take a different perspective: if the intelligent body makes the same decisions in the predicted state and in the real state, this prediction is good.
One of them is the real state, the predictive state, the historical trajectory. Analog translation: literal translation is not necessarily a good translation, the key is the reader's ability to make the same judgment from the original.
Methods: Behavioural consistency reward BehR
Figure 2: BehR training process. The above is a traditional practice (in contrast to MLE for supervised learning and reinforcement learning with F1 as a reward): comparing predictions with real-state text. Below is BehR: Frozen reference intelligences score for the same action in two states, respectively, and the difference in score is a reward signal.
The idea behind BehR (Behavior Consistency Reward) is simple: to find a frozen reference body that produces a pairwise probability of the same action in the real and predictive states, respectively, the closer the two approaches, the higher the reward:
Specifically: in two states, a consistent score is the most rewarding; removing the target commodity will result in a change in the score; the reward will plummet; removing unrelated commodities will have almost no effect on the score; the reward will remain unchanged. Training using GRPO (Group Relative Policy Optimization): The world model generates several candidate predictions for each sample, calculating BehR, and doing relative group integration is strategic in scale.
The results of the experiment
We have verified on WebShop (e-commerce navigation) and TextWorld (text adventure) that the baseline covers Qwen2.5-7B and LLaMA3.1-8B, assessing that the intelligence contains Qwen3-8B, Qwen3-32B, GPT-4o, and GPT-5 in total.
Table 1: Multi-Smart Body Assessment Complete Table. Real = Smart Body Success Rate in the Real Environment; WM = Smart Body Success Rate in the World Model; W2R = Success Rate of Replication of Action Sequences in the World Model to the Real Environment; CR for W2R vs. Real Success Rate; $$CR_{pw}$$ for Task Pairing Consistency, which is the main indicator of the paper.
Out of the 16 group configurations, the baseline of the BehR world model was upgraded to 13 groups, while the remaining 3 groups remained flat without degradation. The most significant improvement was on WebShop:
Qwen3-8B: up from 0.345 to 0.483 ((+13.8 percentage points, about +40%);
GPT-4o: Increased from 0.760 to 0.840 ((+8.0 percentage points, about +10.5%)
The strong smart body baseline on TextWorld has approached the scoring ceiling, and BehR's role is more to maintain this near-complete consistency.
Two downstream applications
Offline evaluation. When the world model generates an environment that is simpler than the real one, weak intelligence can also interfere. The real success rate of Qwen3-0.6B on TextWorld is zero, while the baseline world model classifies 42.5% of the tasks as successful. Nearly half of the tasks are false positives.
Forward planning. Let the smart body propose five candidate moves at each step, using the world model to simulate the best. BehR World Model helps Qwen3-8B increase the success rate on WebShop from 14.5% to 25.5% ((+11.0pp), above the baseline of +9.0pp. The weaker the smart body benefits the more it needs to "think and act first".
Summary
We uncovered the indicator reversal phenomenon in the evaluation of text world models and shifted the training goal from "status consistent" to "behaviour consistent", proposing behavioral consistency rewards for BehR: using the decision differences in reference intelligences to be frozen instead of similarity in text as a training signal. The entire process requires only feedback from locally frozen reference intelligences without the need for additional human preference labeling.
We hope that this work will give the community a fresh look at a question: the value of a world model is not in how much the predictions generated look like real environments, but in whether it will allow intelligent agents to make consistent decisions.
Author's note
The paper was produced in collaboration with MIT, MBZUAI, Beijing University and Microsoft. The team included Youling Huang, Guanqiao Chen, Junchi Yao, Lu Wang, Fangkai Yang, Chao Du, Chenzhuo Zhao, Pu Zhao, Qingwei Lin, Saravan Rajmohan, and Dongmei Zhang.
