An AI-generated graph can achieve high scores on aesthetics, punctuation and human preference metrics, but it contains distorted limbs, interpenetrating objects, or unreasonable physical phenomena. These errors are not always visible from existing scores.
This exposes a blind spot in the evaluation of text images: whether the objects, interactions, and spatial relationships in the image conform to the constraints of the real world; the lack of specialized, systemic measurements, which are difficult to quantify, compare, or make clear improvement goals.
TerraVis, which is hosted by NeurIPS 2026 Evaluations and Datasets Track, presents a "world consistency" assessment framework around this issue: covering three levels of 18 categories of issues, both scoring and pointing to specific issues, visual evidence and severity.
TerraVis: Towards Evaluation of World-Grounded Visual Consistency in Text-to-Image Generation through MLLM Workflows
He is also the co-founder and co-editor-in-chief of the Chinese-language newspaper Shuai Fu, Jing Gu, Jian Zhou, Zicheng Duan, Gengze Zhou, Qi Wu.
Unit: Australian Institute of Machine Learning (AIML) at the University of Adelaide; xAI; Responsible AI Research Centre
Meeting: NeurIPS 2026 Evaluations and Datasets Track
The paper:
The code is:https://github.com/ShyFoo/TerraVis
The data collection:https://huggingface.co/datasets/ShyFoo/TerraVis-Annotations
The online experience is open. You can upload your own map to see the world's consistency scores and individual judgments; you can also browse the built-in examples to see intuitively how TerraVis "finds problems".
It's the first time I've seen this.
Figure 1 High scores of traditional indicators do not necessarily correspond to a higher world consistency. The best results of selecting images, aesthetics, alignments and preference indicators generated by the same cue words are not the same as judging the consistency of the human world.
01 What the world's consistency is evaluating
World-Grounded Visual Consistency is concerned with the question of whether the macro-objects, interactions, and scenes in an image are logical at the structural, spatial, physical, biological, and common sense levels.
It has a clear distinction from "whether to draw a theme". The computer desktop image in the paper obtained a TIFA alignment score of 1.00, but the structure of the laptop computer is clearly distorted, with a consistency rating of 0 for the human world. The other picture is the opposite: the prompts require "no fountain of water flowing", and the model draws a fountain of fountain. It does not follow the prompts, but the image itself conforms to reality, and both humans and TerraVis give the highest world consistency rating.
Figure 2 illustrates a different problem of alignment with world consistency. The left diagram corresponds to the pronouns, but there are changes in the structure of the laptop computer; the right diagram deviates from the requirement of "no flow of water", but the fountain itself corresponds to reality.
In this case, a lower world consistency score does not mean a creative failure, but it indicates that the picture uses a surreal setting.
World consistency is not the same as photo-like writing. Simplified cartoon textures, flat-colored illustrations, sketch omissions of details should not be taken as a problem; but a hand holding a cup in the cup body violates the space constraints between objects and has nothing to do with painting. Stylised images can be judged as long as they contain recognizable macro objects or scenes; unsuitable content such as pure abstract patterns, icons, graphs and interfaces returns N/A.
02 Dismantle intuition into verifiable problems
In order to translate "looking out of alignment" into executable checks, TerraVis divides the problem into three layers, 18 categories: 7 types of object-level checks for identity, structure, anatomy, texture and texture; 8 types of interaction-level checks for contact, support, motion, optics; 3 types of scene-level checks for relative size, shading, and regional continuity.
Figure 3 Tri-layer 18 class examinations of world consistency. Objects focus on individual objects at the object level, interactions focus on the relationship between objects and their environment, scenarios focus on the overall spatial organization; red circles indicate representative visual evidence for various problems.
The first three steps can be performed by multi-mode large models, and the fourth step uses fixed formulas. Each type of problem reports the most significant abnormality, and retains the affected objects and visual evidence.
Figure 4 TerraVis' four-step evaluation process. Only images are input; non-applicable images return N/A. The problem of obtaining worldwide consistency scores and traceability of applicable images is detailed after a class-by-class inspection and a primary classification.
"Main" and "secondary" depend on the content of the problem affecting the subject, prospective object, or core event. The notes that affect the subject, prospective object, or core event are the major violations; the notes that only affect the background or edge elements are the minor violations, and cannot be easily distinguished by the size of the button. The main experiment assigned the two powers of 1 and 0.5 respectively, and then counted by the index function: 1 point for the undetected problem, a minor violation of 0.61, a major violation of about 0.37. The lower the score, the more or more important the problem detected.
The method of questioning is equally critical. The research team rewrote the name of the abstract infringement to specific visual conditions, such as asking "do two objects that should normally be in contact look like they are permeating each other". Thus, the overall judgement of the hypothesis is broken down into one-by-one checks that have evidence to verify.
03 13500 Effects of Assessment of Artificial Marking Inspection
On two benchmarks, COCO-T2I and GenAI-Bench, the team evaluated five text-based models of SD 3.5-Large, Qwen-Image, FLUX.2-dev, GPT Image 1.5 and Nano Banana Pro. They randomly extracted half of the 4,500 images, each individually scored by three markers, for 13,500 artificial markings; the automated assessment model for the main experiment was GPT-5.5.
According to the majority voting criteria for "at least two markers believe there is a problem", images of world consistency problems in the two benchmarks accounted for 48.6% and 69.7% respectively. This proportion was influenced by word-of-mouth, particularly GenAI-Bench, which contained fictional or non-realistic requirements, and therefore could not be directly explained as the failure rate of a model to execute a generated task.
The key to measuring whether the automatic indicator is effective is whether it is consistent with the human world consistency score. After combining images of five models, TerraVis achieved Spearman ranking coefficients of 0.44 and 0.43 respectively on the two benchmarks, the highest in the comparison method; the best RAHF performance on the existing indicator was 0.29 and 0.37 respectively.
Figure 5 Spearman correlation coefficients for the conformity of indicators to the human world are calculated by combining images of the five generating models. TerraVis achieved the highest correlation on both benchmarks; this measures the ability of the indicators to reflect the conformity of the world rather than their superiority in the original assessment task. GPTScore is output constant on COCO-T2I, and correlation coefficients cannot be calculated.
The control version of TerraVis-H received the same classification description and manual assessment guidelines, but directly gave the overall score, with the corresponding coefficients being 0.28 on both benchmarks, lower than TerraVis's 0.44 and 0.43 respectively. Even with the same assessment criteria, the individual tests were more effective than the overall score.
The results also explain the partial improvement: the recall rate of TerraVis from TerraVis-H increased from 15.4% and 19.8%, respectively, to 51.9% and 48.3%, respectively. That is, it detected more images that humans thought were problematic; the price was lower than the accuracy rate of 75.8% and 89.1% respectively, and misrepresentation still exists.
Dissolving experiments further support the role of the three-tier classification: removing any one level, the relevance decreases, with the object level contributing the most. Removing 18 class inspection items one by one also causes the relevance to decrease, but the single impact is smaller, with the greatest decrease being structural variation.
The process can also be adapted to different painting and evaluation models. The corresponding coefficients on the stylised image are 0.54, writing the actual image is 0.41; due to fewer stylised samples, this supports "the same can be evaluated" and is not enough to prove that the model is better at evaluating the stylised image.
04 Change a dimension and model ranking will change.
The change in global consistency is directly reflected in the model rankings. In the GenAI-Bench, GPT Image 1.5 takes the lead on many traditional indicators, but TerraVis and the Human World Consistency Score both give the first place to FLUX.2-dev.
Figure 6 The five models in the GenAI-Bench are ranked under different indicators.#1First place. FLUX.2-dev tops both the TerraVis and Human World Consistency Scores; Qwen-Image's Aesthetics 1 also doesn't translate to world consistency. Colours only represent the relative rankings within each indicator.
These results indicate that advantages over traditional indicators do not automatically extend to global consistency. The authors observe that FLUX.2-dev tends to generate more concise scenarios, which may reduce the chances of breaches, but this interpretation is still a qualitative observation and cannot be defined as "the simpler the picture, the more accurate".
In addition to the overall score, TerraVis can also give a violation of each model's composition. Symbol content problems such as text, identification, and other symbols, ranking in the top three in each of the two benchmarks; object-level problems account for the most, but interaction with scene-level combinations still account for about one-quarter to one-half.
Figure 7 The five models detect violations on two benchmarks. The horizontal axis of a problem represents the proportion of violations detected in the entire model, reflecting the composition of the problem type, and does not represent the incidence of this type of problem in all generated images.
It's easy to spot problems and it's easy to lose sight of them.
Specific cases remind us that automated assessments can still be wrong. TerraVis can detect some of the page codes that are ignored by humans, misjudge the structure of cats, or miss out on the variations in children's arms in dense scenes. Faced with the flaming birds in the restaurant, the three markers each gave 3, 5, 1 points, indicating that certain familiar boundaries are difficult to agree on even for humans.
Figure 8 TerraVis differs from human judgments in four categories. From left to right: discovering text problems that are partly ignored by humans; condemning as a violation images that humans agree are reasonable; omitting to examine abnormalities in the hands and arms that humans agree exist; the same image triggers marker disagreements.
Multi-phase processes require more time to reason, and there are still misrepresentations and leaks at the detection link, and 18 types of inspections need to be continually revised as models evolve. The value of TerraVis is to make world consistency a quantifiable evaluation goal, and to connect scores to specific visual problems, helping researchers compare models, analyze types of failures, and determine improvement directions.
06 Upload your charts to try TerraVis for yourself
TerraVis offers an online evaluation page: drag in, paste in or select an image, click Evaluate, which can be submitted, without deploying a model, or filling in a generating prompt.
During the evaluation, you can see the progress of 18 inspections; after completion, you can look at the total score, the type of problem, the specific visual evidence and the primary rating. The controller reads the feedback and understands why it gives the score and decides which findings are worth adopting.
If you don't have a finished image, you can also browse the built-in examples of the site to see the pre-computed evaluation results.
TerraVis adds to the world consistency dimension that has been missing in the long-term evaluation of text graphics, giving a systematic way of measuring "if it makes sense to draw". It transforms the intuition of "it doesn't look right", into quantifiable scores and verifiable visual evidence, to provide a clearer goal for improving the production model. As images become more realistic, world consistency should also align with aesthetics, image alignment and human preferences, becoming the core standard for measuring the progress of text graphics models.
First author's brief
He is currently a third-year PhD student at the Australian Institute of Machine Learning (AIML) at the University of Adelaide, where he is a Ph.D. assistant professor. His main research interests are factually consistent image generation and vision-language learning, robotics and reinforcement learning. He has received numerous papers from top international conferences such as ICLR, NeurIPS, EMNLP, CoRL.