Understanding Unified Multimodal Models (UMMs) is becoming an important development direction for multimodal models. Compared to traditional understanding models + generating models, UMM hopes to complete both visual understanding and image generation in the same model, so that text and images can be understood, reasoned and generated in a unified context.
But there is a more basic question here:
Do they really help each other when visual understanding and generation are put into the same model?
Existing work has not provided a consistent answer to this. On the one hand, understanding and generation may share visual knowledge, thus forming a transition towards a positive direction; on the other hand, the two objectives may also compete for model capacities, generating representational conflicts, and ultimately only functionally unified, but without truly forming a learning synergy.
Uncovering Understanding Generation Synergy in Native Unified Multimodal Models: From Representation, Task to System is an attempt to answer this question in a controlled native multimodal scenario.
Link to the paper:https://arxiv.org/abs/2609.01607
In order to avoid introducing additional visual representation precedents, the study chose to break down the relationship between visual understanding and generation into three levels: Representation → Task → System under the native multi-modal architecture (pixel in, pixel out).
The level of representation:
Understanding and generating help each other
But just sharing can lead to conflict.
First, the paper examines how visual understanding tasks and generating tasks affect each other's learning visual representations under different architectures.
The experiment discovered an interesting two-way relationship.
In the Dense Sharing architecture, text, clean visual tokens used to understand and noised visual tokens used to generate are all subjected to the same set of LLM parameters.
Further frozen feature probing suggests that features used for visual understanding have improved performance in tasks such as ImageNet classification, ADE20K segmentation and NYUv2 depth estimation.
In other words, visual generating tasks can help understand tasks to learn better visual representations.
When the authors use modality-decoupled MoT to insert a visual token into a separate visual branch, the result is the opposite: increased generation and decreased understanding.
Further measuring the CKA between text features and generative visual features found that adding comprehension training would significantly enhance the alignment between the two:
Visual comprehension of tasks can help generate a better alignment between task representation and text representation.
This shows that at the representational level, understanding and generating can provide valuable learning signals to each other. The problem is that if two targets are forced to use the same set of computational pathways, one of the targets is easily dominated and the other is degraded into auxiliary supervision.
Thus, the paper further proposes Task-decoupled MoT: Understanding Visual Tokens continues to enter language-anchored branches while generating visual tokens using generation-specialized branches; at the same time, the two paths remain connected through text semantics and shared attention.
This "partially specialize, partially share" design avoids the problems of both upgrades and downgrades in the first two architectures.
The mission level:
Mutual understanding of knowledge and generating tasks can achieve mutual promotion
After solving the representation conflict, the paper poses a more practical question:
What kind of training to understand tasks and generate tasks together promotes each other?
Deciding whether the two can promote each other is whether they rely on the same underlying knowledge or capability.
The paper selected three case studies: geometrical reasoning, SVG and 3D Spatial Intelligence for validation.
For example, in geometrical reasoning scenarios, understanding a geometric problem requires identifying entities, relationships, and constraints in a graph, while generating or editing a geometry requires models to master these structural knowledge as well.
With the addition of geometric image generation and editing tasks, not only are geometry tasks like Geometry 3K, PGPS9K upgraded, but gains are also transferred to more general mathematical reasoning benchmarks like MathVerse and MathVista.
The authors further converted the exact same generation data into understanding tasks such as image description as a baseline. The results showed that merely adding data from the same source does not bring the same benefits, showing that learning how to generate a geometrically constrained graph is itself an effective oversight of a different description than a language.
The SVG experiment is more straightforward. The model requires studying Image→SVG and SVG→Image at the same time, with both directions essentially dependent on the same vector structure.
After the joint training, Image→SVG was upgraded from 80.64 to 81.70, SVG→Image from 80.21 to 86.52. The paper significantly improved the model's ability to directly visualize the corresponding icons from SVG code by further exploring the findings of the joint training.
Similar bi-directional enhancements have also been seen in 3D spatial intelligence. Joint training has been effectively improved on a range of 3D spatial intelligence-related understanding and generation tasks.
These experiments together lead to one conclusion:
When understanding and generating dependent on shared latent knowledge, the two are more likely to form a true two-way migration.
The system level:
It's possible to link two models together.
Why do we need UMM?
At the system level, the authors discuss another very realistic problem.
If there is already a separate understanding of the model and a generating model, then in the face of a complex task that requires two capacities at the same time, it is entirely possible to combine proprietary models to complete the task together using an agentic pipeline.
So what are the advantages of end-to-end UMM?
The authors chose the reasoning-intensive image editing task to perform a rigorous, fair comparison between a given source image and an implicit editing request, where the model must first understand the image and user intent, then reason on the specific changes to the content, and ultimately produce results.
Under the same base model and under the same task supervision, the authors compared modular Planner→Executor with end-to-end UMM.
The results showed that the end-to-end model performed better overall on both RISEBench and KRIS-Bench.
This shows that when understanding, reasoning, and generating needs to be closely coupled, the value of a unified model is more than simply "a more convenient model to deploy".
End-to-end optimization may itself be part of the source of capabilities.
The Real Value of UMM
This work tells us that under a reasonable architecture, the value of UMM comes from the tasks that really need to be understood and generated at the same time.
One is at the task level, understanding and generating shared underlying knowledge and capabilities. For example, in physical intelligence, world modeling and policy/action learning rely on shared modeling of environmental states, physical laws, and dynamic changes, so UMM can be a direct combination of VLA+WAM to become a better model of physical learning.
The other is at the system level, where the task itself requires an understanding of continuous interactions with generating, such as various types of agentic generation and visually generated multimodal reasoning, which can become UMM's native capabilities, thus achieving higher upper limits and efficiency than modular systems.
