Multi-cycle image editing allows the user to gradually adjust objects, appearances, and scenes in the same image through continuous instructions. As image generation enters interactive creation, the user needs to continue to modify based on intermediate results, and the model needs to retain previously confirmed content while completing new requirements.
However, existing models tend to have significant deviations after a few rounds: textures are increasingly heavy, local details are gradually distorted, and areas that do not require modification may also change. These problems, respectively, involve graph stability and consistency of content across rounds, limiting the ability of the model to continue editing.
Researchers from the Mercer Institute, the MMLab at the Hong Kong University of Science and Technology, Stanford University and the Institute of Intelligent Industry at Tsinghua University have jointly proposed the multi-cycle image editing training method MT Forcing to improve the problem of image degradation and content drift in continuous editing based on the unified multi-mode model of regression scattering.
The paper's main contributions include three points:
1. Proposes Single-Turn DMD, which uses single-cycle instructors to guide students to multi-cycle self-generation, improving image degradation in continuous editing.
2. Introducing the Teacher-Forcing Guidance, which helps the model retain the cross-cycle content through preheating and normalization over a consistent reference trajectory.
3. Combining the two types of complementary targets into MT Forcing to verify quality stability, content consistency and improvements to follow instructions on SenseNova-U1 and SenseNova-U1.5.
- MT Forcing: Towards Stable and Consistent Multi-Turn Image Editing with Autoregressive Diffusion
- Address of the paper:https://arxiv.org/abs/2610.25404
- The main page of the project:https://houyuanchen111.github.io/mt-forcing.github.io/
- The code is:https://github.com/houyuanchen111/MT-Forcing
- Model weight:https://huggingface.co/houyuanchen/MT-Forcing
Why is the change getting worse? The history of training and reasoning is different
Degradation in continuous editing originates from exposure bias. In training, the model typically edits the next round of study based on a given reference history; in practical reasoning, it needs to continue to modify based on its previously generated images.
This distinction can be expressed using the conditional distribution in the paper:
H⋆ represents the reference history, H represents the history of the model's self-generation, and ci represents the current instruction.
The content in reference history is usually coherent, accurate, and self-generated history may already have textural biases, pseudo-shadows, or accidental modifications. If models are mainly studied in reference history, there is a lack of experience in dealing with these biases; errors may continue to accumulate once incompletely accurate output is taken as the next round of conditions.
Therefore, MT Forcing allows the student model to self-rollout during training: each round follows its own output editing, so that the training covers the actual history encountered when the reasoning.
Single-Turn DMD：
Transmitting single-cycle quality to multi-cycle editing
MT Forcing takes advantage of two observations: the typical loss of image quality from single editing is limited, and the editing model can perform multiple operations in a single instruction. Therefore, teachers can start from the original and complete the cumulative editing requirements up to the current round each time.
For example, the mother adds a vase and then asks the mother to turn it into a blue vase, which can be organized into a blue vase. The teacher receives instructions to integrate the prototype and the students continue to generate it in the original order. Both correspond to the same cumulative editing intent, but the teacher's supervision is not affected by the cumulative effects of recurring editing errors.
The students then edited their own histories in stages; single-stage teachers provided distribution matching supervision, reference trajectories and content consistency guidance.
Single-Turn DMD compares the output distributions of students at each turn independently with the output distributions of teachers at each turn.
The same pattern with the integrated instruction constitutes the distribution of the teacher condition CT. Pθ and pφ respectively correspond to the distribution of students and teachers, and t is the spread noise time; the formula compares the distribution differences between the two in the same noise-adding process.
This matches the distribution, without requiring students to pixel-by-pixel reproduce a teacher's image. Students learn to deal with biases in their own history independently and maintain output quality through teacher supervision, thereby mitigating differences between training and reasoning.
Teacher-Forcing Guidance：
Complementing the links between history and output
One-wheel teachers were able to provide clear results, but the integration instruction did not specify all the visual details that had already been established in the previous rounds.
In the case of teapot in the paper, teachers independently execute the cumulative instructions for each stage, each picture being clearer, the shape and arrangement of the flowers changing in succession.
The content changes when single-wheel teachers edit independently, enlarging the region to show the differences in teapot patterns.
Teacher-Forcing Guidance uses a coherent reference editing trajectory to monitor the correlation between history and the output of the next round. First, through the teacher's forced preheating, the model learns to retain the cross-cycle content; then it continues to retain low-weight reference oversight in DMD distillation to avoid losing this ability in distributed matching updates.
The objectives of the joint training are:
The DMD module guides the autonomously generated output distribution, while the TF module monitors the constraint of the cross-cycle content by reference to historical flow matches, and the lTF controls the weight of both.
The two supervisors, respectively, work on generating distribution and historical editing behaviour, training the same student model together. In dissolution experiments of papers, only DMDs will result in textural changes, and only reference supervisors will result in later textural degradation; the complete method improves on both types of problems.
Qualitative Results: Reducing False Shadows and Maintaining Modifications
The paper compares the two basic models in the 1st, 4th, and 8th-wheel output. Towards the 8th-wheel, the car scene on the baseline shows filtering and illumination with rough textures, while the snowmobile scene shows spots and excessive contrast.
With the addition of MT Forcing, the image texture is cleaner, the landscape structure is more consistent, the previously completed red body, the roof trunk, as well as the tents in the snow, ski boards and road signs are preserved.
The two basic models are the results of editing rounds 1, 4, and 8, with the left side being the baseline and the right side being the results after MT Forcing is added.
Additionally, the longer number of rounds before and after training can further illustrate the effectiveness of MT Forcing.
Quantitative results: Eight rounds in a row, with later gains more apparent
Evaluated using a gradual editing subset of the MTEdit-Bench, consisting of 100 group sessions, 8 rounds each. The research team compared the two basic models to incorporate MT Forcing following the following instructions: IF, content consistency, CC, paint quality resistance to drift, DRS, and geometric alignment.
For SenseNova-U1.5, the eighth-round IF was upgraded from 4.52 to 9.17, the CC was upgraded from 3.22 to 7.68, both scoring 10 points. The DRS scored 5 points, the baseline dropped from 4.09 in the first round to 3.03 in the eighth round, while the MT Forcing remained at 4.20 and 4.35 in the eighth round.
Quantitative contrast of MT Forcing
SenseNova-U1 also received improvements, with the eighth round of IF and CC raising 7.45 and 6.82 points respectively. Starting with the fourth round, the four indicators of both models were better than their respective baselines.
The paper also measures geometrical changes using the average displacement of adjacent outputs between matching points. The eight-wheel average of the SenseNova-U1.5 decreases from 6.34 pixels to 0.74 pixels, a decrease of about 88.4%, indicating a more stable spatial position in the matching region.
Introduction by the author
The first co-author of the paper is Dr. Chen Hsiangwu, PhD student at MMLab of the Hong Kong University of Science and Technology, and Wang Van Zhou, director of the project, Liu Xiaou of the Hong Kong University of Science and Technology. The study was completed during Chen Hsiangwu's internship at the Hong Kong University of Technology.
© THE