Anchoring on Reality: Breaking the Pseudo-Target Ceiling in Makeup Transfer
Team: vivo BlueImage Lab × Harbin University of Industry × Nanjing University
Keywords: migration tolerance, human beautification, virtual testing, limits on fake targets
The article links to:https://arxiv.org/abs/2606.31089
The main page of the project:https://csbowei.github.io/ART/
Imagine a specific need: to transfer the full appearance of one reference photo to another image. Not to the extent of replacing the mouth red number, but to the extent of the distribution of the stage lights, the orientation of the facial patterns, the position of the eye-angle diamonds corresponding to each other, while strictly maintaining the five senses of the migrants structure, expression and background.
This is the problem to be solved. After nearly a decade of development, the migration of everyday tears, shadows, and lipstick is relatively mature. But once faced with complex patterns such as stickers, bright films, artistic facial painting, existing methods generally fail: textures are blurred, patterns are missing, and even personality is changed at the same time.
ECCV 2026 Anchoring on Reality: Breaking the Pseudo-Target Ceiling in Makeup Transfer ART focuses on two core questions: what is stuck in this realm and how to break through it.
Figure 1: From everyday blurring to complex artistic blurring, from exaggerating facial expressions to facial blurring, ART is able to fully migrate to blurring while stably maintaining a personality identity.
Motion: The lower left corner is a reference frame, showing the transition from the source map to the result of the migration.
One, the root of the problem:
Without real couples, there is no real supervision.
To train a replacement model, the ideal data is a pair of samples: the same person, the same angle, the same expression, a color, a ribbon, only one layer of texture differs between the two images.
The problem is that this kind of data hardly exists physically. You can't have thousands of personalized photos taken in exactly the same light and posture.
Without matching data, the industry has taken two alternative routes.
The GAN method, which is represented by circular consistency, field contrast, does not directly tell the model what the target is, but instead trains with indirect constraints such as "make up and unpack and restore". This method can learn colors and overall atmosphere, but can't learn "put a drill in a specific position at the corner of the eye".
Route 2: Using large models to generate pseudo-targets. This is the mainstream practice of the widespread model era in recent years: ask a large image editing model to synthesize a "result map" based on color shots and reference shots, and use it as a supervisory target to train the student model.
But this path has a structural flaw, and as the method evolves it becomes increasingly apparent: every mistake in the pseudo-goal is inherited by the student model as it was. Producing the model loses bright details, the student learns to lose details; producing the model changes the facial structure of the character, the student learns to change identity.
This is the "pseudo-target ceiling" proposed by the paper: it is used to provide supervised synthetic images that ultimately become the ceiling of model quality.
The core idea of ART can be summed up in one sentence: surveillance signals should not come from synthetic pseudo targets, but should come from real photos themselves.
Second method: two stages of training.
Turn a real reference map into a surveillance signal
ART does not abandon pseudo-targets, but redefines its position in training: from the ultimate goal of supervision, degradation is the starting point of initialization.
Stage I: Initialization of pseudo-targeting. Using the "Source diagram, reference diagram, reference diagram, reference diagram, pseudo-targeting", the trio trained the migration model to master the basic capabilities of the model first: the spacing distribution on the face, color mapping, the general position of the stickers.
Stage II: Reality-anchored refinement. This is the core of the paper.
The result of the migration model output is not considered a final image, but is preserved as a layer of separable makeup carrier, which can be understood as an independent, mathematically guided "makeup layer". Then, "take" the layer of makeup from the source's face, and superimpose it on the colored version of the reference image (generated by the unloading model of Stage I), and ask a question:
Is it possible to rebuild a real reference map using this layer of migration?
If the migration is complete and accurate, the reconstruction results should be consistent with the reference diagram. As long as there are any omissions, shades are less bright, the location of the petals deviates, the border appears a false shadow, and the reconstruction error is exposed. This error is the monitoring signal: it spreads back to the migration model along the detachable carrier in reverse, indicating exactly where the container was omitted or painted incorrectly.
Figure 2: Stage I uses pseudo-objectives to build basic migration capabilities and train unloading models; Stage II preserves the result of migration as a tiny container, rebuilding a real reference map to reverse-correct the result of the migration.
The key value of this design is that the monitoring signal comes directly from the actual image itself, rather than from the imitation of the pseudo-objective. Every detail, texture, boundary, and spatial layout in the real image becomes a signal that can be directly learned. The learning goal of the model goes from "imitating the results of the synthesis" to "pairing the real image".
In Stage II, there is also a need to mark a spinning wheel: to start from where to start. ART starts with the imposition of controlled noise. The noise intensity determines how much of the model retains the false target precedent: noise is too small, the detail is constrained by the draft, it is difficult to get rid of its errors; noise is too large, freedom is too high, the character structure and background are easy to drift. Experiments have shown that 0.6 is optimal.
Three, MF2K:
The first 2K-resolution real-world scene to capture the data set
In addition to the methodology, the paper also paves the way for an infrastructure that has been missing for a long time in this field.
There are two short boards in the data set: low-resolution, thin-haired, light-film-like structures that do not physically exist at low-resolution, models that do not learn; narrow coverage, artistic facial painting, dense stickers, and real scenes such as male figures are seriously missing.
The research team collected about 100,000 high-resolution images from the Internet and supplemented the FFHQ dataset with original images on Flickr, followed by face-cutting, CLIP characterization, weighting, quality and gesture filtering, visual language model labeling, and artificial replication, ultimately creating MakeupFaces2K (MF2K): 8,573 2048×2048 images, covering four categories: 3,139 colors, 2,063 colors, 1,798 colors, and 1,573 colors.
Figure 3: The composition of the MF2K category, as well as the local details of the existing dataset - high frequency details such as hairs, bright spots, lip wrinkles - can only be retained in full at high resolution.
The fourth experiment:
Increased complexity and identity stability
The experiment evaluated four sets of MT, LADN, MT-Wild, and MF2K, with a total of 400 non-overlapping source maps. The reference maps for the MF2K test set were all from a collection of art samples that focused on the migration capabilities of complex patterns. The comparison approach covers three generations of technology routes: GAN method (PSGAN, EleGANt), dedicated spread model (Stable Makeup, SHMT, MAD), and general business model (Nano Banana Pro, GPT Image, 1.5):
Qualitative results. On a daily basis, the GAN method is often used for edge hardness and skin color distortion, and although large models have high image quality, they often change the structure of the person's face. The gap in complex faces is further widened: most methods only migrate overall tones, and content with pixel-grade standard answers such as stickers and facial painting is generally lost or distorted.
Figure 4: Qualitative comparison of daily intake.
Figure 5: The qualitative contrast of the intricate molding can be observed in high-frequency detail, such as the molding lines, the distribution of bright pieces, the shape of the stickers.
Quantitative results. The evaluation covers five dimensions: two large visual models, respectively, evaluate the resemblance of the lens (MSimG/MSimQ), facial recognition model, identity identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification, identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification identification and identification of the identification of the identification of the identification of the identification of the identification of the identification of the identification of the identification of the identification of the identification of the identification of the identification of the identification of the identification of the identification of the identification of the identification of the identification of the identification of the number of the identification of the identification of the identification of the number of the number of the number of the number of the number of the number of the number of the number of the number of the number of the number of the number of the number
Table 1: Complete quantitative comparisons on four test sets.
In a study of 21 participants, 864 anonymous blind reviews, ART was ranked first in all three dimensions of resemblance, identity consistency, and image quality, with an average score of 3.67 or 4.03 or 3.83 respectively.
Table 2: Results of user research.
Five: Direct verification: whether the ceiling was actually broken
It is better than other methods than breaking through the pseudo-target ceiling. It may also be better than the pseudo-target itself. Two sets of direct verification have been designed for this paper.
Verification one: the output is in direct contrast to the pseudo-objectives used for training. In the face of the same training sample, the ART is able to correct the head gesture changes in the pseudo-objective, wrong patterns and semantic hallucinations (the pseudo-objective had previously generated letters that did not exist in the forehead) and restore clearer capture details. The final output of the model is of higher quality than the object it imitated during training.
Figure 6: The red box marks gestural deviations in pseudo-objectives, wrong patterns and semantic hallucinations, and ART is corrected one by one.
Verification 2: Replace a weaker pseudo-target generator. Replace the pseudo-target source from Nano Banana Pro with a significantly weaker StableMakeup: the latter on MF2K has an MSimG of only 6.73, but the ART variant trained with its generating results reaches 8.56; replace the strong pseudo-target, the ART reaches 9.22. The quality of the pseudo-target affects the starting point, but no longer determines the endpoint.
Figure 7: Even from low-quality pseudo targets, ART is able to restore a clear, accurate complexity.
Another explanation that needs to be ruled out is the data advantage. For this reason, the research team used MF2K to re-train StableMakeup, whose MSimG is 6.58, and did not get the same improvement. The two stages of fusion pointed the same way to the method itself: MSimG, ID, L2-M at 8.82 or 0.57 or 7.68, respectively, only at Stage I. After joining Stage II, the improvement to 9.22 or 0.74 or 4.31 tons came from real image supervision, not data or initialization.
ART was the first framework to support 2048×2048 output clutch migration. The paper further compared 512,1024,2048 to three training resolutions: as the resolution increases, the adhesive edges, bright particles and wax detail gradually become clearer, and the small wave transformation loss introduced in high-resolution training further strengthens the learning of high-frequency clutch structures. This shows that 2K output is trained, not amplified.
Figure 8:512 → 1024 → 2048, the details of tears in the red box are gradually clearer.
Six, the limits.
ART currently has no explicit modeling of lighting and materials. When the lighting conditions of the source chart and the reference chart differ greatly, the high intensity of light in the reference lens may be transferred to an unmatched lighting environment, producing unreal local effects. This is one of the directions of subsequent work.
Figure 9: Failure case: High intensity light in a reference image is transferred to a source image that does not match the light.
Seven: Two revelations from this work.
It is worth noting that the two most important aspects of this work are seen later.
First, the competitive focus for image enhancement is shifting from the "general filter" to the "detail sense". Users' needs have long since passed the gray-white stage to the "restore of a specific appearance" stage, whereas stickers, bright slides, facial drawings are precisely the most refined and easily exposed scenarios of algorithm shorts.
Second, pairing data scarcity is a shared bottleneck for the entire real-world image editing field, and ART provides a movable solution. Changing hairstyles, de-wrinkling, ageing. Almost all real-world editing tasks are centered in the same place: it is impossible to obtain "the same person, only changing a single property". ART demonstrates a viable path: synthetic data is used to initialize and supervision is completed with real data.
Synthetic pseudo-objectives are not the answer, they are just the first step towards real distribution. The phrase applies both to capacity migration and to more restricted problems of matched data scarcity.
Author's note
vivo BlueImage Lab
The Blueprint Laboratory is primarily responsible for mobile imaging algorithm innovation, including cutting-edge research in image/video processing, image/video interaction, image/video enhancement, multi-modal understanding of large models, and more.
Dedicated to continuously enhancing the algorithmic capabilities of vivo mobile imaging, users can take clearer, more beautiful photos and videos. While actively exploring the applications of emerging technologies such as augmented reality, physical intelligence, and efforts to provide users with a richer and more convenient imaging experience.