An e-commerce image, which is both accurate to show merchandise, and also fits the user's preferences, carries the business expectations of value transformation. When image editors begin to read user preferences, can models transform different creative intentions into common sense e-commerce material that can be used to further stimulate business value potential?
KwaiMind is an image editing baseline model for real-world e-commerce content production. It combines general image editing and e-commerce expertise to meet diverse creative needs while retaining product details, accurately presenting text, and supporting the construction of an e-commerce task evaluation system that forms a complete loop from data production, model training to performance evaluation and business feedback.
According to the technical report, KwaiMind scored first in multiple general image editing benchmarks; in the self-build e-commerce review Ecom-Bench, Visual Total was ranked first in the open source model review. In the online commodity mainframe material preference A/B experiment, KwaiMind brought about a relative improvement in the actual click-through rate of about 2.44%, demonstrating the commercial application potential of e-commerce image editing.
Figure 1 Comparison of Ecom-Bench visual scores with Ecom-CTR rankings. CTR is the predictive cumulative number of rankings.
From General Editing to Targeted Repair
Covering real material production needs
KwaiMind will add deletion, replacement, configuration adjustment, and localization problems to the same set of instructional editing capabilities, supporting clothing testing, product detail display, decorating wear, background replacement, action adjustment, etc.
Figure 2 KwaiMind editing cases, including clothing, accessories, merchandise scenes and general image editing.
As an example of an e-commerce point of sale chart, the model needs to adjust the background and composition and render text according to the commodity chart and the editing instructions, while highlighting the visual performance of the commodity, while maintaining its appearance, material and key details. These requirements also test the general editing ability of the model.
Figure 3 KwaiMind scored the highest overall score in the open-source review in all four groups of general editors.
Collaboration between four intelligent agents
Maintenance of about 1.8 million high-quality training samples
KwaiMind has built a multi-smart body data engine, which integrates quality filtering, sample repair, directional addition and instruction marking into a single process, enabling tasks to be automatically streamed, and samples to be traceable. The four types of intelligent body work are:
Filter Agent: Pre-filter the input image and duplicate the editing results;
Generation Agent: Recoverable samples based on diagnostic repair, supplementing with insufficient coverage of task data;
Caption Agent: Generate editing instructions of different grades;
Coordinator Agent: Maintain the status and distribution of samples, coordinate task scheduling, process welding and retesting.
At the same time, the system introduces the Human-in-the-Loop mechanism, which transfers low-confidence, high-difference samples through artificial audits, and reverses the process of production of the results of the audits, combining processing efficiency and data quality.
Figure 4 Selected task composition and representation samples of SFT data.
Multi-intelligent data engines constructed and maintained about 1.8 million high-quality training samples from about 26.2 million candidate data for use in the Continued Fine-Tuning phase. They further selected about 23.49 million pairs of data for use in the SFT phase, including 11.5 million for general editing samples and 11.99 million for e-commerce samples. The basic editing capabilities of the general data rigidity model, while e-commerce data reinforces specialized capabilities around clothing details, merchandise display and text editing, jointly enhancing the model's compliance with instructions and image aesthetics.
Special optimization with multi-teacher distillation
Integration into a unified base
Real e-commerce tasks often require models to do many things at once: execute editing instructions, retain product details, write text, and generate images that are more attractive to users. These goals vary in the difficulty of optimizing and the rhythm of convergence.
KwaiMind uses a combination of specialized capability optimization and multi-faculty distillation: in addition to general editing capabilities, the model further strengthens visual quality, product consistency, text accuracy and commercial click-through attractiveness. DiffusionOPD brings together the capabilities of different experts into a unified model.
Figure 5 Training Framework with DiffusionOPD Multi-Teacher Distillation. Specialized teachers provide guidance on the students' own noise trajectories.
DiffusionOPD is an online distillation strategy that is gradually guided by teachers along the noise trajectory of students to bring the ability to migrate closer to the actual generation process of students. KwaiMind integrates general editing, word processing, commodity retention with user preferences into the same model, allowing a single editor to accommodate multiple requirements, such as retention of packaging details and text when changing context, while generating a more user-friendly display effect.
Ecom-Bench：
System defines e-commerce tasks and evaluation systems
E-commerce image editing requires clear task boundaries, as well as a set of evaluation criteria that can determine whether material is available.
For this purpose, the team built the Ecom-Bench, which included 11 tasks, 1,100 cases, and 100 cases per task. These tasks included virtual testing, dress details, gesture adjustment, pattern wearing tasks, background replacement, product extraction, point-of-sale display, and poster-related tasks such as word editing and propaganda removal.
Around these tasks, Ecom-Bench has established a comprehensive assessment system consisting of six general dimensions and eight e-commerce special dimensions:
Adherence to the General Dimensional Attention Guidelines, integration of naturalness, physics and light rationality, non-edited area retention, image quality and overall aesthetics;
The e-commerce dimension focuses on checking the identity of goods, texture accuracy, fabric texture, patterns and gestures, clothing fit, graph edge, point of sale communication and product display integrity.
For each task, two general dimensions and two e-commerce dimensions are selected to determine the assessment focus according to the task's requirements. The following table shows the specific task measurement dimension allocation:
Table 1 Ecom-Bench Assessment Dimension Allocation of each task
When scoring, the review model combines reference charts, editing instructions and generating results to give an integer score of 1-5 for each of the four dimensions according to a clear scoring criterion.
The combined scores of individual cases use geometric averages to make critical defects such as text errors, commodity identity deviations more noticeably affect the combined scores, reducing material availability issues covered by single high scores. Each task is scored as an average of the combined scores of 100 of the tasks, with a total score equal to the average for 11 tasks.
As shown in Figure 1, KwaiMind's Ecom-Bench visual aggregate is the first open-source model within the report's rating setup and rating model. In addition to visual quality, the team also evaluated the predicted click preferences of the material generated by Ecom-CTR. The output of different models was ranked, with the number of times each model entered the top three cumulatively as CTR points to measure relative commercial attractiveness. KwaiMind's performance on this indicator reflected the model's consistency with material availability and user preferences.
Below is a visual comparison of the materials, covering the point of sale exhibitions, clothing details, clothing exhibitions and virtual trials. In combination with reference graphics and editing instructions, you can intuitively see the differences between the different models in the production of commodity materials.
Figure 6 Exhibition of clothing
Figure 7 Detailed clothing chart
Figure 8 Map of points of sale
Figure 9 Virtually Testing
Verification of business:
Material generated to real click enhancement
In order to verify the application value of KwaiMind in the actual business, the team initiated A/B experiments on the commodity mainframe material selection line. In the experiment, KwaiMind brought about a relative improvement in the actual click-through rate of about 2.44%.
In offline evaluation, for random quality sampling of commodity charts, KwaiMind's estimated CTR was higher than the proportion of the original commodity charts, up from 12.16% before training to 37.41%; compared to unoptimized baseline charts, its estimated CTR win rate was 91.89%.
Summary
The introduction of KwaiMind allows us to see more possibilities for image editing models to enter the actual workflow of e-commerce. From experimental demonstration, background adjustment to product detail retention and text modification, the model needs to understand not only how to change the screenshots, but also how the product should be presented, and what content should be more attractive to users. For developers, the unified editing base and supporting assessment system provides a solid foundation for e-commerce applications and production workflows. For entrepreneurs and authors, these capabilities to combine into the same model means reducing the cost of repeated production and modification, building bridges between creative ideas and real available materials, and leaving more space to try different ways of presenting products.
In the future, the team will continue to expand the scope of tasks, improve the robustness of dedicated reward models, and enhance the ability to edit combinations and multi-reference graphics. As models continue to improve and gradually integrate into everyday creative tools, the creation, modification and optimization of e-commerce materials are expected to become more convenient.
The project and the KwaiMind Technology Report have now been released:
The KwaiMind project:
Technical Report:
This article uses AI generated and AI edited diagrams to show product capabilities only, and does not represent real people, real goods or real business cases.