Vice President of Qualcomm Technology: Where is the controllability of image generation?

This article is part of the PRO Members Communication, and follows "PRO Members of Machine Heart" at the end of the article to read more on the topic.

In today's field of generative AI, T2I technology has evolved from the cutting-edge research of computer vision to tools that are easy to use by the general public, producing results that often "look great" visually. However, when users demand accurate identity retention, complex spatial constructs or ultra-high-resolution outputs, existing models still frequently expose systemic flaws such as confusion, command neglect and memory explosion. These flaws cause practitioners to be unable to integrate models into professional workflows.

Fatih Porikli, senior vice president of Qualcomm Technology, was recently interviewed to discuss unresolved issues in the field of image generation, as well as several approaches to address these challenges proposed by his team at CVPR. Porikli pointed out that the next frontier in image generation is to bridge the gap between "realistic images" and "precise results" to make the system manageable, efficient and reliable, thus consistently generating high-quality images that match the user's true intentions.

Directory

01. Why is image generation trapped in the trap of "looking good"?
What specific technical challenges are faced in terms of controllability, quality and efficiency of image generation?...

Why is image generation trapped in the trap of "looking good"?

For example, when a pronoun requires generating multiple characters, models tend to generate highly similar facial features, hairstyles, and even expressions for all characters. The Porikli team proposed the Disco "Diversity in Image Generation" program, which minimizes existing base models by reinforcing learning "RL" to optimize "identity diversity" itself as a goal.

1 The Porikli team, in their paper "Disco" introduces the group relative strategy optimization (GRPO) and designs a reward function that simultaneously encourages "diversity in the diagram" (different faces of different characters in the same diagram) and "cross-sequence diversity" (different faces of the same pronouns are generated multiple times) and strictly restricts the accuracy of the number of characters.[2-1]

2 Assessments show that Disco has a 98%-99% "unique facial accuracy", a baseline model that far outstrips the goal of explicit diversity. At the same time, the team has also released a set of DiverseHumans benchmarks to promote community standardization assessments.

2· Faced with the problem of having to deal with multiple tasks in a single model, Porikli argues that for human artists to paint, they should first plan the architecture, then render the details, and so should AI. In their paper "Ar2Can", the team of Porikli drew on the creative experience of human artists and proposed the architectural logic of rendering the scene planning and the details of each other.[2-2]

3 Ar2Can divides the image generation process into two stages. First, the Architect module predicts structured layouts based on the pronouns (such as the coordinate position of the three characters) and then the Artist module synthesizes realistic images based on the space plan.

1 The "Architect" module contains two variables. "Architect - A" uses a supervised Qwen model to predict boundary frames. "Architect - B" uses an enhanced morphology-optimized Flux-Schnell model to self-planning human posture, thus producing a set of boundary frames (center coordinates) with an optional posture skeleton.

2 The "Artist" module is based on the spatial planning provided by "Architect" to synthesize the final photo-reality images. It uses group-relative strategy optimization (GRPO) training and uses a combined reward system to combine spatial alignment with identity similarity to ensure accurate facial rendering and identity is maintained.

3 Architect-A's accuracy on the MultiHuman-TestBench was 90.2%, with Anyone and GPT-Image-1 scoring 89.8% and 87.9% respectively. Architect-B's multi-identity similarity was 68.2%, and the open-source contrast method MH-OmniGen listed in the paper was 54.5%.

Based on the limitations of the existence of a single image generation model, Porikli believes that future image generation will move into the "Agentic (intelligent embodiment) " pipeline. The system no longer relies on a single giant model, but on cue word characteristics to route tasks to expert models that specialize in dealing with diversity, text generation or specific styles, to achieve optimal output by sorting.

How can reinforcement learning and task-solving enhance the "controllability" of generations?

1. The image resolution of most top-generation models is currently limited to about 1K (1024×1024). If the scattered model is to be used directly to generate ultra-high resolution images of 4 million to 16 million pixels, existing conventional programs will either require the continuous preservation of features in hidden spaces that are multiplied on a spatial geometric scale, triggering a severe cache explosion, or require long computations that take minutes or even more than 10 minutes in the cloud.