Recently, have you posted a post entitled "How did I win the KDD Cup with DeepSeek!"?
The author of this post is Kaggle's classmate, the world's number one "invincible fork-burner". Recently, he and his teammates, who have won a combined 15 domestic algorithm competitions, captured the TAAC × KDD Cup 2026 Industrial Track championship with a $150,000 prize.
The championship team of the Industrial Racecourse is "The Sunshine of the Sun".
This competition, which we have also introduced before, is a competition in the Tencent Advertising Algorithm Competition (TAAC) linked to KDD (see: "Lunks" exploded, the token became "hard currency", the AI competition became more important), with a prize pool of up to $88,50,000 (approximately RMB 6 million).
The competition attracted 13,913 entries from 52 countries and regions, representing 5,746 teams, covering 907 schools around the world.
Last week, the competition was screened at the KDD convention venue on Jiju Island, South Korea, where the academy track, the industrial track, and the two original breakthrough awards were unveiled once upon a time.
The Academic Track Championship team lozyyeah.
As one of my classmates wrote, there was an interesting change in the contest: the contestants were almost all "AI Native", and large models and generating tools were naturally the foundation of their daily workflow, where they could not meet, with AI learning; the results didn't come up and let AI come up with ideas; code was written slowly, with vibe coding...
It's even more interesting that the "Daylight" team, made up of "Fork Burning Bag" and teammates, came up with a "pressure test" directly to the domestic model: without calling any API, without switching between GPT and Claude, with only one DeepSeek web dialog box running out of the "Champions' workflow", all over the web calling out "Shows", and posts were heavily redirected.
The Sunshine team used the DeepSeek web version of the game to play the overall process.
However, it is also mentioned that the convenience of the tool is one aspect, more importantly the "research mindset", which is also the key to the distance players in the competition.
The most intuitive feeling was that the 6 million-dollar theme was actually "squeezed out" and that almost every winning theme had its own original breakthroughs. These programs, even potentially allowing non-advertising LLM studies to benefit from them, were worth looking at one by one.
But before that, we have to go back to the beginning: what kind of question is this?
The difficulty of this question is:
It's two words for "unity".
The official theme for this year's competition is "The unification of sequence modeling and feature interaction for mass-recommended applications". It sounds winding, but it actually says one thing: using a unified modeling method to predict whether a user will actually take the next step after seeing an ad, such as clicking, buying, registering, or downloading.
This prediction is based on two main types of data: a user behaviour sequence, such as a continuously rolling timeline, such as a user browsing a circle of friends in the morning, watching the news at noon, having coffee at a small programming point in the afternoon, and following up on the way home; a non-sequential multi-domain feature, such as a user's "human settings", a list of ads, and some context signals and cross features.
Of course, for reasons of fairness and privacy, these data were de-sensitized, without any original text, images, URLs, or personally identifiable information. So the selector did not "understand the content", but in a set of anonymous IDs and vectors, ranking "people" and "goods" in the same representation space.
Note that the real difficulty lies in the two words "unity".
In the past, this prediction task was carried out separately: a model dealt with non-sequence characteristics (e.g., see who the user is), a model dealt with sequences of behaviours (e.g., see what the user has recently done), and finally put the two-sided judgments together. This way of "all-in-one summing up" would not only result in repetitive computation and higher systemic complexity, but also make it difficult to fully capture the finite interrelationship between sequence behaviour and multi-domain characteristics.
This year's theme of "Unified Modeling" is to find a new model structure that allows different shapes, different time dimensions of information to interact in the same model. It's like inventing a "super hybrid car" that combines gasoline and electricity in a unified energy expression at the bottom of the system to produce energy efficiently.
But it's not easy to build a "super-mixer". The official statement is straightforward: it's not even possible to unify all the features of a "forced embedding" network. It requires you to combine many non-sequential fields, contextual features, and behavioral flows with strict timing, across multiple behavioural domains, deeply integrated in the same architecture.
More demandingly, fish and bear control must be combined: the accuracy rate must go up, but also support deep stacking, and to implement the efficient training and reasoning of large batches on high-performance GPUs.
Rui Li, a Meta recommendation system expert who was invited to speak at the event, was very clear about the accounts, one technical account and one economic account.
The core problem with the technical book is: Agent can write code for you, order airline tickets, return emails, but will you let Agent browse videos for you, browse circles for you, chat with friends for you?
More importantly, the recommendation is a question that has no standard answer, and the signal noise is extremely high. So improving the recommendation system is more like a stress test, and Rui Li himself admits that "the devil is in the details".
In addition, it is because the recommendation has been bubbling in this noise for decades that we have practiced a set of "reading from noise behaviour data". And this set is probably the most lacking lesson in LLM personalization today.
In a scenario like this, an increase of 0.1% means billions of dollars and 0.01% is not a small amount of money. According to Rui Li, that's a lot more than the bonuses issued this year.
These values are the attraction of the race.
The championship program:
How to maximize the effect?
The first thing to do is to look at the two championship teams' schemes, which have solved the two most pressing real-world problems after the Unified Modeling.
Industry Race Champions: How to Create a Better Query
One of the most easily overlooked issues in the recommendation system was captured by the Industrial Track champion "Sunshine" team: Query quality.
We can think of historical behavior as a database, where query is a retrieval request that the model sends in. The sequence of requests is blurred and difficult to convert into a valid signal.
Although Cross-Attention is used to read user behaviour sequences, Query is largely produced by simple MLP, with little interaction between users, advertisements, and context.
So the team shifted the focus of optimization from "putting in more history" to "generating better queries", and thus came up with QueryFormer: first allowing users, ads, and other non-sequential features to fully interact, forming higher-quality queries, and then using it to find information that is most relevant to current ads from a sequence of behaviors.
Specifically, QueryFormer breaks down Dense Embedding from different sources by field boundaries to generate Token; then through Self-Attention it completes user-side, ad-side internal interaction, and then through Cross-Attention it models user-ad match relationships. Thus, before the model reads history, it knows what is facing "what kind of user" and "which ad".
In the sequence retrieval phase, QuerySeqCrossAttn uses ad-side Query to dynamically extract relevant behavior from four business domains; SeqQueryCrossAttn uses Cross-Attention to replace fixed MLP to generate Query.
The team also learned different viewpoints of Query from the same input using the Multi-Column Embedding Matrix. As the number of columns increased, the model effects continued to improve, but later gains gradually decreased, so the team ultimately chose a more balanced configuration of the effects and reasoning efficiency.
The end result validates the team's core judgment that, in the face of a long and tumultuous user history, improving the quality of "recovery issues" is more effective than simply increasing the length of the history or the scale of the model.
Academic Track Champion: How to transport useful information across layers?
"The Sunshine Side" solves "how to ask the right questions" and "lozyyeah" focuses on how to focus on intentions.
In their research, they found that existing unified recommendation models already allow user characteristics, ad characteristics, and behavioral sequences to interact with each other, but these modules mainly address "how this layer converges". As the network stacks up, user intentions may be overshadowed, sequence information may be overcompressed, and the resulting collapse and expansion effect may be unstable.
To this end, the team proposed CRAFT, which shifted the focus of research from Feature Interaction to Feature Transport. It not only considers how features communicate, but also controls how useful information passes through multi-layered networks.
Specifically, the model will first sort the inputs into non-sequence Token, sequence abstract and Intent Token. Each CRAFT Block will first aggregate non-sequence information, play it back into scaling, bias and residual gate control parameters, and dynamically adjust the Intent Token and sequence state; after this step is completed, the original Cross-Attention, RankMixer, and other modules will begin to interact characteristically.
In this case, Intent Token is no longer just used to retrieve query sequences, but is used as an information carrier across a multi-layered network. The model can both move it to a more appropriate representation space based on the current sample, and pull it back to the initial state when necessary, thus reducing user intent and sequence evidence that is blurred in multi-layer interaction.
Experiments have shown that as the CRAFT Block increases, the model effect steadily improves, and the display space of Intent Token becomes more abundant, indicating that useful information can indeed continue to be transmitted in deeper networks. However, this gain does not increase indefinitely as the model grows: modestly increasing the width effect is best for continuous expansion rather than stagnation.
Therefore, the value of CRAFT is not only to make the model bigger, but to provide a more manageable path of expansion that allows the network to retain and transmit really useful information as capacity increases.
If there is no ranking threshold,
What are the two original breakthrough awards breaking?
The two championship armies are working to solve the two real-world problems of how information can be "accepted" and "transferred", while the two original breakthrough awards continue to ask questions about how and how long the systems that carry this information should fit together.
It is necessary to specify the number of original breakthrough awards. Tencent officially assigned the two awards a special position, they are not tied to the overall ranking, and each prize is $45,000, specifically awarded to exploration in a technologically unmanned area. Even if the final ranking is not in the lead, as long as the program is bright enough, there are original breakthroughs, the same can be awarded. Last year's competition even temporarily added the award at the awarding site, in order to reward a ranking that is not in the top three but the program is outstanding.
Unified Framework Original Breakthrough Award winning team T-Squared.
Scaling Law's groundbreaking team won the G.G. Award.
This emphasis on technological originality gives the two breakthrough awards more space for exploration.
The Unified Framework Original Breakthrough Award: What are the communication boundaries between different features?
As we mentioned earlier, traditional recommendation models are often like a precise factory: non-sequential characteristics are processed by one module, user behaviour sequences are passed on to another module, until they are finally merged. Unified Framework Original Breakthrough Team T-Squared hopes to bring these steps together into a stream of flow, allowing users, commodities, contexts, and multi-domain behaviour sequences to interact earlier and more deeply in the same mainstream.
However, unification does not mean putting all tokens together as exchanges. Multiple behavioral domains differ significantly in meaning, length, and time patterns, and unrelated information can interfere with each other if fully connected Attention is adopted, and strong behavioral domains can suppress weak behavioral domains.
The model first builds Global Memory and Domain Memory. The former aggregates global information from different sources, while the latter retains separate summaries for each behavioural domain to avoid premature blending of domain characteristics. The DualQ module also revolves around current candidate ads, establishes links between users, goods and context in advance, and reproduces queries to suit different domains.
After entering the MaskRec Block, TopoMask Attention decides what information can be read by each type of token based on the default topological relationship. It is similar to moving the cause Mask in a language model from "time sequence" to "characteristic relationship": all tokens are updated in the same main sequence, but are not fully visible to each other, thus preserving uniform modeling and structure.
Experiments have shown that this unified structure of controlled communication is able to steadily enhance the effectiveness of structured organizations with tight characteristics that are particularly important: too tightly compressed to lose information, too thin to disassemble and too easy to dominate training, and to induce over-adaptation.
As such, the core value of TopoMask is not just to "put features together" but to design a reasonable way of connecting different information so that the communication is fully communicated and not blended to maintain boundaries.
Scaling Law's groundbreaking award: What should be installed in the new capacity as the model grows?
The Scaling Law breakthrough prize winner gg's team focused on a more fundamental question when recommending model extensions: where should new parameters be used?
Official Baseline has been able to uniquely process static features and behavioural sequences, but polynomial Dense and user image characteristics have been prematurely compressed, and details in multivalued features are easily lost in polarization. Continuing to broaden the model may only allow it to process the same limited amount of information repeatedly.
To this end, the team proposed CosFormer, the core idea of which is to extend the synergy between "information" and "model width". The traditional method usually keeps the input tokens unchanged, only increasing the parameters of processing these tokens; CosFormer requires that each time the model capacity is increased, the tokens be added simultaneously to a set of tokens that carries new information. In other words, the model not only becomes stronger, but also sees more valuable things.
These new tokens are organized into four categories of semantic information: static characteristics, explicit cross-characteristics, sequence-related characteristics, and universal characteristics. For example, Dense characteristics from different sources are no longer prematurely compressed, user interactions with candidate ads are pre-modeled, and time and historical window information forms separate channels.
As the information evolves, CosFormer also addresses "how to use them well". MaskNet-driven binary-flow architectures will use different channels depending on the current sample selection and overload; Token-type Private FFN allows the same Token sharing network and different semantic types to be processed separately. This will both improve the level of specialization of the model and avoid over-adaptation for each Token.
In experiments, information-width synchronization expansion yields sustained and stable gains, with empty tokens that simply add parameters or add no new information quickly encountering bottlenecks. As new information gradually repeats, the marginal gains of the model also decrease, forming an experiential "information frontier".
This illustrates that the Scaling of the recommended model cannot only look at parameters: only new capacities carrying new effective signals and supporting the corresponding information selection mechanism can be truly transformed into effect enhancement.
Putting the four schemes together, the levels are clear:QueryFormer solves the input end problem (how to ask right), CRAFT solves the transmission layer problem (how to secure), TopoMask redefines the connectivity rules of the architecture layer, and CosFormer touches the boundaries of the rules of the extension layer. From "how to ask", to "how to connect", to "how to extend", they spell out a main line of technology from local to system, from the present to the future".
In addition to the $6 million prize pool,
What did the game really leave behind?
Jumping out of the game itself, from an external point of view, the game is truly immersed in two things: the same for the industry and the same for the people.
Rui Li made it clear in his speech that as long as we continue to invest in computing power, the recommended model can continue to recycle profits. That is, the logic of the Scaling Law, which has been verified in LLM, has been systematically moved into the recommendation, which is equivalent to illuminating this path once again.
Second, it's data. Recommendation research has been running slowly for so many years, and one of the reasons why it hasn't been easy to get around is that it's hard to get industrial data. Rui Li mentioned that when he read it 20 years ago, he couldn't find a single set of recommendations that could be used for research. LLM is running fast, largely because it has open data, open benchmarks, and not in the field of recommendation.
After connecting with KDD, the player platform is no longer a company-owned racetrack, but an international arena. Everyone competes on the same platform with more than 10,000 players from 52 countries and regions.
What's more true is the second half: the end of the race isn't a penalty shootout, Tencent doesn't see it as a one-off selection, but instead places it in a long-term talent layout: the race is responsible for getting global tech talent to communicate, demonstrating ability, the Blue Cloud program is responsible for continuous attention and support after the race, covering discovery, absorption, cultivating several links.
Waiting for next year's race, and today's proven technical routes, may be the starting point for the next round of competition. By then, how far will the new AI Native players push the recommendation system?