Today's AI has been able to help you code, run experiments, and evaluate. Every day of an algorithm engineer's life has almost turned into: having an LLM organize the results of yesterday's experiments, having an LLM implement new ideas, having an LLM monitor run experiments... it seems like just telling the need to tell AI that it can do most of the algorithm work.
At present, the only thing that has not been replaced by AI is the creativity of algorithm engineers, which is the ability to find ways to improve AI systems. Usually, the next step for the model to change is highly dependent on the experience and intuition of the algorithm engineer.
In contrast to using a general-purpose model directly, Astar offers a different approach: to deepen the study of the evolutionary history of the AI system itself, the submission of code and the results of experiments that are scattered throughout history. After internalizing these real evolutionary experiences, Astar is able to adapt to explore future evolutionary directions based on the current state of the AI system.
In practice, Astar is able to increase the efficiency of algorithm engineers in generating new ideas by 10 to 100 times, and its proposed evolutionary direction is significantly better than that of real experts and general base models.
Based on Ali's core recommendation business, Astar is even better at hitting the offline HitRatio score of 23.6%, and the online GMV score of 4.86%.
Address of the paper:https://arxiv.org/abs/2608.27287
An inevitable dilemma: AI's generation is too dependent on boring experiences
It is well known that there are significant differences between ordinary software engineering and AI models.
Modifying a piece of ordinary code, running a test case, can take a few seconds. But modifying an industrial-grade AI model often requires an extremely expensive verification cycle. From modifying the code, retraining to offline evaluation, it usually takes hours or even days.
In a bunch of seemingly feasible improvements, which of them should be given priority over valuable computing?
In the past, this relied heavily on the intuition of senior algorithm experts. Experts needed to understand the business, data distribution, model architecture, and training dynamics of the future. The rapid evolution of a team model was largely hampered by this scarce experience.
Some might ask, since today's generic model is so smart, why not just throw the code at them and think about it?
However, the complexity of real industrial scenarios often exceeds expectations. Large generic models absorb a large number of public papers and open-source code, which delve deeper into various avant-garde principles, but in the face of specific business data and highly customized existing systems, this complexity of generic knowledge is sometimes difficult to translate directly into a real-world experience.
The iterative cycle of AI systems still relies heavily on artificial experience, and think of the idea as a link in the chain.
Another idea: to go to the system itself, to mine in the caves of evolutionary history.
Since general knowledge is not perfectly adapted, where is the real field experience? The Astar team has captured an extremely valuable source of data: historical versions of AI systems themselves.
Every time Git is submitted, it fully records what the model has changed from the previous version to the next, and the resulting loss curve and decline in business indicators.
Following this thought, the team decided to turn these experiences scattered throughout version history into model energy training data. But to really run along this path, Astar had to cross four mountains:
(C1) Too little data: the real iterative record of a warehouse is very limited, only looking at the adjacent two submissions of modifications, and the available surveillance samples are poorly available.
(C2) The full library is a watershed: a large amount of code submitted is actually just logging, changing file paths, clearing unnecessary code, and it is difficult for the model to learn the core algorithmic evolutionary logic if it is fed directly to the model.
(C3) Space is too big: the improvement direction of the model is like a sea of smoke, is it the dynamic architecture? change the loss function? or change the optimizer?
(C4) Verification is too expensive: it takes a lot of trial and error feedback in reinforcement learning to make AI learn to explore on its own. But running a real verification experiment takes days and the cost of verification is unbearable.
In the face of these four major challenges, how did Astar break through one by one?
Astar's Exercises on Dancing with the Stars
In order to remove this hard bone, Ali and his team designed an extremely clever flow line that refined the theory of evolution from the seemingly chaotic submission of engineering code.
There's too little data to break: two or two combinations, so data can be broken.
Since there were too few changes to the two adjacent submissions, the Astar team adopted a clever two-by-two enlargement strategy. They were no longer limited to the evolution of the adjacent versions, but instead matched any two experiments in history (such as versus version 1 and version 5) and then pulled out their complete loss curve, whose loss was lower and whose version was better.
This not only instantly expands scarce records into intensive training samples, but also enables models to learn both short-term mitigation techniques and long-term mitigation and cross-cutting upgrades.
It's been a long time since I've been able to do this, and it's been a long time since I've been able to do this.
In the face of a mixed amount of unused code, Astar set up a two-way scrubbing process.
The first method is to perform logical filtering: through code-calling graph analysis (Reachability) and abstract grammar trees (AST), stripping all printed logs, dead codes and format adjustments.
The second way is to filter evolutionary intentions: use LLM to make semantic analysis of the remaining pure code. If a code change cannot be categorized as a clear filter of optimized intentions, it is filtered out as noise.
Space is too big to crack: three-tiered guidance, layer-to-layer evolution
Astar's solution is: layered generation.
During the training, the team played stringent hierarchical hints to the data: hierarchical hints: primary direction -> secondary segmentation modules -> thirdary specific actions.
This allowed Astar to think like a top architect in reasoning: it doesn't start by blindly changing parameters in a huge code library, but first determines the main direction (e.g. a cluster optimization strategy), then focuses on segmentation modules (e.g. a cluster Muon optimizer / gradient convergence), and finally implements specific actions (e.g. a cluster gradient noise reduction cluster).
Breaking verification is too expensive: offline betting, one second
In order to address the costly fatal wounds of reinforcement learning, the team used historical positive-negative samples to train a reward model (Reward Model). Faced with a generated evolutionary scheme, this reward model can predict the probability of reducing loss in just one second.
With this secondary-level vacuum agent, the system is not only able to generate some of the most reliable on-board validations at light speed in mass inspiration, but also provides Astar's reinforcement training (RL) with a continuous source of low-cost feedback that enables it to proactively explore new directions that its predecessors have never attempted.
Learn about the four challenges facing evolutionary direction and the four designs that Astar responds to from the backdrop of modern history
It's a small model, but it's also a big one.
Based on the above carefully processed evolutionary material and efficient proxy reward models, Astar is able to conduct standard three-phase training: intermediate training (mid-training)→ supervised fine tuning (SFT)→ reinforcement training (RL) and demonstrates amazing evolutionary guidance capabilities.
In the face of the same recommendation system optimization scenario, the general large model (including the front-end GPT-5.5 ‧Claude-4.8-Opus, etc.) can provide direction, but there are some limitations to the direct execution success rate in real business.
Even the smallest version of the parameter, 0.6B (~600 million), has a success rate of 54.35% to generate effective single optimization directions (S@1)!
Not only did Astar significantly outperform the strongest generic model GPT-5.5 ((30.71%), and even overwhelmed senior human algorithm experts ((32.29%), but the 8B version of Astar also boosted the single-generation success rate to 67.86%!
This data reveals a trend: in the field of vertical engineering decision-making, the development of high-quality models, embedded in real-world system feedback, is a key path to performance improvement, as is the blind increase in the size of the model's general parameters.
Production quality comparison diagram, comparison of Astar with the general large model, human experts. RM@k represents the success rate of k iterations based on the reward model ratings, and S@k represents the success rate of k iterations under real execution.
The efficiency comparison is even more exciting. In the past, human experts have come up with a reliable idea, written code, etc. training results that can validate several to a dozen directions a week. And now with the Astar+ reward model's first-second startup + automated execution of verification rings, the test cost of single-handed ideas drops from a few hours to a few days, plummeting to a few minutes.
Comparison between traditional algorithm workflows and Astar's time consumption
To verify its true potency, the team directly let Astar take over the evolution of the Ali Lazada's core ad recommendation system. Without any human intervention, Astar led 20 rounds of fully automated iterations, not only improving the Hit@200 rate to 23.6% on offline indicators, but also driving the GMV (commercial transaction total) to 4.86% improvement, advertising revenue to 1.82% improvement, and hit rates and orders to a significant increase in sync!
Know that in an e-commerce advertising system that's so mature, so huge, and so powerful that it's been repeatedly manipulated by countless smart minds, AI's own improvements have been able to extract gold and silver enough to make the industry sweat a little!
Astar's improved Lazada Ad Recommendation System has significantly improved several core business indicators online
So, what did Astar give you?
You might be wondering, where is Astar's advice better than General Key Models?
One of the characteristics of Muon is that it forces the weight of all directions in the gradient by positive synchronization. If the data is clean, this characteristic is excellent; but in situations of noise and random recommendation, this kind of noise can only increase the noise that should be suppressed.
In the face of this situation, the general suggestion of modification is often to adjust the learning rate of the fluid or to return the fluid to the Adam optimizer fluid. However, Astar gives a highly mathematically beautiful targeted modification scheme: Spectral Denoising is done before the fluid is positive.
It proposed using the Marchenko-Pastur law of random matrix theory to calculate a definite mathematical boundary: a signal above this boundary, normally flattened; a signal falling into the noise distribution range, directly compressed. This not only perfectly preserved the advantages of the Muon optimizer, but also smoothly solved the noise amplification problem, without adding any additional training parameter burden. This improvement of the puncture point was based on Astar's deep understanding of the characteristics of the system's data distribution and the mathematical nature of algorithms.
It's written in the end: Open the Endless Cycle of Evolution of AI
Giving a high-quality recommendation is just the beginning. The team devised an ingenious closure for Astar: Astar offers directions -> Code Agent writes code running experiments -> verification results backflow -> re-engineers Astar as a training data breastfeeder. This means that Astar is a living model: its guidance capabilities will also grow as the system evolves.
In the past, we have relied heavily on the brain power of algorithm experts in this most central area of evolution of AI systems. The advent of Astar, which paved the way for the key puzzle of automated evolution, has enabled AI to optimize decision-making for the first time with traceable, portable pathways.
But this does not mean that algorithm engineers are retiring. On the contrary, it shows that a new, more advanced distribution model is coming.
When AI tries to fine-tune and test-tune models like Astar, future algorithm engineers will be freed from the tedious shifting labor. Humans can put more energy and creativity into breakthrough architectures, defining new business models, and designing smarter mold flywheels. The path to automated evolution of AI systems is just beginning, and the wisdom of algorithm engineers remains an indispensable leader in this quest.