Recently, recursive self-improvement (RSI) is rapidly becoming one of the areas of focus in the field of Agent.
In the past, when we talked about RSI, it was easy to think of a futuristic scenario: AI modifies its own model and then continues to improve the next generation of systems with its stronger ones. But really down to today's Agent, RSI already has a more realistic path.
How to write Prompt, when to call tools, how to manage Context and Memory, how to recover after failure, when to reflect, when to stop... The Agent Harness around the model itself determines how much the same basic model can eventually do.
So a very natural closing loop emerged: Agent execute tasks, observe failures, automatically modify Harness; Agent execute tasks again after modification, and then continue to modify according to new feedback.
But the question also arises: if an agent changes itself over and over again based on the same set of tasks, is it in the process of evolution, or is it just going to do more and more of the same?
Link to the paper:https://arxiv.org/abs/2609.24972
It's a bit of a mystery.
The main page of the project:
The biggest risk of the RSI: the stronger the change, or the more refined?
Existing Harness Evolution typically follows a very intuitive process: first let the current Agent execute on a set of missions, generating feedback based on success and failure trajectories; then let the LLM modify the Harness and evaluate the new Harness on the same set of missions.
The problem is that these evolve tasks will be used over and over again. The 1st Harness modification comes from these tasks, the 2nd continues to optimize based on the modified performance, and the 10th, 30th continues to do so. The whole process gradually becomes an adaptive search for limited data.
The paper notes that this leads to three types of clustering problems: benchmark-specific fitting, evaluation noise chasing, and complexity accumulation. Agents may remember the unique pattern of the current Benchmark, mistakenly attribute random points to real improvement, and even constantly add Prompt, context, and control logic to make a slight improvement on the evolve set in exchange for more and more test-time compute.
Eventually, a very counter-intuitive phenomenon emerges: evolution scores continue to rise, but after really leaving the Evolution Set, earnings decrease rapidly, even disappear.
In Figure 1, the improvement on the Evolution Set is not the same as the improvement on the OOD.
The real question is not whether the Harness can continue to evolve, but how much remains after these changes leave the task that has been repeatedly seen.
RRSI: It's not about limiting what the agent can change, it's about regularizing how it changes itself.
The key design of the Regularized Recursive Self-Improvement of Agent Harnesses (RRSI) is not to lock in the agent's self-improvement capabilities.
Prompt can be changed, Control Flow can be changed, Tool can be changed, Memory、Skill、Context Management can be changed, and even Subagent can be added or removed. In other words, Agent still has an open self-modification space.
The real regularize of the RRSI is how the Agent searches the mouse in this space. It breaks the Harness RSI into two questions: Proposal is responsible for deciding which mouse should try to change in the next round, and Selection is responsible for deciding which mouse changes actually qualify as permanent mouse.
Figure 2 shows that the RRSI simultaneously Regularize Proposal and Selection, rather than limiting the editable space of Harness.
At the Proposal end, the RRSI first limits how many intertwined modifications Self-Improvement can insert simultaneously. The former allows for a broader exploration, but as Evolution progresses, fewer and fewer mechanisms allow for simultaneous modifications each round, making each change in the latter more easily attributable.
At the same time, it will retain the full Evolution History: which hypothesis has succeeded in the past, which mechanism has failed, how much score has been added to a modification, how much cost has been paid, will continue to be evidence of the next round of Search.
The selection end is more stringent: a Candidate will not automatically become the next generation of Harness as a result of a score gain. Benchmark-specific modifications will be eliminated in advance; points in the evaluation noise range will not be easily written into a permanent state; additional Context and Token must be demonstrated with sufficient Performance Gain value; mechanisms that do not contribute in the long run will also be directly Prune.
In other words, the RRSI is not about stopping Agent change, but about requiring every change that you want to keep permanently to prove itself.
A very counterintuitive result: Evolution Score is higher and evolution is worse.
The most interesting thing about this paper is that it's not a specific benchmark that makes a few points, but the Ablation group below.
In Agentic Workspace, if Regularization is removed, ordinary Unregularized Evolution can push the evolve score to 92.8, which is higher than the RRSI of 90.5.
Unregularized Evolution's OOD Average was only 40.3, while the RRSI was 43.6. At the same time, each of the previous Trials required 3.80M Policy Tokens and the RRSI was only 2.42M.
In other words, higher Evolve Score does not lead to better Self-Improvement, but instead corresponds to worse Transfer and higher Inference Cost.
Figure 3 shows that after Regularization, the Evolve Score is higher, but the OOD is worse, the Token is more.
For the RSI, maximizing the improvement threshold may be an overfitting in itself.
What really matters is how much is left after leaving the Evolution Set?
The RRSI was ultimately validated on a total of eight benchmarks in the coding, agentic workspace, and engineering design triangles.
The most important experimental setting here is: Harness is only on one Benchmark Evolution, then freezes, no more modifications are made, and tests are directly placed on a Benchmark that has never been seen in the entire Self-Improvement process.
The results show that in Coding, Harness from Terminal-Bench 2.1 Evolution was upgraded from 82.0 to 83.8 on SWE-bench Verified; in Agentic Workspace, JobBench was upgraded from 36.0 to 40.7, GDPval from 48.8 to 52.3, APEX-Agents from 34.2 to 37.9; in Engineering Design, Frontier-Eng was upgraded from 17.7 to 22.0.
The highest OOD improvement reached +4.7 points, and all held-out splits were upgraded.
Figure 4 The RRSI migrates the improvements of Evolution across all three domains to the unseen Benchmark.
It's a better generalization without the need for a heavier harness
Harness Evolution also has a very realistic problem: Agents can make the system more complex by adding more instructions, more context, and more recovery mechanisms. Scores may increase as a result, but each time Inference becomes more expensive.
The results are intuitive: RRSI consumes about 2.42M Policy Tokens per Trial on Agentic Workspace, while Unregularized Evolution reaches 3.80M. RRSI is also in a better area with fewer tokens, higher OOD tokens compared to other Evolved Harnesses.
This suggests that the RRSI is not dependent on the Agent getting more and more for Transfer. Regularization, on the contrary, prevents the complexity of the increase in revenue support and removes mechanisms that are no longer contributing.
Figure 5 shows the horizontal axis as Policy Tokens / Trial and the vertical axis as OOD Average.
The 30-round evolutionary curve: the only real change left is 10 times
So what does it look like to be born with a regulated self-evaluation?
The chart below shows the 30-wheeled Harness Evolution on the Gemini 3.5 Flash on the Coding mission Terminal-Bench 2.1. Eventually, the Incumbent Harness increased from 64.6 to 78.7, an increase of 14.1 points.
In the 30 rounds of Evolution, only 10 Candidates were actually accepted and became the new Incumbent; nine Candidates did not produce enough measured gain, six were rejected during the Screening phase, two did not pass the Smoke Test, and three rounds did not produce a Proposal.
The light blue area in the chart is the noise-adjusted floor. In other words, the score of the Candidate is slightly altered and does not indicate that the Agent has made any real progress.
Figure 6 shows Gemini 3.5 Flash on the 30-round RRSI Evolution Curve on Terminal-Bench 2.1. The blue line is Incumbent and the light blue area is Noise-adjusted Floor.
Recursive Self-Improvement focuses not on making more changes, but on making changes that are really useful.
The RSI is moving to the next level, and the problem is not just whether AI can change itself.
In the past, when people discussed recursive self-improvement, the most natural question was: Can AI change itself?
But when the Harness RSI really starts to run, a potentially more difficult question is emerging: how does AI know if the modifications it's just made are real Progress, or Benchmark-specific Optimization? Once scores go up, learn a better execution mechanism, or just happen to use Evaluation Noise?
If Self-Improvement itself is an adaptive search process that uses limited feedback repeatedly, then it naturally encounters very familiar problems in machine learning.
The core idea of the RRSI is simple: Self-improvement itself requires regularization.
More importantly, Generalizable Self-Improvement: allowing the Agent not only to become more and more good at changing its own behavior, but to gradually learn to judge which changes are real progress and which changes are merely a matter of increasingly seeing itself.