It's the first time in the history of the world.
Over the past two years, concepts such as self-reflection, self-correction, self-play, agentic RL, skill learning, test-time adaptation, open-ended evolution, self-evolving agents have emerged.
They sometimes refer to models continuing to train, sometimes to Prompt or tool automatic optimization, and sometimes to recursive self-improvement (RSI).
The problem is that the levels and intensities of change described by these terms are not the same, but there is a long-term lack of a unified coordinate system.
Recently, a team of researchers from institutions such as the University of Guelph, KAUST, the University of Alberta and the Swiss AI Laboratory released a review that redefined "self-improvement" from a systemic level, and put model training, memory evolution, tool creation and Agent architecture searches on the same map.
The article links to:https://arxiv.org/abs/2607.13104
The main page of the project:https://selfimproving-agent.github.io/
The code link:https://github.com/selfimproving-agent/awesome-Self-Improving-Agents
It is worth noting that the review quickly gained attention in the overseas AI community after its release.200+StarsThere are also a number of paper recommendations on X.800 likesIt also reflects a clear signal that self-improving agents are becoming a new focus for researchers and developers overseas.
Figure 1: Support for GitHubWarehouseIn a week or so, I got 200+ stars.
Figure 2: The paper recommendation post on X received about 800 likes.
Figure 3: Overview of the modern self-improvement agent.
What is self-improvement?
The paper defines the base model Agent as:
In this case,θ is the base model parameter;Σ is the script structure that runs around the model, including Prompt, Memory, Tools, and control logic such as routing, scheduling, and security constraints.
This definition delineates an important boundary: temporary planning, dialogue history and one-time reflection in the context above belong only to the running time state; true self-improvement only occurs when the system permanently modifiesθ orΣ based on trajectories, evaluations or verification results produced by its own execution, and allows the next task to start from a different starting point.
Thus, self-reflection and self-correction themselves are not necessarily equivalent to self-improvement. One task can "think again" to improve the present answer without leaving any cross-tasking capabilities behind. In contrast, a verified and written rule of long-term memory, a repaired and continuously reusable tool, even without updating the model weight, has changed the future behavior of the Agent.
Self-improvement is also not a concept that emerged suddenly in the age of big models. From feedback control, learning how to learn, self-referencing procedures, to Gödel Machine and self-learning, the related ideas have continued for decades. The change that the basic model has brought is that natural language has become a unified medium of modification: errors can be written as criticism, experiences can be compressed into memory, tools and workflows can be directly generated and rewritten. Self-modification is thus transformed from low-level code and weight-searching to a more expressible or verifiable engineering process.
Figure 4: The basic model provides cognitive abilities, prompt, memory, tools and control logic together determine how agents perceive, reason and act.
Two routesWriting into a model, or writing into a system
The first line is:Foundation Model ImprovementAgents themselves generate training examples, evaluate signals, or gain exploration experience from real-world environments and models, and then rewrite parameters through SFTs, preferential optimization or reinforcement learning. These updates are slower, more costly, and not easy to roll back, but once successful, the capabilities can be shared across tasks.
The paper divides its learning signals into three categories: first, self-generating examples and reasoning trajectories, second, scores, preferences or criticisms given by models, and third, real or simulated experiences from code execution, web interaction, games, and robotic environments. Self-Instruct, Constitutional AI, and WebRL reflect these ideas, respectively.
The risk is also direct: models may re-train their own mistakes into weighting, causing biases to widen, model collapse or catastrophic forgetting; they may also learn to drill for reward functions and world models. Parameter updates must therefore be accompanied by data filtering, external verification, version management and rollback.
The second line is:Scaffolding ImprovementThe model parameters remain unchanged, the system rewrites Prompt, organizes and eliminates Memory, selects, repairs or creates Tools, and even reconfigures planners, routers, and entire sets of Agent codes. Work such as TextGrad, Mem0, Voyager, Gödel Agent, Darwin Gödel Machine can be put into this route. It is lighter, more transparent, and easier to verify and withdraw, making it the most active direction in current engineering practice.
The paper isskillIt is particularly noteworthy that the skill is not a fifth component outside of the Prompt, Memory, and Tool, but rather a "sequencedable, reproducible update". The same skill can be stored as a tool and its call order, be it a workflow or memory, or it can be consolidated into a weight or control logic. In other words, the skill describes the "reserved capability unit" rather than where it exists.
This definition also distinguishes between two types of skills: object-level skills used to complete external tasks, such as repeatedly summoning a process of "collecting resources"; sub-level skills that act directly on the Agent itself, such as rewriting Prompt, organizing Memory, creating tools or triggering parameter updates. The latter, once able to improve "how to improve itself", starts touching on the most core of the RSI's index structure.
But it needs to cool down: today, most so-called RSIs, which are still "limited self-modification" restricted by target functions, test sets, sandboxes and limit boundaries, are far removed from unrestricted intelligent explosions. The focus of the paper is not on science fiction, but on how to make these updates measurable, attributable, and reversible.
The two routes are not mutually exclusive. A more rational system would allow the scaffolding to undertake rapid, localized, reversible exploration, and then repeatedly verify effective experience by distilling into the parameters, forming a double time scale for "quickly adapting to slowly hardening".
Figure 5: Basic model updates and scripting shelf updates are two main paths.
Six major application scenarios
The paper system addresses six categories of applications:Software engineering, web navigation and automation, game and strategy reasoning, scientific discovery, artificial intelligence and robots, general computer control.What they all have in common is that Agents can get feedback from the environment; the difference is whether feedback is reliable, how expensive it is to get it, and whether failure can be withdrawn.
Software engineering has compilers, unit testing, and continuous integration, which are the most intensive testing ground for feedback and the easiest to verify results. Web Agents face constantly changing pages, DOMs, and APIs that need to blend success and failure trajectories into memory, grounding strategies, or replicable tools. Games can provide replaceable environments, clear rules, and self-play experiences, but can also allow Agents to override fixed opponents or simulators.
Scientific discoveries extend the loop to assumptions, experiments, and evidence management, where the difficulty lies in innovation, replicability, and feedback latency. Physical intelligence needs to transcend simulation and reality, dealing with sim-to-real gaps and physical security. General computer control needs to form a portable procedural experience across applications and operating systems.
In these scenarios, a skill can be a code repair process, a web strategy, a game skill library, an experimental workflow, a robotic action program, or a desktop operating script.
Figure 6: Six representative applications of self-improvement agents, and commonly used environments and sources of feedback in various fields.
How to evaluate a system that is constantly changing
Traditional benchmarks only measure the final scores of static models, but the subjects of self-improvement agents are an updated trajectory. A credible experiment should at least answer: How much improvement per round under a fixed budget?
The paper distinguishes executable metrics from Judge-based evaluation: the former is suitable for code and tool calls, the latter covers open, long-chain tasks. If the same model is both improved and responsible for scoring, the Agent may just become more and more "arbitrary". Therefore, the generator and the evaluator should be separated and calibrated with independent Judge, hidden testing and verifiable subset.
Evaluation should also cover two levels: mechanism-level evaluation to isolate the contribution of Prompt, Memory, Tools or parameter updates; field-level evaluation to test its ability to withstand distribution changes, execution costs and security constraints.
Figure 7: Self-improvement agents often use benchmark distributions, covering model layers and script layers, static tasks and interactive tasks.
From rapid exploration to slow solidification
The paper's core design principles can be summarized in eight words:It's easy to find, and it's easy to find.
When feedback, task changes are fast, it is a priority to modify Prompt, Memory, Tools, and control logic because these changes are transparent, inexpensive, and roll-back. Only when a strategy has been fully proven to be able to migrate across tasks does it distill or minimize model weight to become a stable long-term capability.
The paper proposes six directions: continuous adaptation during testing, proactive exploration and curiosity, combined optimization of parameter distillation and model scaffolding, improved efficiency with limited resources, multi-agent collaborative evolution, and open world robustness in the face of environmental change.
The deeper the evolutionThe importance of safe borders
When an Agent is able to modify Memory、Tools and even its own code, the security objects themselves change. Once a Prompt Injection is no longer just a temporary interference, it may be written into memory or tool logic and become a permanent loophole. Therefore, Critic should be considered a governed infrastructure, and updates and approval updates should not be completely controlled by the same closed loop. Each modification is subject to functionality, permissions, and security checks.
Future advances in AI will not necessarily come from more parameters. More importantly, the key to the so-called "self-evolving agent" is whether the agent can draw experience from failure, encapsulate it as a replicable skill, manage its own memory and tools, modify its workflow, and re-evaluate its validated capabilities. When AI moves from "starting from scratch with each task to "leaving verifiable growth with each task", it really crosses the threshold from the tool to the learning system.
What is the meaning of this verse?
What this review really provides is not yet another grand prophecy about "AI self-evolution", but a set of engineering language to judge whether progress actually occurs: whether experience is preserved, whether capabilities can't be replicated, whether gains can be migrated across tasks, whether updates can be verified.
Agents in the future may become better at changing themselves, but more important than "evolving" is that they know what to change, why to change, and when not to change.