Where did the experience go after Agent completed the mission?
An agent receives a project sorting assignment that finds a file in the work directory that pretty much understands what the user wants, but fails to read an email containing the latest message modification. It then proceeds to plan along the obsolete information, generates an invalid sorting, and finally writes the file to the wrong location.
The execution chain breaks down in the middle, and then it's all based on the wrong answers based on the wrong reasoning. After seeing it, you'll obviously let AI "re-examine the problem" again.
In most systems today, Agent's experience is kept in a log, with the execution experience generated by the model every day in the real world almost never entering the model parameters. This means that the next time the model encounters a similar task, it may still step over the same crater.
NeoHorse-1, recently released by Keynote, tried to change this. They opened up two-dimensional models of 4B and 9B, and the trained idea sounded a bit twisted: to have a routing system that was responsible for the model's "activation", by the way, record all of the models' paths, and then teach them to a small model, so that it can walk through a group of models on its own.
Link to the paper:https://arxiv.org/abs/2609.08183
HuggingFace：
GitHub：
This is what they call the first closed-loop verification of the RSI.
The model provides underlying infrastructure support with Infra-optimized technology capabilities, with teams from Tsinghua University and Beijing University participating in algorithmic and training methodology research to jointly explore improving the data utilization efficiency and training effects of Agent post-training.
The ultimate goal of the big model: self-replication
Recursive Self-Improvement (RSI) is a controversial and hotly debated term in the field of AI in 2026. Its vision is not complicated: to allow AI to recycle its products as an experience in the process of R&D, to participate in the design of the next generation of stronger ones, to cycle back and forth, to self-upgrade.
In June, Anthropic released a blog post titled When AI Builds Itself, which said it observed early signs of RSI internally. This week, OpenAI posted its first blog post about its RSI progress. OpenAI said it had achieved its goal set last fall: to create an "automation research internship" by September of this year.
But Anthropic, OpenAI itself acknowledges that the full RSI era has not yet arrived, and will not necessarily happen. A large body of work that has emerged in academia this year, from Self-Harness, Continual Harness, to various benchmarks for self-evolution, has largely remained at the stage of "improving a part".
The focus of the controversy is specific: the RSI targets in the mouths of AI giants are vast, but in practice they are a black box that the outside world can't check. You can't see how the training data flows, you can't reproduce that closed loop, and you can only choose to believe or not believe.
In AI Agent, which is already widely deployed, engineering-level self-transformation is also ongoing. For Harness, the performance determinants are not limited to the underlying model. Just as DSH is for DeepSeek V4 Pro, a properly designed Harness allows the model to reach its full potential.
Currently, there are two parallel paths of evolution in the field of Agent. One is to update external systems when reasoning, such as writing successful experiences into Memory, Skill or Harness; the other is to take the Harness framework directly into model training.
NeoHorse-1 attempts to achieve a positive loop of training in the simplest way, showing a clearly feasible process, with its models being Qwen3.5 4B and 9B, and dismantles a self-improving closed loop into four parts that can be checked per loop and are all open source.
To understand it, you have to start with a routing system called OpenSquilla.
Harness Framework, which is naturally a training ground for AI
A few weeks ago, OpenSquilla's open-source Harnes framework opened the door to innovation at the company's infrastructure level. OpenSquilla focuses on running and scheduling complex intelligent tasks through high-efficiency and low Token costs.
A complex task often involves dozens of rounds, each of which calls for the most expensive flagship model, and tokens can't stay on top. OpenSquilla's practice is to install a "router" on the Harness framework, which determines how strong a step needs to be before each step is performed, and then allocates it to the most suitable and cheapest model or model combination. Depending on benchmark test results, this mechanism can save 60% to 80% of token costs.
The further insight of NeoHorse is that the router leaves three things behind for every step it takes: it predicts what capabilities the step requires, records the actual deployment model, and observes how the step is ultimately executed.
In other words, the record of people chatting with AI mainly contains "questions, answers, answers", while Routing Harness results in "demand forecasting, routing decisions, routing execution processes, routing environmental outcomes".
This is exactly a complete teaching archive, and NeoHorse decided to do it. The idea behind the Harness-based AI training model is roughly four steps.
The first step is to keep Agent's "thinking, thinking, thinking, thinking" chain. The real ability of Agent lies in the intersection of reasoning, tool calling, environmental reproduction, reactivation and the final delivery of products. NeoHorse takes every complete interaction into a single track, and then cuts the training units down to the user's turn. The current round of reasoning and thinking is the goal of the model to be learned, with an earlier history as an up-to-date context.
The second step is to determine whether the trajectory cannot be used. The system uses a precision rule check tool to determine whether there is a corresponding return, whether the event sequence is established, whether there is no final response, and whether the execution branch can be closed. The complete trajectory of the structure enters the next step; the partially recoverable record only retains the complete fractions of the causal relationship; the trajectory to which the event belongs is isolated.
NeoHorse then performs semantic evaluations in six dimensions from goal achievement, instruction follow, tool use, evidence consistency, error recovery and correct termination. NeoHorse records several states of passing, warning, failure or failure to evaluate, respectively, so that the system sees not only the execution of the task, but also that failure is more likely to occur in the acquisition of evidence, tool selection or termination judgment.
NeoHorse uses the C0 to C3 layers of Routing Harness as difficulty signals: the training progresses in three phases, focuses on low-demand file tasks, then gradually blends into high-demand file hard tasks, while deliberately leaving a portion of the simple sample to the final stage, preventing the model from becoming a problem at a later date. The route record reflects real user coverage of the model, price and availability fluctuations, closer to deployment than abstract difficulty labels.
There is one detail that is easily overlooked, and the system does not directly label "which model is ultimately called" as difficulty. Because the actual route may also be affected by task specification, service availability, and system strategy. NeoHorse uses the need for routers to re-evaluate the response before it occurs, based on request and historical status. It determines when the sample appears, rather than simply telling the model, "It's a difficulty".
In the fourth step, NeoHorse also uses guided On-Policy Distillation. Traditional distillation is more like letting students imitate the standard answers that teachers have already written; whereas in On-Policy Distillation, the student master responds and the teacher provides the next step of supervision along where the students actually go.
Finally, the model evaluation results will form a "capacity shortboard map" that reflects which scenarios have low completion rates, which errors are most frequent, and which capability levels are weakest. The next round of training data is tilted to these regions while maintaining basic capability coverage.
How much capacity has been upgraded?
NeoHorse-1 was evaluated in both 4B and 9B dimensions, covering 10 Benchmarks such as Harness Agent, Tool Calling, Multipurpose Interaction, Code Generation and Instruction Following.
Compared to itself, the average score of 4B models increased from 58.94 to 64.87, an increase of 5.93 points; 9B increased from 65.60 to 69.04, an increase of 3.44 points. The overall result for NeoHorse-1-4B has approached 65.60 points for the Qwen3.5-9B base. The report also shows that, under the same course configuration, Harness trajectory-trained checkpoints have an average of 6.26 points higher than the publicly synthesized tool data Toucan on five comparable tests.
These results need to be looked at separately. NeoHorse-1-4B achieves SOTA upgrades of the same size, focusing mainly on Harness Agent, tool interaction and code execution of such processes with clarity, feedback verifiable tasks; but this scale is still tough for long-term maintenance, repeat test repair, and strategy change based on environmental feedback. Parameter scale does not lose meaning.
A data analysis task rightly marks this boundary. When there were no pandas in the environment, NeoHorse-1-4B did not recognize the prototype in time that it had been blocked by the environment, but repeatedly tried to install dependencies, manually analyze and repair the script, ultimately failing to generate a report. Such a task needed to be delivered to a model with a higher capacity in the model pool.
In addition to the numbers, the agent is more intuitive in the change in the behavior level. In the ticketing diary task, the base model deals with the state of the work item and repeatedly rewrites it after the contradiction, as more than twenty records change, the NeoHorse will initially supplement the evidence, identify the updated constraints, verify the consistency after the delivery, and finally the 19 work item matching rate reaches 100%.
So this isn't the RSI?
Is this the real RSI?
There are at least three levels of AI self-improvement: the model detects errors and retests in the current task, which is self-correction; the system writes the experience into Memory、Skill or Harness, which is run-level adaptation; and when the experience further changes the model parameters, the updated model produces better new experiences and continues to accumulate in a multigenerational cycle, which is closer to complete recursive self-improvement.
NeoHorse already has several key features: Harness generates execution data, assesses discovery capability gaps, matches data with adjustments, trains updates models, and updates models that can return to multiple model pools. The team generalizes this as a link between Data-RSI and Model-RSI.
However, the current technical report only validates one round of "evaluating, selecting, choosing, and updating". It does not prove that second- and third-generation models can continue to gain gains, nor does it allow the "how to improve the model" mechanism itself to evolve automatically. As the model gets stronger, the original data may lose value; repetitive learning from its own trajectory may also increase bias or cause capacity degradation.
Thus, as the basic law of motion says, NeoHorse completed a repeatable single-cycle closed-loop verification on the RSI.
Although it is not proof that the RSI has arrived, RSI can now be seen as a reproducible engineering problem. Today, this new direction of optimization is particularly noteworthy in the fundamental model of algorithmic regression towards parameters.
The model and the Harness propeller have started to rotate
OpenSquilla is the place where the model executes and experiences, while NeoHorse is the carrier after the experience enters the model parameters.
The ideal state of the aircraft is that OpenSquilla completes the task with multiple models and leaves a real runway; NeoHorse learns from it and gradually undertakes more work that would have required expensive model processing; Agent gains more use after the cost decreases; more execution and new data is generated, driving the next round of training.
From an industry perspective, the Harness framework that previously carried models has long been seen as lacking in technical content, and NeoHorse concludes that: whoever builds a system that runs continuously, leaves a structured record of every step, gets the next generation of Agent models with the cheapest and most difficult to replicate training data.
The model growth path proposed by NeoHorse-1 takes the importance of Harness to a new level. It is now increasingly central to connecting user tasks, environmental feedback, and model updates.
This is only a preliminary experiment, and the future Agent system may allow the model to continue to grow from each execution.