The paper Towards principled knowledge editing methods for large language model reasoning [1], recently published in Nature Machine Intelligence, cuts from knowledge editing to further exploring how large models use knowledge to reason and how to observe changes within the model through active intervention. What if the "experiments" on how to learn, organize and use knowledge within this research model no longer depend entirely on human completion but on the AI itself?
Just as doctors, psychologists, and brain scientists study human beings from different levels to understand our behavior, cognition, and brain mechanisms, we can also ask further: Can AI become the "scientist" who studies machine intelligence itself?
Paper:
Code:
Project Web Site:
Let AI find out for itself how intelligence is born.
Unlike traditional AI Scientist, which primarily helps scientists study drugs, materials, and biology, Mechanist focuses on a deeper question: how do models get this knowledge and how do they calculate it internally?
Mechanist organized the process of mechanical research into a complete loop of "hypothesis generation, experimental execution, result verification, iterative modification". However, understanding model mechanics is not easy: the intelligence of large models is generated by input, internal representations and complex interactions with mass parameters, often in a blink of an eye, and it is difficult to find a causal mechanism that really works solely by behavioural correlation. To support reliable mechanical findings, Mechanist constructed two basic resources:
One is the Scientific Knowledge Graph assistive hypothesis, which includes a mechanical knowledge graph composed of about 130,000 explanatory research papers and an interdisciplinary knowledge base covering 26 disciplines and 43 million papers; and the second is the basic analytical toolkit of Big Model Intelligent Mechanics, which covers 32 categories of mechanism analysis, causal intervention and verification methods.
The Mechanist does not simply let the big model summarize the text or write the experimental code, but lets the AI study the real "do" mechanisms. It needs to come up with falsifiable assumptions, design and execute experiments, and then verify and correct its own judgments based on the results. Only when the conclusions can be tested with different models, data and methods, is a more reliable mechanism found.
What did the Mechanist find?
The most interesting thing about Mechanist is that it didn't stop at the demonstration of the process of automated scientific research, but went on to discover some previously obscure internal laws of the model.
How does the model distinguish and use "facts I know" and "facts others believe"?
For example, a model knows that "the 2026 World Cup Final will be held in Jersey", but if it is told that "James thinks the final will be in Los Angeles" and then asked "where the final will be in reality" and "where James thinks it will be", the model needs to maintain two different knowledge states at the same time.
Mechanist divides these phenomena into three scenarios: World Knowledge, Personal Belief, and Attributed Belief, and observes that many models such as GPT, Gemini, Claude, Qwen, Pythia, OLMo and others have common errors in all three scenarios: sometimes the wrong beliefs of others interfere with the model's own factual judgment; sometimes the model's own knowledge in turn overshadows its judgment of others' beliefs.
In Pythia-1B, only about 0.05% of the parameters are associated with this type of belief-state reasoning, and can be further divided into different functional attention heads: L4.H1 (level 4's 1st head, later synonymous with this) mainly involved in Attributed Belief, whereas L9.H1, L7.H5, L12.H1 heads are closely related to Personal Belief. Causal intervention gives stronger evidence: After shutting down L4.H1, Attributed Belief accurately ranges from 0.86 to 0.34, while Belief remains unchanged. This indicates that there may not be a unified belief region within the model, and that different beliefs may be independent of Personal Belief's internal calculation mechanisms [2-4].
Mechanist also tracks how these mechanisms are formed. During the pre-training process of Pythia-1B, attributed belief capabilities emerge early, while personal belief is gradually formed in subsequent training; as training progresses, the causal importance of the corresponding heads also increases. This means that the advanced cognitive abilities of the model do not emerge suddenly after training, but may evolve gradually with specific internal mechanisms. Researchers can therefore not only track when the model "has what capabilities", but also further track when the abilities emerge, what mechanisms are supported by, and how these mechanisms evolve, thus advancing the study from the static model interpretation to the exploration of the process of intelligence formation.
After finding the mechanism, it can be controlled and improved
If the mechanism study could only tell us "there's a head here", the value would still be limited. The Mechanist went on to take a step: directly intervene in these belief heads, like directly stirring "the spinning inside of the model".[5-6] This mechanistic intervention was more effective than simply adding prompt hints to the model, resulting in +15.3% and +8.8% and +3.5% improvements in Pythia-410M, Pythia-1B and Pythia-2.8B, respectively, while maintaining a lower error disruption rate.
Biological discoveries based on mechanical design
The same idea has also been used for biosynchronous base models such as Evo2. If DNA sequences with higher α-spindle content are to be produced, traditional methods typically require a large number of candidates to be generated and then filtered through structural predictions. The Mechanist tries to find traits within the model that are related to the α-spindle and then performs direct steering. In 900 generating sequences, target characteristic intervention will increase the average predicted α-spindle content from 43.8% to 56.6%. This means that traits that were originally used for "interpretation" within the model can actually be further transformed into a spindle that controls the behavior of the model.
From knowledge editing to "experimental science" of machine intelligence
In retrospect, knowledge editors and Mechanist are forming a systematic research pathway: Understand AI, Improve AI. It's not simply dismantling the model, but understanding it and being able to control it again.
Knowledge editors are initially concerned with how the model knows how to change, and then gradually deepen into where the knowledge exists, how it connects, why the model believes in certain knowledge, and whether changing this knowledge and the internal mechanisms can further influence the reasoning and behavior of the model. Mechanist further automates this thinking, forming a closed loop of editing, observation, verification, and reintervention, allowing model mechanism research to move from passive analysis to active experimentation.
This also raises a bigger question: Will future AI be able to study its own mechanisms and continually improve itself based on the findings? The recurring self-improvement (RSI) discussed at this stage often emphasizes AI automatically writing code, generating data, optimizing training processes. But if AI is just constantly trying to make mistakes and cannot understand why it succeeds or fails, this self-improvement is in essence still closer to automated searches [7-8].
True regressive improvements may need to go further: finding problems, understanding mechanisms, designing experiments, validating assumptions, targeting interventions, and then observing changes after the interventions. In this sense, Mechanist explores not just a new AI Scientist system, but a paradigm of research aimed at machine intelligence: using large models as experimental objects, finding mechanisms as scientific problems, and using causal interventions as experimental means.
From a knowledge change in the model, to understanding why intelligence is created, how it is intercepted, to whether machine intelligence is continually improving itself based on its understanding of its mechanisms. This is perhaps the path to AGI through Mechanism Science of AI.
References
[1] Towards principled knowledge editing methods for large language model reasoning, Nature Machine Intelligence, 2026
[2] Language models cannot reliably distinguish belief from knowledge and fact，Nature Machine Intelligence, 2025
[3] Language Models Use Lookbacks to Track Beliefs, ICLR, 2026
[4] How Large Language Models Encode Theory-of-Mind: A Study on Sparse Parameter Patterns, npj Artificial Intelligence, 2025
[5] Toward universal steering and monitoring of AI models, Science, 2026
[6] Steering Llama 2 via Contrastive Activation Addition, ACL, 2023
[7] AlphaEvolve: A coding agent for scientific and algorithmic discovery, 2025
[8] Accelerating scientific discovery with Co-Scientist, Nature, 2026