Claude, are you in last place?
PhAI Labs recently published an AI research capability check, combined with subsequently published scientific literature, that examines the hypotheses of different models that could provide inspiration for future research ideas and directions of exploration. The Claude Opus 5 group corresponds to 2.3%, the last of the five groups in the chart.
The Discovery RSI was 6.3%, about 2.5 times that of the GPT group, 2.7 times that of the Claude group.
Figure 1 The five model groups inspired future research: The Discovery RSI is 6.3% overall.
This comparison is observed: the research hypothesis proposed by AI is whether a specific research basis can be found in subsequent published papers. For example, an experimental control that can be adopted, a measurement method, or a gap in evidence that is worth further testing.
This corresponds to a more general research capacity: to propose a valid, testable research hypothesis and to develop it into research that can be carried out in practice.
A study needs to start from assumptions worth exploring and gradually develop viable investigative and verification schemes. Where the research questions come from, why they are worth doing, and what evidence can change judgments needs to be clarified in the exploration.
In conjunction with PhAI Labs, several universities and research institutions have developed the Discovery RSI. The paper, co-authored by 23 authors in the fields of AI and science, includes the signature of 16 institutions at home and abroad, bringing together the research power of institutions such as Stanford University, MIT, Harvard Medical School, Beijing University, Oxford University, and Princeton University.
The co-authors include James Zou, associate professor at Stanford University, and Leo Anthony Celi, associate professor of clinical research at MIT's Computational Physiology Laboratory; Leo Celi, associate professor of medicine at Harvard Medical School; and PI, whose team built and maintained the MIMIC database, providing an important foundation for clinical data research.
This work will link the formation of research problems to the ongoing learning of the Agent: the model needs to come up with a well-discussed research program, be constantly revised under the constraint of data and evidence, and receive professional human evaluation.
The paper:
The code is:https://github.com/Gen-Verse/DiscoveryRSI
The model:https://huggingface.co/collections/Gen-Verse/discoveryrsi
Meanwhile, PhAI Labs is conducting a three-day live broadcast of Discovery RSI technology, which is welcome for viewing in the live room. The live broadcast presents more intuitively how the research agent detects problems, develops hypotheses, and forms testable research programs. More importantly, whether the hypotheses proposed by the research agent will inspire future research.
It is also an opportunity for researchers and technological developers who want to understand how AI can be truly involved in scientific discoveries to move from a paper approach to a specific scientific process.
It's the first time I've seen it.
Preview of live content
Discovery RSI: From proposing hypotheses to learning from long-term research
A good assumption is not enough to keep the research agent growing.
In one study, Agent may have added critical controls, or it may have found that the original program could not be used to test existing data. Assuming that the research experience continues to change and accumulates with it. However, how to turn these experiences into the ability to use the model the next time is still challenging.
For this purpose, DiscoveryRSI has proposed a set of regressive self-improvement paradigms for scientific discoveries that allow the development process to be the source of model learning.
The system first organizes the study hypothesis with the test program. Each hypothesis requires a clear study of the inferences, test methods, criteria and alternative interpretations, explaining why the problem is worth studying, and what results can support or undermine the original judgment.
Different versions are preserved in the research hypothesis tree. Multi-intelligent entities continue to evolve around existing directions, fix defects, combine different ideas or restart when necessary. The literature, data and research decisions on which they are based are also recorded at the same time each time they are modified.
Figure 2 Discovery RSI: Studying tree assumptions, contribution attribution and recurrent improvements to models and workflows.
With these records, the system can trace further where research progress has come from. Hypothesis Credit Optimization (assuming contribution attribution and optimization) connects scientific evaluation to specific contributions supported by records and translates effective behavior into different weighted training goals.
One Agent found critical evidence, the other proposed necessary controls, subsequent revisions and corrections to the study design defects, which can be individually identified and used to guide model optimization. The team trained the Discovery RSI-27B based on this mechanism.
Improvements can also occur in the workflow. Harness Replay reviews problems such as repeated exploration, tool failure or loss of context when the model remains unchanged, and tests targeted adjustments. Rediscovery will update the evaluation for new research tasks and continue to accumulate experience. These mechanisms together constitute a framework for cross-sectional learning.
The accompanying DFM-Bench will extend the scope of evaluation from individual assumptions to assumptions revised, research pathways and entire rounds of research processes, focusing on whether research has accumulated evidence, made reasonable judgments, and made scientific contributions.
The experiments covered clinical and population health, treatment focus priorities, disease mechanisms, the four life sciences scenarios in population multicommunology, using data sources such as HCC, MIMIC-IV, eICU, UK Biobank and GEO, ArrayExpress.
Figure 3 Shared scientific discovery processes in the life sciences and cell fields: developing research hypotheses from evidence and problems in exploration, evaluation and revision.
In the field of batteries, there are four lines of research around electrological diagnosis, life expectancy prediction, material degradation and the use of historical data. The system has covered 300 research episodes, 351 research hypotheses, on the DFM-Bench: 58.47% of the research evolution has been determined to be substantive progress, the overall research pathway score reaches 81.52, the overall organization score reaches 71.25. More importantly, each candidate traces back to data, measurement conditions, experimental design, counter-examples, and follow-up decisions, making the research process involving Discovery RSI testable, replicable, sustainable generation.
From biomedical to cell research, multidisciplinary experiments provide initial support for the effectiveness of the new paradigm of DiscoveryRSI scientific discovery and demonstrate its potential for cross-disciplinary application.
Around this framework, technical reports experiment with four research questions: whether the models are more research intensive, how research behaviour changes, how professional researchers evaluate them, and whether the hypotheses generated can connect to useful research clues.
02 Continuous learning from research experience, improving the quality and verifiability of research assumptions
PhAI Labs compared the basic model to a complete one- or two-round self-improvement model in 241 remaining research questions using a standardized automated evaluation standard to examine the quality assumptions.
The average score rose from 4.86 to 5.18, then rose to 5.39. The second round outperformed the basic model in all four areas, increasing the magnitude from 0.28 points in the direction of the disease mechanism to 0.94 points in the direction of the population multispecies.
Figure 4 shows two rounds of self-improvement, with the assumption that automatic quality scores in four areas are higher than in the basic model.
The most prominent improvement was the assumption of whether it could enter actual testing. Falsifiability increased from 4.00 to 6.20, consideration of alternative interpretations increased from 4.44 to 5.98; scores on whether data can answer questions increased from 3.73 to 4.99, and feasibility increased from 4.79 to 5.87.
These results demonstrate that research experience can drive models to write a program more specifically, clarify how to conduct tests, and interpret different results. While the premise is mainly focused on these aspects, the automatic scores of importance and novelty have not yet risen simultaneously.
03 Optimizing research strategies in self-improvement to make exploration deeper and more grounded
In addition to scores, PhAI Labs also recorded Agent selection during the study.
The proportion of steps that have been hypothesized to evolve and repair increased from 77.5% to 91.7%; the proportion of restarted steps decreased from 21.7% to 5.0%.
Figure 5 After improvement, Agent develops and corrects assumptions more frequently, reducing restarts.
This change was particularly evident in the second half of the study: 32% of the basic model chose to restart the step, 3% after two rounds of improvement, and 93% for evolution or repair. In the automated evaluation of the research pathway, evidence and reasoning standardisation improved by 1.25 points, technology and knowledge exploration improved by 0.34 points, and continuity and comprehensiveness improved by 0.24 points.
Research output also progresses to execution. The paper reports that in matching operations, the unique assumptions chosen for the Discovery RSI-27B were increased by 16.7%, compared to the base model, and successfully compiled Harbor tasks were increased by 50%.
These tasks will organize the input, run environment, output and verification requirements into clear execution units so that the selected computational research program can move on to the next stage of investigation. After compilation, the corresponding scientific conclusions will still require actual execution and verification by evidence.
04 Experts evaluate the value of research and promote the maturity of assumptions in revision
The study further set up an expert-led human resource review: 27 physicians, medical students and PhD students with biomedical or clinical research backgrounds evaluated 292 proposals under the supervision of three senior advisors.
Among the reviewed study trees, Discovery RSI-27B had 41% revisions resulting in improvements, 47% consistent with only 12% variance. GPT-5.6 Sol's improvement to variance ratio was 31% and 32%; GPT-6 Luna was 25% and 31% respectively.
Figure 6 Artificial evaluation: 41% of assumed revisions led to improvement, 12% difference, with an average revisions rating change of +0.07.
The average change in scores per revision was +0.07 for Discovery RSI-27B, higher than +0.03 for GPT-5.6 Sol and −0.01 for GPT-6 Luna. Compared to GPT-6 Luna, the improvement ratio was 16 percentage points higher and the variance ratio was 19 percentage points lower.
On two artificial indicators of the importance of the study problem, the argument of rationality and the boundary of evidence, Discovery RSI-27B received 3.10 and 3.37, respectively, higher than GPT-6 Luna's 2.94 and 3.16 respectively. 49% assumed that the median value of the five-dimensional mass rating was reached or exceeded, compared to the corresponding model ratio of 33%.
The paper summary also reported that only 19 percent of the 292 reviewed proposals were rejected, with the remaining proposals including prioritizing, revising or delaying.
Specific cases also show the changes brought about by the revision. In a MIMIC-IV study pathway, the system gradually clarified the start time and the 48-hour timeframe, matched the variables to real data fields, and supplemented the threshold for the determination of acute aluminum damage. The mean of six artificial metrics increased from 2.83 to 3.50 and the amount of data adaptation and result information increased by 2 points.
Figure 7 of the MIMIC-IV study pathway case: The average rating of six human subjects increased from 2.83 to 3.50.
This group of results gives a more intuitive meaning to the study of experimental algae: once effectively revised, it can be implemented as a clearer observation time, more rational variable definition or more complete judgement criteria. Artificial evaluation also shows the advantages of different models; the above revision results measure the changes within the study tree, providing different levels of evidence, respectively, with cross-sectional model update experiments.
Do these assumptions inspire further research?
Returning to the initial ranking, the team further contrasted AI-generated assumptions with subsequent published papers, looking for specific links for further research.
Reference sets were shared by all model groups. The team completed 420 forward and backward cross-references, obtaining 17,790 citations along the citation line and 16,440 papers analyzed, and then selected sources according to the time limits recorded by each group.
Two production runs of DiscoveryRSI produced 1,176 hypothesized versions, 74 of which were assessed accordingly, with an overall ratio of 6.3% and the highest of the five model groups in the chart.
Figure 8 illustrates the results of comparisons between the overall and sub-field literature. The sample size of each group varies, using descriptive proportions.
These links take the research hypothesis to specific experimental design and testable problems and provide clues for subsequent artificial reviews. The current results come from automated literature controls, with different sample sizes for each group, and are descriptive comparisons; it cannot directly prove that the AI hypothesis has been adopted or verified by future research.
To test whether AI can inspire future research, it is necessary to continue to track the assumptions made, and whether researchers will conduct new explorations and validations around it. This is also the direction in which this technology live wants to further present and continue testing.
It's time to move on from DFM, RSI-Anything, to Discovery RSI.
DiscoveryRSI is a continuation of PhAI Labs' longstanding research on scientific intelligence and recurrent self-improvement.
DFM (Discovery Foundation Models) proposes a model of research aimed at Discovery Intelligence, focusing on how AI discovers valuable unknowns, forms research problems, and modifies judgments in external evidence to gradually accumulate discovery capabilities.
RSI-Anything, based on JEPA-Anything's predictive capabilities, explores how to use the world model to choose actions that are worth checking, and uses real feedback to improve strategies or workflows.
DiscoveryRSI takes this direction further into research problems and hypotheses themselves: hypotheses that change over the course of research, evidence and design decisions, and how to become a self-improving learning material for research agents.
The difficulty of the research task also includes what the next step should be to study. Whether one direction is worth pursuing, whether new evidence alters the original explanation, and exactly which design flaws failures have exposed, all affect the subsequent exploration. Discovery RSI preserves these changes by studying trees, connecting progress to specific behaviour by contributing attribution, and applying experience to model training and new rounds of research.
From the world model-assisted action to the traceable learning process around the research hypothesis, this work brings PhAI Labs' recurring self-improvement research into more open scientific discovery issues. The two rounds of quality improvement of model updates and professional human review provide new experimental support for this route.
Together, these efforts point to a long-term goal: to allow AI to acquire new evidence from real research, and to transform experimental experience into the ability to make subsequent discoveries. Longer time, more rounds of continuous improvement, will still be the next direction the team needs to verify.
About PhAI Labs and the research team
PhAI Labs is a new research organization aimed at the next generation of intelligent systems (Neo Lab), which has a long-standing focus on the pioneering fusion of AI and scientific discovery.
The team members' research background covers universities such as Stanford, Oxford, Princeton, and Beijing University, and they have cutting-edge experience in AI research and scientific experimentation. The Discovery RSI further combines the power of machine learning, biomedical and clinical research into a single study.
From DFM's definition of discovery intelligence, to RSI-Anything's exploration of recursive improvement mechanisms, to DiscoveryRSI's concrete implementation of research hypothesis development and experiential learning, PhAI Labs is continually advancing a research pathway: getting models involved in real research and making each round of exploration a source of next-generation capacity growth.
The team welcomes researchers to use code and models to further explore the ongoing study of scientific agents through open source research, scientific experiments and external collaboration to test these mechanisms.
The official website of PhAI Labs:
PhAI Labs Discovery RSI technology live broadcast is underway
Welcome to the Live Room
Learn how to ask questions from a research agent
The process of scientific discovery that leads to continuous learning
Direct entry ↓
© THE