I'm not worried about the lack of medical facts behind it.
In an online questionnaire, what might be behind the idea of eating your fatty liver without worrying about it?
The consultant was a formerly obese B-cell lymphoma patient, and the doctor misjudged fatty liver characteristics as high-density fatty liver based on the image description, thus excluding fatty liver. This judgment appears to be in line with the medical context, but critical medical facts are not consistent with the performance of real images.
In the past, security schemes targeted at online clinics have largely relied on keyword rules or simple text classification. These approaches have been effective against obviously erroneous screening, but confronting borderline cases that require a combination of a patient's overall context, clinical guidelines and evidence chains to judge, are often overlooked. A misspelled word may lead to the opposite conclusion.
Recently, Shenzhen AI Security Laboratory and the team at Shenzhen University have developed MedGuard, which integrates medical fact checks into clinical processes. It is not just another AI questionnaire system, but a layer of security infrastructure that can be embedded into the healthcare services process.
MedGuard: An LLM-Based Gatekeeper for Detecting Clinical Risks in Chinese Telemedicine Consultations
Cooperation institutions: Xi'an Group and Xi'an University
Nature Portfolio, influenced by factor 18.0)
The study will focus on clinical safety in medical safety: a multidisciplinary form of medical services, such as real physicians and AI-assisted clinical trials, that actively discovers factual errors, logical contradictions, critical missteps and potential drug risks during physician conversations, diagnostic conclusions, drug use regimens and treatment recommendations.
MedGuard has demonstrated excellence in both benchmarking and clinical professional evaluation: based on a 7B parameter, based on the research-constructed MedGuardEval medical risk assessment benchmark, which covers real-world online questionnaires, complex clinical cases and drug knowledge triangles. MedGuard achieves SOTA levels in fine-grained dialogue-level risk identification tasks, overall superior to the mainstream open-source model and representative medical agent systems such as HealthCareAgent, where F1 has a baseline average of 22.1%. Not only that, in 5-point clinical evaluations, which are based on research predictions, 3-point clinical assistive references are available; 126-point clinical certified professionals across 10 laboratories have improved the average rating of MedGuard in seven safety-related assessments, with an average of more than 4.2 percent lower overall legal and ethical errors.
The difficulty is not to generate answers, but to judge the reliability of the answers.
Internet clinics have made it possible for patients to see their doctor at any time and anywhere, but they have also fundamentally changed the way clinical risks are presented.
An online consultation can last for hours or even days. Disease history, symptoms, test results, prescriptions and treatment recommendations are scattered across dozens of rounds of conversations. Whether it's helping physicians conduct clinical trials or supporting AI in participating in medical services, AI really doesn't play a role in generating a seemingly professional feedback loop, but in understanding the clinical context fully and accurately identifying which medical facts need further verification.
The study categorizes medical fact-checking in online clinics as a three-pronged challenge.
First, critical information is overwhelmed by long conversations. Clinical risks are often hidden in long links, repeated conversations, critical negative words, dose information, past history and patient specifics are easily overwhelmed by context information, leading to missing important risk signals.
Second, the boundaries of risk are difficult to define. Medical safety is not a simple matter. Some clinical recommendations need to be combined with patient-specific situations, guidelines, and a comprehensive assessment of clinical experience. Excessive sensitivity groups misjudge reasonable clinical behavior as a risk, generating a large number of ineffective warnings, and increasing the burden on doctors; too conservative groups may miss out on issues that really need attention.
Third, the results of medical judgments need to be traceable. Medical risk identification. It is not difficult to explain why there is a problem. Lack of traceable evidence in the chain of risk suggestions makes it difficult for doctors to quickly review and to form effective interventions.
MedGuard makes medical judgments available
Verifiable fact-checking framework
Breaking down complex clinical processes into verifiable medical facts
MedGuard's idea is to break down complex physician-patient interactions into a set of independently verifiable atomic medical claims. The so-called atomic medical statement means that each statement expresses only a clinical proposition that can be independently judged to be false or risky, such as whether the image shows a lack of fatty liver disease or whether the current dose of sodium is appropriate for that age group of patients.
At the same time, the system maintains the patient's overall context, extracting critical information such as age, gender, past history, allergy history and current medication use, re-culturally re-writing statements in distributed multi-cycle interactions, supplementing and omitting information, and maintaining critical clinical constraints such as negative relationships, dosages, and time.
This process involves the extraction of critical medical information, the context-building of patients, the declaration of culture, the quality review and the weighting of four links. It solves a key problem in medical fact-checking: if the medical facts to be verified are incomplete in themselves, the ability to retrieve and reason is strengthened, and the ability to verify the wrong objects is also possible. By constructing patient-specific atomic medical declarations, MedGuard establishes a stable input for subsequent evidence-checking.
Figure 1 shows MedGuard's process of verifying the facts of real-life clinical cases. The system extracts critical medical statements from clinical interactions and combines medical evidence to identify potential risks.
Closed-loop verification capability based on uncertainty-driven evidence verification
MedGuard does not directly classify every medical statement as a simple risk/no-risk assessment, but rather as a discretionary assessment, producing three states of No Risk, Low Risk, and High Risk. Low Risk does not represent a low risk, but indicates that current judgments based on insufficient or semantic bias need further verification.
High-confidence No Risk and High Risk statements can be used to complete preliminary judgments along a rapid path; only entering uncertain areas can trigger the process of external medical evidence verification. This mechanism of uncertainty-driven verification enables the system to concentrate limited retrieval and reasoning resources into the most reviewed border cases while controlling ineffective alerts while reducing leakage.
Figure 2 shows MedGuard's closed-loop evidence verification process: the system revolves around verifying medical statements, generating retrieval strategies through planners, using medical knowledge resources to obtain evidence, and making final risk judgments by reasoning modules that combine statements with evidence.
Upon entering the verification pathway, MedGuard's planners will translate medical statements into structured retrieval tasks, clarifying core medical entities, adaptive evidence/taboo constraints, and corresponding areas of disease or drug knowledge. The retriever obtains relevant evidence from a knowledge base constructed by six authoritative Chinese medical resources, which contains 165,535 disease records and 187738 drug records.
Figure 3 shows MedGuard's medical knowledge base and evidence reasoning process: the system draws relevant evidence from disease and drug knowledge bases, respectively, and forms the Evidence Chain that supports ultimate risk judgments.
The summary further extracts evidence fragments directly related to the statement, key facts such as preserve doses, taboos, interactions, and quantitative indicators. Unlike the one-time retrieval, which is generated after a single retrieval, MedGuard will explicitly judge whether the current evidence is sufficient: if the evidence is missing, the relevance is insufficient or the conclusion cannot be supported, the planner will analyze the reasons for the failure, adjust the retrieval strategy and trigger the retrieval and summary again until sufficient evidence is available to support the conclusion.
Ultimately, the reasoner combines medical statements with refined evidence, outputting Risk/No Risk conclusions, explanations and chain of evidence, forming a statement understanding → uncertainty pathways → evidence planning → retrieval summaries → adequacy judgments → the complete closed loop of reasoning output.
It's not just about identifying risk, it's about determining the limits of risk more accurately.
In fine grain and rough grain dialogue evaluations, MedGuard demonstrated stable risk recognition capabilities. Based on the 7B parametric construct, fine grain dialogue risk recognition tasks reached the SOTA level, overall superior to mainstream open source models and representative medical agent systems such as HealthCareAgent, where F1 improved by an average of 22.1% compared to baseline, and rough grain evaluation improved by 5.8%; whereas, as a fact-checking input statement extraction capability, it improved by an average of 23.2% compared to other large models.
Figure 4 shows MedGuard's performance in fine-grain, coarse-grain risk identification and medical declaration extraction tasks, confirming its ability to identify critical risks in complex medical contexts.
But beyond the numbers, what's more worthy of attention is MedGuard's ability to judge the limits of risk. The key to medical safety is not just detecting risks, but also distinguishing the issues that really need to be addressed from reasonable clinical behavior.
Returning to a previous online questionnaire in patients with large B-cell lymphoma, doctors described fatty liver errors as high-density imaging and thus excluded fatty liver. MedGuard conducted fact-checking based on authoritative evidence, accurately identifying this medical fact-checking error, while maintaining reasonable clinical practices such as physician-reported MRI and thyroid-protein screening recommendations, reflecting its ability to judge the limits of clinical risk.
The study further invited 126 clinical professionals from 10 clinical departments to evaluate 604 clinical trials. The average score on key dimensions such as factual accuracy, reduction of errors, legal and ethical safety and overall availability was more than 4.2 points (5 points); the scores of doctors in different departments were more consistent, demonstrating better clinical accuracy and cross-disciplinary applicability.
Figure 5 shows the results of MedGuard's multi-dimensional clinical evaluations, including different evaluation dimension scores, classroom coverage, and consistency analysis of physician scores.
Another set of data is equally noteworthy: in a retrospective verification of 10,000 real-life questionnaire records, MedGuard controlled the risk detection fluctuations in different physiological systems at ±0.03; simultaneously generating 2234 risk warnings, compared to the model generating 4785, showing that it does not rely on increasing the number of warnings in return for safety, but effectively controls the burden of unnecessary alerts while maintaining risk detection capabilities. In other words, every warning it issues is worth more serious scrutiny by clinicians.
Embedding clinical safety capabilities into real medical processes
MedGuard is ultimately not an independent questionnaire system, but a layer of security infrastructure that can be embedded in healthcare services processes. It can receive complete questionnaire records, physician clinical conclusions or AI-assisted generated clinical recommendations, verify critical medical facts, and output structured results: pending verification statements, risk ratings, supporting evidence, interpretation and correction recommendations.
In practical applications, MedGuard can cover three key areas, embedding fact-checking capabilities into medical processes, and forming a continuous, scalable risk detection mechanism.
1. Pre-reponse or prescription verification: the completion of medical factual verification prior to a conclusion by a physician or AI;
2. Acknowledgement of the risks of the questionnaire procedure: In the course of the dialogue, real-time indication of the potential medical factual risks;
3. Historical review of clinical quality: verification of the quality of medical facts regarding the review of clinical questionnaires completed.
In the Medical Fact Check service system, MedGuard transforms medical safety into an executable process: identifying critical medical facts, completing validation based on authoritative evidence, and triggering reviews when necessary. The system is positioned as a capability to assist in clinical safety rather than as an alternative to medical decision-making. Its core value is that when clinicians face complex cases, multiple peer reviewers who rely on authoritative evidence assist in shutting down.
From a long-term dialogue understanding, dismantling medical statements, to evidence-driven verification of uncertainty, MedGuard explores a path of safety technology to the era of medical intelligence: the safety of medical intelligence, not only by avoiding the production of harmful content, but by discovering unreliable clinical facts, explaining the basis for risk, and intervening in a timely manner without interfering with normal medical services.
As AI moves toward the clinical stage, this kind of gatekeeper is perhaps more important than any kind of cool generation capability.