Real-World Effectiveness of AI-Enabled Clinical Decision Support in Primary Care: Systematic Review
Journal of Medical Internet Research ·
Background: AI-enabled clinical decision support systems (CDSS) are being used in primary care, but their effects on clinical decisions and patient outcomes remain uncertain. Objective: This study aimed to synthesize real-world evaluations of clinician-facing AI-enabled CDSS in primary care and assess evidence for diagnostic performance, changes in care, clinician efficiency, and patient-important outcomes. Methods: Following PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) 2020, 5 principal bibliographic databases (PubMed, Scopus, Web of Science Core Collection, Cochrane CENTRAL, and CINAHL Ultimate), a supplementary EBSCOhost platform search, and 2 trial registries were searched for publications from 2011 through 2025. Eligible studies evaluated a learned (data-derived) or case-based AI component used by clinicians in routine primary or ambulatory care and reported a clinical, behavioral, process, efficiency, or diagnostic accuracy outcome. Screening was performed independently by 2 reviewers. Risk of bias was assessed with design-appropriate tools, and GRADE (Grading of Recommendations Assessment, Development and Evaluation) was applied to 4 outcome bodies. Deterministic rule-based tools were retained only as contextual comparison. Results: A total of 66 reports were assessed at full text, and 24 studies met the review criteria; 10 were AI-enabled and formed the appraised review population, while 14 deterministic studies were contextual only. Detection findings were inconsistent: AI electrocardiography increased new low ejection fraction diagnoses (odds ratio [OR] 1.32, 95% CI 1.01-1.61), whereas an electronic health record machine learning dementia marker used alone did not (adjusted OR 0.84, 95% CI 0.63-1.11). Diagnostic studies showed high melanoma discrimination (area under the receiver operating characteristic curve [AUROC] 0.960) but only moderate glaucoma discrimination (AUROC 0.80; sensitivity 65%). Nonrandomized before-after studies reported increased urinary tract infection treatment success (from 75% to 80%) and changed diabetic retinopathy screening criteria in 3 of 4 general practitioners. A pre-exposure prophylaxis trial was null overall; a multicomponent fall prevention CDSS improved shared decision-making, with inconclusive medication-change effects. Clinicians reported lower estimated record review time in asthma care. Neither trial assessing patient-important outcomes demonstrated benefit from AI-CDSS; one musculoskeletal functional outcome favored usual care. Conclusions: Evidence published through December 2025 showed mixed effects of AI-enabled CDSS in primary care. Some studies reported improvements in detection or care processes, but patient-important benefit was not demonstrated. Heterogeneity and limited outcome measurement prevented causal explanations for these differences. Future trials should assess clinical decisions and patient outcomes together. Trial Registration: PROSPERO CRD420261302104; https://www.crd.york.ac.uk/PROSPERO/view/CRD420261302104
Background: AI-enabled clinical decision support systems (CDSS) are being used in primary care, but their effects on clinical decisions and patient outcomes remain uncertain. Objective: This study aimed to synthesize real-world evaluations of clinician-facing AI-enabled CDSS in primary care and assess evidence for diagnostic performance, changes in care, clinician efficiency, and patient-important outcomes. Methods: Following PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) 2020, 5 principal bibliographic databases (PubMed, Scopus, Web of Science Core Collection, Cochrane CENTRAL, and CINAHL Ultimate), a supplementary EBSCOhost platform search, and 2 trial registries were searched for publications from 2011 through 2025. Eligible studies evaluated a learned (data-derived) or case-based AI component used by clinicians in routine primary or ambulatory care and reported a clinical, behavioral, process, efficiency, or diagnostic accuracy outcome. Screening was performed independently by 2 reviewers. Risk of bias was assessed with design-appropriate tools, and GRADE (Grading of Recommendations Assessment, Development and Evaluation) was applied to 4 outcome bodies. Deterministic rule-based tools were retained only as contextual comparison. Results: A total of 66 reports were assessed at full text, and 24 studies met the review criteria; 10 were AI-enabled and formed the appraised review population, while 14 deterministic studies were contextual only. Detection findings were inconsistent: AI electrocardiography increased new low ejection fraction diagnoses (odds ratio [OR] 1.32, 95% CI 1.01-1.61), whereas an electronic health record machine learning dementia marker used alone did not (adjusted OR 0.84, 95% CI 0.63-1.11). Diagnostic studies showed high melanoma discrimination (area under the receiver operating characteristic curve [AUROC] 0.960) but only moderate glaucoma discrimination (AUROC 0.80; sensitivity 65%). Nonrandomized before-after studies reported increased urinary tract infection treatment success (from 75% to 80%) and changed diabetic retinopathy screening criteria in 3 of 4 general practitioners. A pre-exposure prophylaxis trial was null overall; a multicomponent fall prevention CDSS improved shared decision-making, with inconclusive medication-change effects. Clinicians reported lower estimated record review time in asthma care. Neither trial assessing patient-important outcomes demonstrated benefit from AI-CDSS; one musculoskeletal functional outcome favored usual care. Conclusions: Evidence published through December 2025 showed mixed effects of AI-enabled CDSS in primary care. Some studies reported improvements in detection or care processes, but patient-important benefit was not demonstrated. Heterogeneity and limited outcome measurement prevented causal explanations for these differences. Future trials should assess clinical decisions and patient outcomes together. Trial Registration: PROSPERO CRD420261302104; https://www.crd.york.ac.uk/PROSPERO/view/CRD420261302104