Improving Postpolypectomy Surveillance Decisions Using an Automated Analysis of Colonoscopy and Histopathology Reports: 2-Center Validation Study
Journal of Medical Internet Research ·
Background: Postpolypectomy surveillance intervals derived from colonoscopy findings and histopathology should be guideline-based, yet these are frequently not adhered to, contributing to under- and oversurveillance. This motivates a more structured and automated approach to improve adherence. Objective: We developed and evaluated an automated, implementable, guideline-aligned decision support tool that extracts colorectal polyp features from free-text colonoscopy and histopathology reports and produces auditable surveillance recommendations. Methods: We extracted paired colonoscopy and histopathology reports from a National Health Service (NHS) trust and designed a 4-stage prompting pipeline using open-weight large language models to separate multispecimen histopathology reports into individual containers (“pots”), align endoscopic findings with specimens, and normalize each pot into a structured schema. The surveillance intervals were then computed by a deterministic rule engine encoding guideline rules. We evaluated extraction performance on 100 procedures (215 pots), assessed surveillance recommendation performance on a same-site cohort (n=100), and performed external validation on a cohort from a second trust (n=100). A separate retrospective audit compared automated recommendations and observed routine scheduling against a clinician-derived reference based on full electronic health record review in 127 patients. Results: Key extraction fields showed end-to-end accuracy (specimen type=0.995; location=0.981; diagnosis=0.981; excision completeness=0.949), with mean report-level pot comprehensiveness of 0.968 (SD 0.112). Overall surveillance recommendation accuracy was 0.96 (95% CI 0.90-0.98) at Guy’s and St Thomas’ NHS Foundation Trust (GSTT) and 0.93 (95% CI 0.86-0.97) at Sandwell and West Birmingham NHS Trust (SWB) and showed strong class-level performance (no follow-up -score of 0.98 at GSTT and 0.97 at SWB; site check -score of 0.93 at GSTT and 0.91 at SWB; 3-year follow-up -score of 0.86 at GSTT and 0.80 at SWB; cancer alert -score of 1.00 at GSTT and 1.00 at SWB). The comparison to current clinical practice showed that the tool would improve guideline adherence primarily by reducing potential undersurveillance. Conclusions: This hybrid approach using open-weight large language models for structured extraction and guideline-encoded rules for decision-making can generate accurate, lightweight, and auditable postpolypectomy surveillance recommendations. Such a tool has the potential to improve safety netting and guideline adherence in NHS colonoscopy workflows.