Background and Aims
Determining surveillance intervals for patients with colorectal polyps is critical
but time-consuming and challenging to do reliably. We present the development and
assessment of a pipeline that leverages natural language processing techniques to
automatically extract and analyze relevant polyp findings from free-text colonoscopy
and pathology reports. Using this information, we categorized individual patients
into 6 postcolonoscopy surveillance intervals defined by the U.S. Multi-Society Task
Force on Colorectal Cancer.
Methods
Using a set of 546 randomly selected colonoscopy and pathology reports from 324 patients
in a single health system, we used a combination of statistical classifiers and rule-based
methods to extract polyp properties from each report type, associate properties with
unique polyps, and classify a patient into 1 of 6 risk categories by integrating information
from both report types. We then assessed the pipeline’s performance by determining
the positive predictive value (PPV), sensitivity, and F-score of the algorithm, compared
with the determination of surveillance intervals by a gastroenterologist.
Results
The pipeline was developed using 346 reports (224 colonoscopy and 122 pathology) from
224 patients and evaluated on an independent test set of 200 reports (100 colonoscopy
and 100 pathology) from 100 patients. We achieved an average PPV, sensitivity, and
F-score of .92, .95, and .93, respectively, across targeted entities for colonoscopy.
Pathology extraction achieved a PPV, sensitivity, and F-score of .95, .97, and .96.
The system achieved an overall accuracy of 92% in assigning the recommended interval
for surveillance colonoscopy.
Conclusions
This study demonstrates the feasibility of using machine learning to automatically
extract findings and classify patients to appropriate risk categories and corresponding
surveillance intervals. Incorporating this system can facilitate proactive and timely
follow-up after screening colonoscopy and enable real-time quality assessment of prevention
programs and providers.
Abbreviations:
CRC (colorectal cancer), EHR (electronic health record), NLP (natural language processing), PPV (positive predictive value)To read this article in full you will need to make a payment
Purchase one-time access:
Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online accessOne-time access price info
- For academic or personal research use, select 'Academic and Personal'
- For corporate R&D use, select 'Corporate R&D Professionals'
Subscribe:
Subscribe to Gastrointestinal EndoscopyAlready a print subscriber? Claim online access
Already an online subscriber? Sign in
Register: Create an account
Institutional Access: Sign in to ScienceDirect
References
- Colorectal cancer statistics, 2020.CA Cancer J Clin. 2020; 70: 145-164
- The impact of screening on colorectal cancer mortality and incidence: Has it really made a difference?.Dig Dis Sci. 2015; 60: 681-691
- Recommendations for follow-up after colonoscopy and polypectomy: a consensus update by the US Multi-Society Task Force on Colorectal Cancer.Am J Gastroenterol. 2020; 115: 415-434
- Colorectal cancer screening: estimated future colonoscopy need and current volume and capacity.Cancer. 2016; 122: 2479-2486
- Colonoscopic polypectomy and long-term prevention of colorectal-cancer deaths.N Engl J Med. 2012; 366: 687-696
- Long-term risk of colorectal cancer after removal of conventional adenomas and serrated polyps.Gastroenterology. 2020; 158: 852-861
- Utilization and yield of surveillance colonoscopy in the continued follow-up study of the polyp prevention trial.Clin Gastroenterol Hepatol. 2009; 7 (quiz 497): 562-567
- Utilization of surveillance colonoscopy in community practice.Gastroenterology. 2010; 138: 73-81
- Patient-, provider-, and system-level factors in low adherence to surveillance colonoscopy guidelines: implications for future interventions.J Gastrointest Cancer. 2014; 45: 500-503
- Receipt of colonoscopy following diagnosis of advanced adenomas: an analysis within integrated healthcare delivery systems.Cancer Epidemiol Biomarkers Prev. 2019; 28: 91-98
- Predictors of poor adherence of US gastroenterologists with colonoscopy screening and surveillance guidelines.Dig Dis Sci. 2015; 60: 971-978
- Low rates of colonoscopic surveillance among patients with high-risk adenomas [abstract].Am J Gastro. 2019; 114: S161
- Gastroenterology visitation and reminders predict surveillance uptake for patients with adenomas with high-risk features.Sci Rep. 2021; 11: 8764
- Post-polypectomy guideline adherence: importance of belief in guidelines, not guideline knowledge or fear of missed cancer.Dig Dis Sci. 2015; 60: 2937-2945
Stenetorp P, Pyysalo S, Topić G, et al. BRAT: a web-based tool for NLP-assisted text annotation, In Proceedings of the Demonstrations at the 13th Conference of the European Chapter of the Association for Computational Linguistics, 2012.
- spaCy: industrial-strength natural language processing in Python.Zenodo, 2020
- Clinical decision support with natural language processing facilitates determination of colonoscopy surveillance intervals.Clin Gastroenterol Hepatol. 2014; 12: 1130-1136
- Development of an automated algorithm to generate guideline-based recommendations for follow-up colonoscopy.Clin Gastroenterol Hepatol. 2020; 18: 2038-2045
- Emerging role of artificial intelligence in GI endoscopy.Gastrointest Endosc. 2020; 92: 1151-1152
- Natural language processing accurately calculates adenoma and sessile serrated polyp detection rates.Dig Dis Sci. 2018; 63: 1794-1800
- Natural language processing accurately categorizes findings from colonoscopy and pathology reports.Clin Gastroenterol Hepatol. 2013; 11: 689-694
- Natural language processing as an alternative to manual reporting of colonoscopy quality metrics.Gastrointest Endosc. 2015; 82: 512-519
- Multi-center colonoscopy quality measurement utilizing natural language processing.Am J Gastroenterol. 2015; 110: 543-552
- Anatomic and advanced adenoma detection rates as quality metrics determined via natural language processing.Am J Gastroenterol. 2014; 109: 1844-1849
- Applying a natural language processing tool to electronic health records to assess performance on colonoscopy quality measures.Gastrointest Endosc. 2012; 75: 1233-1239
- Public reporting of colonoscopy quality is associated with an increase in endoscopist adenoma detection rate.Gastrointest Endosc. 2015; 82: 676-682
- Development of the Parkland-UT Southwestern Colonoscopy Reporting System (CoRS) for evidence-based colon cancer surveillance recommendations.J Am Med Inform Assoc. 2016; 23: 402-406
Article info
Publication history
Published online: May 31, 2021
Accepted:
May 24,
2021
Received:
March 4,
2021
Footnotes
DISCLOSURE: The following authors disclosed financial relationships: J. Elmore: Editor in Chief, UpToDate. All other authors disclosed no financial relationships. Research support for this study was provided by the Melvin and Bren Simon Gastroenterology Quality Improvement Program (F.P.M., C.S., A.M.), UCLA Health Innovation Grant (E.P., F.P.M., O.K., G.G., C.M.), and National Science Foundation (W.H.) (grant 1722516).
If you would like to chat with an author of this article, you may contact Dr Hsu at [email protected]
Identification
Copyright
© 2021 by the American Society for Gastrointestinal Endoscopy
