Advertisement

Quality assurance of computer-aided detection and diagnosis in colonoscopy

Published:March 26, 2019DOI:https://doi.org/10.1016/j.gie.2019.03.019
      Recent breakthroughs in artificial intelligence (AI), specifically via its emerging sub-field “deep learning,” have direct implications for computer-aided detection and diagnosis (CADe and/or CADx) for colonoscopy. AI is expected to have at least 2 major roles in colonoscopy practice—polyp detection (CADe) and polyp characterization (CADx). CADe has the potential to decrease the polyp miss rate, contributing to improving adenoma detection, whereas CADx can improve the accuracy of colorectal polyp optical diagnosis, leading to reduction of unnecessary polypectomy of non-neoplastic lesions, potential implementation of a resect-and-discard paradigm, and proper application of advanced resection techniques. A growing number of medical-engineering researchers are developing both CADe and CADx systems, some of which allow real-time recognition of polyps or in vivo identification of adenomas, with over 90% accuracy. However, the quality of the developed AI systems as well as that of the study designs vary significantly, hence raising some concerns regarding the generalization of the proposed AI systems. Initial studies were conducted in an exploratory or retrospective fashion by using stored images and likely overestimating the results. These drawbacks potentially hinder smooth implementation of this novel technology into colonoscopy practice. The aim of this article is to review both contributions and limitations in recent machine-learning-based CADe and/or CADx colonoscopy studies and propose some principles that should underlie system development and clinical testing.

      Abbreviations:

      ADR (adenoma detection rate), AI (artificial intelligence), APC (adenoma per colonoscopy), CADe (computer-aided detection), CADx (computer-aided characterization), FDA (Food and Drug Administration), NPV (negative predictive value), PIVI (Preservation and Incorporation of Valuable Endoscopic Innovations), PMR (polyp miss rate), PPV (positive predictive value)
      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'

      Subscribe:

      Subscribe to Gastrointestinal Endoscopy
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect

      References

        • Hosny A.
        • Parmar C.
        • Quackenbush J.
        • et al.
        Artificial intelligence in radiology.
        Nat Rev Cancer. 2018; 18: 500-510
        • Jiang F.
        • Jiang Y.
        • Zhi H.
        • et al.
        Artificial intelligence in healthcare: past, present and future.
        Stroke Vasc Neurol. 2017; 2: 230-243
        • Bejnordi B.E.
        • Veta M.
        • Van Diest P.J.
        • et al.
        Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer.
        JAMA. 2017; 318: 2199-2210
        • Esteva A.
        • Kuprel B.
        • Novoa R.A.
        • et al.
        Dermatologist-level classification of skin cancer with deep neural networks.
        Nature. 2017; 542: 115-118
        • Gulshan V.
        • Peng L.
        • Coram M.
        • et al.
        Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs.
        JAMA. 2016; 316: 2402-2410
        • Liedlgruber M.
        • Uhl A.
        Computer-aided decision support systems for endoscopy in the gastrointestinal tract: a review.
        IEEE Rev Biomed Eng. 2011; 4: 73-88
        • Mori Y.
        • Kudo S.E.
        • Berzin T.M.
        • et al.
        Computer-aided diagnosis for colonoscopy.
        Endoscopy. 2017; 49: 813-819
        • Kaminski M.F.
        • Regula J.
        • Kraszewska E.
        • et al.
        Quality indicators for colonoscopy and the risk of interval cancer.
        N Engl J Med. 2010; 362: 1795-1803
        • Alagappan M.
        • Brown J.R.G.
        • Mori Y.
        • et al.
        Artificial intelligence in gastrointestinal endoscopy: the future is almost here.
        World J Gastrointest Endosc. 2018; 10: 239-249
        • Kaminski M.F.
        • Wieszczy P.
        • Rupinski M.
        • et al.
        Increased rate of adenoma detection associates with reduced risk of colorectal cancer and eeath.
        Gastroenterology. 2017; 153: 98-105
        • Hassan C.
        • Pickhardt P.J.
        • Rex D.K.
        A resect and discard strategy would improve cost-effectiveness of colorectal cancer screening.
        Clin Gastroenterol Hepatol. 2010; 8: 865-869
        • Rex D.K.
        • Kahi C.
        • O’Brien M.
        • et al.
        The American Society for Gastrointestinal Endoscopy PIVI (Preservation and Incorporation of Valuable Endoscopic Innovations) on real-time endoscopic assessment of the histology of diminutive colorectal polyps.
        Gastrointest Endosc. 2011; 73: 419-422
        • Mori Y.
        • Kudo S.
        • Misawa M.
        • et al.
        Real-time use of artificial intelligence in identification of diminutive polyps during colonoscopy.
        Ann Intern Med. 2018; 169: 357-366
        • Mori Y.
        • Kudo S.
        Detecting colorectal polyps via machine learning.
        Nat Biomed Eng. 2018; 2: 713-714
        • Tajbakhsh N.
        • Gurudu S.R.
        • Liang J.
        Automated polyp detection in colonoscopy videos using shape and context information.
        IEEE Trans Med Imaging. 2016; 35: 630-644
        • Tajbakhsh N.
        • Gurudu S.R.
        • Liang J.
        Automatic polyp detection using global geometric constraints and local intensity variation patterns.
        Med Image Comput Comput Assist Interv. 2014; 17: 179-187
        • Karkanis S.A.
        • Iakovidis D.K.
        • Maroulis D.E.
        • et al.
        Computer-aided tumor detection in endoscopic video using color wavelet features.
        IEEE Trans Inf Technol Biomed. 2003; 7: 141-152
        • Wang Y.
        • Tavanapong W.
        • Wong J.
        • et al.
        Polyp-alert: near real-time feedback during colonoscopy.
        Comput Methods Programs Biomed. 2015; 120: 164-179
        • Fernández-Esparrach G.
        • Bernal J.
        • López-Cerón M.
        • et al.
        Exploring the clinical potencial of an automatic colonic polyp detection method based on energy maps creation.
        Endoscopy. 2016; 48: 837-842
        • Tajbakhsh N.
        • Gurudu S.R.
        • Liang J.
        A comprehensive computer-aided polyp detection system for colonoscopy videos.
        Inf Process Med Imaging. 2015; 24: 327-338
        • Misawa M.
        • Kudo S.E.
        • Mori Y.
        • et al.
        Artificial intelligence-assisted polyp detection for colonoscopy: initial experience.
        Gastroenterology. 2018; 154: 2027-2029.e3
        • Urban G.
        • Tripathi P.
        • Alkayali T.
        • et al.
        Deep learning localizes and identifies polyps in real time with 96% accuracy in screening colonoscopy.
        Gastroenterology. 2018; 155: 1069-1078
        • Wang P.
        • Xiao X.
        • Brown J.R.G.
        • et al.
        Development and validation of a deep-learning algorithm for the detection of polyps during colonoscopy.
        Nat Biomed Eng. 2018; 2: 741-748
        • Klare P.
        • Sander C.
        • Prinzen M.
        • et al.
        Automated polyp detection in the colorectum: a prospective study (with videos).
        Gastrointest Endosc. 2019; 89: 576-582
        • Komeda Y.
        • Handa H.
        • Watanabe T.
        • et al.
        Computer-aided diagnosis based on convolutional neural network system for colorectal polyp classification: preliminary experience.
        Oncology. 2017; 93: 30-34
        • Sanchez-Montes C.
        • Sanchez F.J.
        • Bernal J.
        • et al.
        Computer-aided prediction of polyp histology on white-light colonoscopy using surface pattern analysis.
        Endoscopy. 2019; 51: 261-265
        • Tischendorf J.J.W.
        • Gross S.
        • Winograd R.
        • et al.
        Computer-aided classification of colorectal polyps based on vascular patterns: a pilot study.
        Endoscopy. 2010; 42: 203-207
        • Gross S.
        • Trautwein C.
        • Behrens A.
        • et al.
        Computer-based classification of small colorectal polyps by using narrow-band imaging with optical magnification.
        Gastrointest Endosc. 2011; 74: 1354-1359
        • Kominami Y.
        • Yoshida S.
        • Tanaka S.
        • et al.
        Computer-aided diagnosis of colorectal polyp histology by using a real-time image recognition system and narrow-band imaging magnifying colonoscopy [abstract].
        Gastrointest Endosc. 2016; 83: 243-249
        • Takemura Y.
        • Yoshida S.
        • Tanaka S.
        • et al.
        Computer-aided system for predicting the histology of colorectal tumors by using narrow-band imaging magnifying colonoscopy (with video).
        Gastrointest Endosc. 2012; 75: 179-185
        • Byrne M.F.
        • Chapados N.
        • Soudan F.
        • et al.
        Real-time differentiation of adenomatous and hyperplastic diminutive colorectal polyps during analysis of unaltered videos of standard colonoscopy using a deep learning model.
        Gut. 2019; 68: 94-100
        • Chen P.J.
        • Lin M.C.
        • Lai M.J.
        • et al.
        Accurate classification of diminutive colorectal polyps using computer-aided analysis.
        Gastroenterology. 2018; 154: 568-575
        • Takemura Y.
        • Yoshida S.
        • Tanaka S.
        • et al.
        Quantitative analysis and development of a computer-aided system for identification of regular pit patterns of colorectal lesions.
        Gastrointest Endosc. 2010; 72: 1047-1051
        • Misawa M.
        • Kudo S.E.
        • Mori Y.
        • et al.
        Characterization of colorectal lesions using a computer-aided diagnostic system for narrow-band imaging endocytoscopy.
        Gastroenterology. 2016; 150: 1531-1532.e3
        • Mori Y.
        • Kudo S.
        • Chiu P.
        • et al.
        Impact of an automated system for endocytoscopic diagnosis of small colorectal lesions: an international web-based study.
        Endoscopy. 2016; 48: 1110-1118
        • Mori Y.
        • Kudo S.E.
        • Mori K.
        Potential of artificial intelligence-assisted colonoscopy using an endocytoscope (with video).
        Dig Endosc. 2018; 30: 52-53
        • Takeda K.
        • Kudo S.E.
        • Mori Y.
        • et al.
        Accuracy of diagnosing invasive colorectal cancer using computer-aided endocytoscopy.
        Endoscopy. 2017; 49: 798-802
        • André B.
        • Vercauteren T.
        • Buchner A.M.
        • et al.
        Software for automated classification of probe-based confocal laser endomicroscopy videos of colorectal polyps.
        World J Gastroenterol. 2012; 18: 5560-5569
        • Ştefănescu D.
        • Streba C.
        • Cârţână E.T.
        • et al.
        Computer aided diagnosis for confocal laser endomicroscopy in advanced colorectal adenocarcinoma.
        PLoS One. 2016; 11: e0154863
        • Kuiper T.
        • Alderlieste Y.A.
        • Tytgat K.M.
        • et al.
        Automatic optical diagnosis of small colorectal lesions by laser-induced autofluorescence.
        Endoscopy. 2015; 47: 56-62
        • Rath T.
        • Tontini G.E.
        • Vieth M.
        • et al.
        In vivo real-time assessment of colorectal polyp histology using an optical biopsy forceps system based on laser-induced fluorescence spectroscopy.
        Endoscopy. 2016; 48: 557-562
        • Aihara H.
        • Saito S.
        • Inomata H.
        • et al.
        Computer-aided diagnosis of neoplastic colorectal lesions using real-time; numerical color analysis during autofluorescence endoscopy.
        Eur J Gastroenterol Hepatol. 2013; 25: 488-494
        • Inomata H.
        • Tamai N.
        • Aihara H.
        • et al.
        Efficacy of a novel auto-fluorescence imaging system with computer-assisted color analysis for assessment of colorectal lesions.
        World J Gastroenterol. 2013; 19: 7146-7153
        • Aihara H.
        • Sumiyama K.
        • Saito S.
        • et al.
        Numerical analysis of the autoflorescence intensity of neoplastic and non-neoplastic colorectal lesions by using a novel videoendoscopy system.
        Gastrointestinal Endosc. 2009; 69: 726-733
        • Mori Y.
        • Kudo S.E.
        • Wakamura K.
        • et al.
        Novel computer-aided diagnostic system for colorectal lesions by using endocytoscopy (with videos).
        Gastrointest Endosc. 2015; 81: 621-629
        • Misawa M.
        • Kudo S.
        • Mori Y.
        • et al.
        Accuracy of computer-aided diagnosis based on narrow-band imaging endocytoscopy for diagnosing colorectal lesions: comparison with experts.
        Int J Comput Assist Radiol Surg. 2017; 12: 757-766
        • Ozawa T.
        • Ishihara S.
        • Fujishiro M.
        • et al.
        Novel computer-assisted diagnosis system for endoscopic disease activity in patients with ulcerative colitis.
        Gastrointest Endosc. 2019; 89: 416-421
        • Maeda Y.
        • Kudo S.-E.
        • Mori Y.
        • et al.
        Fully automated diagnostic system with artificial intelligence using endocytoscopy to identify the presence of histologic inflammation associated with ulcerative colitis (with video).
        Gastrointest Endosc. 2019; 89: 408-415
        • Khan S.
        • Yong S.
        A comparison of deep learning and hand crafted features in medical image modality classification.
        in: IEEE 3rd International Conference on Computer and Information Sciences. ICCOINS), 2016: 633-638
        • East J.E.
        • Vleugels J.L.
        • Roelandt P.
        • et al.
        Advanced endoscopic imaging.
        European Society of Gastrointestinal Endoscopy (ESGE) Technology Review. 2016; 48: 1029-1045
        • Hewett D.G.
        • Kaltenbach T.
        • Sano Y.
        • et al.
        Validation of a simple classification system for endoscopic diagnosis of small colorectal polyps using narrow-band imaging.
        Gastroenterology. 2012; 143: 599-607.e1
        • Tamai N.
        • Saito Y.
        • Sakamoto T.
        • et al.
        Effectiveness of computer-aided diagnosis of colorectal lesions using novel software for magnifying narrow-band imaging: a pilot study.
        Endosc Int Open. 2017; 5: E690-E694
        • Hirata M.
        • Tanaka S.
        • Oka S.
        • et al.
        Evaluation of microvessels in colorectal tumors by narrow band imaging magnification.
        Gastrointest Endosc. 2007; 66: 945-952
        • Abu Dayyeh B.K.
        • Thosani N.
        • Konda V.
        • et al.
        ASGE technology committee systematic review and meta-analysis assessing the ASGE PIVI thresholds for adopting real-time endoscopic assessment of the histology of diminutive colorectal polyps.
        Gastrointest Endosc. 2015; 81: 502-516
        • Lami M.
        • Singh H.
        • Dilley J.
        • et al.
        Gaze patterns hold key to unlocking successful search strategies and increasing polyp detection rate in colonoscopy.
        Endoscopy. 2018; 50: 701-707
        • Park S.H.
        • Han K.
        Methodologic guide for evaluating clinical performance and effect of artificial intelligence technology for medical diagnosis and prediction.
        Radiology. 2018; 286: 800-809
        • Torosdagli N.
        • Liberton D.K.
        • Verma P.
        • et al.
        Deep geodesic learning for segmentation and anatomical landmarking.
        IEEE Trans Med Imaging. 2019; 38: 919-931
        • Lalonde R.
        • Bagci U.
        Capsules for object segmentation. 1st Conference on Medical Imaging with Deep Learning (MIDL 2018), Amsterdam, The Netherlands.
        Apr 2018 (Available at:)
        • Goodfellow I.
        • Bengio Y.
        • Courville A.
        • et al.
        Deep learning. Cambridge (Mass).
        The MIT Press, 2016 (Available at:) (Accessed March 11, 2019)
      1. Hussein S, Kandel P, Bolan CW, et al. Lung and pancreatic tumor characterization in deep learning era: novel supervised and unsupervised learning approaches. IEEE Trans Med Imaging. Epub 2019 Jan 23.

        • Mortazi A.
        • Karim R.
        • Rhode K.
        • et al.
        CardiacNET: segmentation of left atrium and proximal pulmonary veins from MRI using multi-view CNN.
        in: Descoteaux M. Maier-Hein L. Franz A. Medical image computing and computer-assisted intervention—lecture notes in computer science. vol 10434. Springer Nature Switzerland AG, Switzerland2017: 377-385
        • Corley D.A.
        • Jensen C.D.
        • Marks A.R.
        • et al.
        Adenoma detection rate and risk of colorectal cancer and death.
        N Engl J Med [Internet]. 2014; 370: 1298-1306
        • Rex D.K.
        • Petrini J.
        • Baron T.H.
        • et al.
        Quality Indicators for colonoscopy.
        Gastrointest Endosc. 2006; 63: 16-28
        • Rex D.K.
        • Bond J.H.
        • Winawer S.
        • et al.
        Quality in the technical performance of colonoscopy and the continuous quality improvement process for colonoscopy: recommendations of the U.S. Multi-Society Task Force on Colorectal Cancer.
        Am J Gastroenterol. 2002; 97: 1296-1308
        • Rex D.K.
        • Schoenfeld P.S.
        • Cohen J.
        • et al.
        Quality indicators for colonoscopy.
        Am J Gastroenterol. 2015; 110: 72-90
        • Hilsden R.J.
        • Bridges R.
        • Dube C.
        • et al.
        Defining benchmarks for adenoma detection rate and adenomas per colonoscopy in patients undergoing colonoscopy due to a positive fecal immunochemical test.
        Am J Gastroenterol. 2016; 111: 1743-1749
        • Kahi C.J.
        • Vemulapalli K.C.
        • Johnson C.S.
        • et al.
        Improving measurement of the adenoma detection rate and adenoma per colonoscopy quality metric: the Indiana University experience.
        Gastrointest Endosc. 2014; 79: 448-454
        • Van Rijn J.C.
        • Reitsma J.B.
        • Stoker J.
        • et al.
        Polyp miss rate determined by tandem colonoscopy: a systematic review.
        Am J Gastroenterol. 2006; 101: 343-350
        • Rex D.
        • Kahi C.
        • Levin T.
        • et al.
        The American Society for Gastrointestinal Endoscopy PIVI (Preservation and Incorporation of Valuable Endoscopic Innovations) on real-time endoscopic assessment of the histology of diminutive colorectal polyps.
        Gastrointest Endosc. 2011; 73: 419-422
        • Mori Y.
        • Kudo S.
        • Misawa M.
        • et al.
        Simultaneous detection and characterization of diminutive polyps with the use of artificial intelligence during colonoscopy.
        Video GIE. 2019; 4: 7-10
        • Ichimasa K.
        • Kudo S.E.
        • Mori Y.
        • et al.
        Double staining with crystal violet and methylene blue is appropriate for colonic endocytoscopy: an in vivo prospective pilot study.
        Dig Endosc. 2014; 26: 403-408
        • Justice A.C.
        • Covinsky K.E.
        • Berlin J.A.
        Assessing the generalizability of prognostic information.
        Ann Intern Med. 1999; 130: 515-524
        • Leggett C.L.
        • Fasge K.K.W.
        Computer-aided diagnosis in GI endoscopy: looking into the future.
        Gastrointest Endosc. 2016; 84: 842-844
        • Wang P.
        • Bharadwaj S.
        • Berzin T.M.
        • et al.
        Assistance of a real-time automatic colon polyp detection system increases polyp and adenoma detection during colonoscopy: a prospective randomized controlled study.
        United Eur Gastroenterol J. 2018; 87: 490-491
        • Brocklehurst P.
        • Field D.
        • Greene K.
        • et al.
        Computerised interpretation of fetal heart rate during labour (INFANT): a randomised controlled trial.
        Lancet. 2017; 389: 1719-1729
        • Keith R.D.
        • Beckley S.
        • Garibaldi J.M.
        • et al.
        A multicentre comparative study of 17 experts and an intelligent computer system for managing labour using the cardiotocogram.
        Br J Obstet Gynaecol. 1995; 102: 688-700
        • Oh J.
        • Hwang S.
        • Cao Y.
        • et al.
        Measuring objective quality of colonoscopy.
        IEEE Trans Biomed Eng. 2009; 56: 2190-2196
        • Cabitza F.
        • Rasoini R.
        • Gensini G.F.
        Unintended consequences of machine learning in medicine.
        JAMA. 2017; 318: 517-518
        • Tsai T.L.
        • Fridsma D.B.
        • Gatti G.
        Computer decision support as a source of interpretation error: the case of electrocardiograms.
        J Am Med Informatics Assoc. 2003; 10: 478-483
        • Chinzei K.
        • Shimizu A.
        • Mori K.
        • et al.
        Regulatory science on AI-based medical devices and systems.
        Adv Biomed Eng. 2018; 7: 118-123
        • East J.E.
        • Rees C.J.
        Making optical biopsy a clinical reality in colonoscopy.
        Lancet Gastroenterol Hepatol. 2018; 3: 10-12
        • Food and Drug Administration, HHS
        Radiology devices; reclassification of medical image analyzers: a proposed rule by the Food and Drug Administration.
        Silver Spring (MD). 2018; (Available at:) (Accessed February 26, 2019): 25598-25604
        • Park S.H.
        Regulatory approval versus clinical validation of artificial intelligence diagnostic tools.
        Radiology. 2018; 288: 910-911
        • Yu K.-H.
        • Beam A.L.
        • Kohane I.S.
        Artificial intelligence in healthcare.
        Nat Biomed Eng. 2018; 2: 719-731