High performance in risk stratification of intraductal papillary mucinous neoplasms by confocal laser endomicroscopy image analysis with convolutional neural networks (with video)

Published:January 15, 2021DOI:

      Background and Aims

      EUS-guided needle-based confocal laser endomicroscopy (EUS-nCLE) can differentiate high-grade dysplasia/adenocarcinoma (HGD-Ca) in intraductal papillary mucinous neoplasms (IPMNs) but requires manual interpretation. We sought to derive predictive computer-aided diagnosis (CAD) and artificial intelligence (AI) algorithms to facilitate accurate diagnosis and risk stratification of IPMNs.


      A post hoc analysis of a single-center prospective study evaluating EUS-nCLE (2015-2019; INDEX study) was conducted using 15,027 video frames from 35 consecutive patients with histopathologically proven IPMNs (18 with HGD-Ca). We designed 2 CAD-convolutional neural network (CNN) algorithms: (1) a guided segmentation-based model (SBM), where the CNN-AI system was trained to detect and measure papillary epithelial thickness and darkness (indicative of cellular and nuclear stratification), and (2) a reasonably agnostic holistic-based model (HBM) where the CNN-AI system automatically extracted nCLE features for risk stratification. For the detection of HGD-Ca in IPMNs, the diagnostic performance of the CNN-CAD algorithms was compared with that of the American Gastroenterological Association (AGA) and revised Fukuoka guidelines.


      Compared with the guidelines, both n-CLE-guided CNN-CAD algorithms yielded higher sensitivity (HBM, 83.3%; SBM, 83.3%; AGA, 55.6%; Fukuoka, 55.6%) and accuracy (SBM, 82.9%; HBM, 85.7%; AGA, 68.6%; Fukuoka, 74.3%) for diagnosing HGD-Ca, with comparable specificity (SBM, 82.4%; HBM, 88.2%; AGA, 82.4%; Fukuoka, 94.1%). Both CNN-CAD algorithms, the guided (SBM) and agnostic (HBM) models, were comparable in risk stratifying IPMNs.


      EUS-nCLE-based CNN-CAD algorithms can accurately risk stratify IPMNs. Future multicenter validation studies and AI model improvements could enhance the accuracy and fully automatize the process for real-time interpretation.


      AGA (American Gastroenterology Association), AI (artificial intelligence), AUC (area under the receiver operating characteristic curve), CAD (computer-aided diagnosis), CI (confidence interval), CNN (convolutional neural network), HBM (holistic-based model), HGD-Ca (high-grade dysplasia and/or adenocarcinoma), IPMN (intraductal papillary mucinous neoplasm), LGD (low- or intermediate-grade dysplasia), LR (likelihood ratio), nCLE (needle-based confocal laser endomicroscopy), PCL (pancreatic cystic lesion), ROI (region of interest), SBM (segmentation-based model), SD (standard deviation)
      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Gastrointestinal Endoscopy
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Zerboni G.
        • Signoretti M.
        • Crippa S.
        • et al.
        Systematic review and meta-analysis: prevalence of incidentally detected pancreatic cystic lesions in asymptomatic individuals.
        Pancreatology. 2019; 19: 2-9
        • Singhi A.D.
        • Koay E.J.
        • Chari S.T.
        • et al.
        Early detection of pancreatic cancer: opportunities and challenges.
        Gastroenterology. 2019; 156: 2024-2040
        • Tanaka M.
        • Fernandez-del Castillo C.
        • Adsay V.
        • et al.
        International consensus guidelines 2012 for the management of IPMN and MCN of the pancreas.
        Pancreatology. 2012; 12: 183-197
        • Vege S.S.
        • Ziring B.
        • Jain R.
        • Moayyedi P.
        • et al.
        American Gastroenterological Association Institute guideline on the diagnosis and management of asymptomatic neoplastic pancreatic cysts.
        Gastroenterology. 2015; 148: 819-822
        • Elta G.H.
        • Enestvedt B.K.
        • Sauer B.G.
        • et al.
        ACG Clinical Guideline: diagnosis and management of pancreatic cysts.
        Am J Gastroenterol. 2018; 113: 464-479
        • Tanaka M.
        • Fernandez-Del Castillo C.
        • Kamisawa T.
        • et al.
        Revisions of international consensus Fukuoka guidelines for the management of IPMN of the pancreas.
        Pancreatology. 2017; 17: 738-753
        • Sahora K.
        • Mino-Kenudson M.
        • Brugge W.
        • et al.
        Branch duct intraductal papillary mucinous neoplasms: does cyst size change the tip of the scale? A critical analysis of the revised international consensus guidelines in a large single-institutional series.
        Ann Surg. 2013; 258: 466-475
        • Springer S.
        • Masica D.L.
        • Dal Molin M.
        • et al.
        A multimodality test to guide the management of patients with a pancreatic cyst.
        Sci Transl Med. 2019; 11eaav4772
        • Scheiman J.M.
        • Hwang J.H.
        • Moayyedi P.
        American gastroenterological association technical review on the diagnosis and management of asymptomatic neoplastic pancreatic cysts.
        Gastroenterology. 2015; 148: 824-848.e22
        • Kamboj A.K.
        • Modi R.M.
        • Swanson B.
        • et al.
        A comprehensive examination of the novel techniques used for in vivo and ex vivo confocal laser endomicroscopy of pancreatic cystic lesions.
        VideoGIE. 2016; 1: 6-7
        • Nakai Y.
        • Iwashita T.
        • Park D.H.
        • et al.
        Diagnosis of pancreatic cysts: EUS-guided, through-the-needle confocal laser-induced endomicroscopy and cystoscopy trial: DETECT study.
        Gastrointest Endosc. 2015; 81: 1204-1214
        • Konda V.J.
        • Meining A.
        • Jamil L.H.
        • et al.
        A pilot study of in vivo identification of pancreatic cystic neoplasms with needle-based confocal laser endomicroscopy under endosonographic guidance.
        Endoscopy. 2013; 45: 1006-1013
        • Napoleon B.
        • Palazzo M.
        • Lemaistre A.I.
        • et al.
        Needle-based confocal laser endomicroscopy of pancreatic cystic lesions: a prospective multicenter validation study in patients with definite diagnosis.
        Endoscopy. 2019; 51: 825-835
        • Krishna S.G.
        • Hart P.A.
        • Malli A.
        • et al.
        Endoscopic ultrasound-guided confocal laser endomicroscopy increases accuracy of differentiation of pancreatic cystic lesions.
        Clin Gastroenterol Hepatol. 2020; 18: 432-440 e6
        • Krishna S.G.
        • Modi R.M.
        • Kamboj A.K.
        • et al.
        In vivo and ex vivo confocal endomicroscopy of pancreatic cystic lesions: a prospective study.
        World J Gastroenterol. 2017; 23: 3338-3348
        • Krishna S.G.
        • Brugge W.R.
        • Dewitt J.M.
        • et al.
        Needle-based confocal laser endomicroscopy for the diagnosis of pancreatic cystic lesions: an international external interobserver and intraobserver study (with videos).
        Gastrointest Endosc. 2017; 86: 644-654.e2
        • Krishna S.G.
        • Hart P.A.
        • DeWitt J.M.
        • et al.
        EUS-guided confocal laser endomicroscopy: prediction of dysplasia in intraductal papillary mucinous neoplasms (with video).
        Gastrointest Endosc. 2020; 91: 551-563.e5
        • Coban S.
        • Brugge W.
        EUS-guided confocal laser endomicroscopy: can we use thick and wide for diagnosis of early cancer?.
        Gastrointest Endosc. 2020; 91: 564-567
        • Corral J.E.
        • Hussein S.
        • Kandel P.
        • et al.
        Deep learning to classify intraductal papillary mucinous neoplasms using magnetic resonance imaging.
        Pancreas. 2019; 48: 805-810
        • Machicado J.D.
        • Koay E.J.
        • Krishna S.G.
        Radiomics for the diagnosis and differentiation of pancreatic cystic lesions.
        Diagnostics (Basel). 2020; 10: 505
        • Basturk O.
        • Hong S.-M.
        • Wood L.D.
        • et al.
        A revised classification system and recommendations from the Baltimore consensus meeting for neoplastic precursor lesions in the pancreas.
        Am J Surg Pathol. 2015; 39: 1730
      1. Paszke A, Gross S, Massa F, et al. PyTorch: an imperative style, high-performance deep learning library. Proceedings of Advances in Neural Information Processing Systems 32, Vancouver, BC, Canada; 2019:8024-8035.

      2. Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. arXiv:14091556 2014.

        • Lambin P.
        • Leijenaar R.T.H.
        • Deist T.M.
        • et al.
        Radiomics: the bridge between medical imaging and personalized medicine.
        Nat Rev Clin Oncol. 2017; 14: 749-762
        • He K.
        • Gkioxari G.
        • Dollar P.
        • et al.
        Mask R-CNN. Proceedings of the International Conference on Computer Vision (ICCV).
        2017: 2961-2969 (Venice)
        • Xu M.M.
        • Yin S.
        • Siddiqui A.A.
        • et al.
        Comparison of the diagnostic accuracy of three current guidelines for the evaluation of asymptomatic pancreatic cystic neoplasms.
        Medicine (Baltimore). 2017; 96: e7900
        • Zhou W.
        • Xu Y.
        • Rong Y.
        • et al.
        Validation of Sendai and Fukuoka consensus guidelines in predicting malignancy in patients with preoperatively diagnosed mucinous pancreatic cystic neoplasms.
        J Surg Oncol. 2018; 117: 409-416
        • Singhi A.D.
        • Zeh H.J.
        • Brand R.E.
        • et al.
        American Gastroenterological Association guidelines are inaccurate in detecting pancreatic cysts with advanced neoplasia: a clinicopathologic study of 225 patients with supporting molecular data.
        Gastrointest Endosc. 2016; 83: 1107-1117.e2
        • Eiterman A.
        • Lahooti A.
        • Krishna S.G.
        Endosonographic diagnosis of advanced neoplasia in intraductal papillary mucinous neoplasms.
        World J Gastroenterol. 2020; 26: 3201-3212

      Linked Article

      • Risk stratification of pancreatic cysts: a convoluted path to finding the needle in the haystack
        Gastrointestinal EndoscopyVol. 94Issue 1
        • Preview
          Improved resolution and quality of abdominal cross-sectional imaging and its widespread clinical use has been associated with an increase in the incidental detection of pancreatic cystic lesions.1,2 Intraductal papillary mucinous neoplasms (IPMNs) are the most common type of incidentally detected pancreatic cysts, and although they are precancerous, most of them will never progress to cancer.3-5 Current risk stratification guidelines for the management of presumed IPMNs aim to identify cysts with advanced neoplasia: high-grade dysplasia or invasive cancer, by use of a combination of clinical and imaging characteristics.
        • Full-Text
        • PDF