Real-time automated diagnosis of precancerous lesions and early esophageal squamous cell carcinoma using a deep learning model (with videos)

Published:August 21, 2019DOI:

      Background and Aims

      We developed a system for computer-assisted diagnosis (CAD) for real-time automated diagnosis of precancerous lesions and early esophageal squamous cell carcinomas (ESCCs) to assist the diagnosis of esophageal cancer.


      A total of 6473 narrow-band imaging (NBI) images, including precancerous lesions, early ESCCs, and noncancerous lesions, were used to train the CAD system. We validated the CAD system using both endoscopic images and video datasets. The receiver operating characteristic curve of the CAD system was generated based on image datasets. An artificial intelligence probability heat map was generated for each input of endoscopic images. The yellow color indicated high possibility of cancerous lesion, and the blue color indicated noncancerous lesions on the probability heat map. When the CAD system detected any precancerous lesion or early ESCCs, the lesion of interest was masked with color.


      The image datasets contained 1480 malignant NBI images from 59 consecutive cancerous cases (sensitivity, 98.04%) and 5191 noncancerous NBI images from 2004 cases (specificity, 95.03%). The area under curve was 0.989. The video datasets of precancerous lesions or early ESCCs included 27 nonmagnifying videos (per-frame sensitivity 60.8%, per-lesion sensitivity, 100%) and 20 magnifying videos (per-frame sensitivity 96.1%, per-lesion sensitivity, 100%). Unaltered full-range normal esophagus videos included 33 videos (per-frame specificity 99.9%, per-case specificity, 90.9%).


      A deep learning model demonstrated high sensitivity and specificity for both endoscopic images and video datasets. The real-time CAD system has a promising potential in the near future to assist endoscopists in diagnosing precancerous lesions and ESCCs.


      AI (artificial intelligence), CAD (computer-assisted diagnosis), ESCC (esophageal squamous cell carcinoma), FN (false negative), FP (false positive), IPCL (intrapapillary capillary loops), NBI (narrow-band imaging), ROC (receiver operating characteristic), TN (true negative), TP (true positive), WCH (West China Hospital)
      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Gastrointestinal Endoscopy
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Rustgi A.K.
        • El-Serag H.B.
        Esophageal carcinoma.
        N Engl J Med. 2014; 371: 2499-2509
        • Chen X.X.
        • Zhong Q.
        • Liu Y.
        • et al.
        Genomic comparison of esophageal squamous cell carcinoma and its precursor lesions by multi-region whole-exome sequencing.
        Nat Commun. 2017; 8: 524
        • Nagami Y.
        • Tominaga K.
        • Machida H.
        • et al.
        Usefulness of non-magnifying narrow-band imaging in screening of early esophageal squamous cell carcinoma: a prospective comparative study using propensity score matching.
        Am J Gastroenterol. 2014; 109: 845-854
        • Minami H.
        • Isomoto H.
        • Inoue H.
        • et al.
        Significance of background coloration in endoscopic detection of early esophageal squamous cell carcinoma.
        Digestion. 2014; 89: 6-11
        • Ishihara R.
        • Takeuchi Y.
        • Chatani R.
        • et al.
        Prospective evaluation of narrow-band imaging endoscopy for screening of esophageal squamous mucosal high-grade neoplasia in experienced and less experienced endoscopists.
        Dis Esophagus. 2010; 23: 480-486
        • Rodriguez de Santiago E.
        • Hernanz N.
        • Marcos-Prieto H.M.
        • et al.
        Rate of missed oesophageal cancer at routine endoscopy and survival outcomes: a multicentric cohort study.
        United European Gastroenterol J. 2019; 7: 189-198
        • Wang P.
        • Xiao X.
        • Glissen Brown J.R.
        • et al.
        Development and validation of a deep-learning algorithm for the detection of polyps during colonoscopy.
        Nat Biomed Eng. 2018; 2: 741-748
        • Hirasawa T.
        • Aoyama K.
        • Tanimoto T.
        • et al.
        Application of artificial intelligence using a convolutional neural network for detecting gastric cancer in endoscopic images.
        Gastric Cancer. 2018; 21: 653-660
        • Yamada S.
        • Doyama H.
        • Yao K.
        • et al.
        An efficient diagnostic strategy for small, depressed early gastric cancer with magnifying narrow-band imaging: a post-hoc analysis of a prospective randomized controlled trial.
        Gastrointest Endosc. 2014; 79: 55-63
        • Nakashima H.
        • Kawahira H.
        • Kawachi H.
        • et al.
        Artificial intelligence diagnosis of Helicobacter pylori infection using blue laser imaging-bright and linked color imaging: a single-center prospective study.
        Ann Gastroenterol. 2018; 31: 462-468
        • Horie Y.
        • Yoshio T.
        • Aoyama K.
        • et al.
        Diagnostic outcomes of esophageal cancer by artificial intelligence using convolutional neural networks.
        Gastrointest Endosc. 2019; 89: 25-32
        • Zhao Y.Y.
        • Xue D.X.
        • Wang Y.L.
        • et al.
        Computer-assisted diagnosis of early esophageal squamous cell carcinoma using narrow-band imaging magnifying endoscopy.
        Endoscopy. 2019; 51: 333-341
        • Swager A.F.
        • van der Sommen F.
        • Klomp S.R.
        • et al.
        Computer-aided detection of early Barrett's neoplasia using volumetric laser endomicroscopy.
        Gastrointest Endosc. 2017; 86: 839-846
        • Endoscopic Classification Review Group
        Update on the paris classification of superficial neoplastic lesions in the digestive tract.
        Endoscopy. 2005; 37: 570-578
        • Cotton P.B.
        Quality endoscopists and quality endoscopy units.
        J Interv Gastroenterol. 2011; 1: 83-87
        • Byrne M.F.
        • Chapados N.
        • Soudan F.
        • et al.
        Real-time differentiation of adenomatous and hyperplastic diminutive colorectal polyps during analysis of unaltered videos of standard colonoscopy using a deep learning model.
        Gut. 2019; 68: 94-100
        • Codipilly D.C.
        • Qin Y.
        • Dawsey S.M.
        • et al.
        Screening for esophageal squamous cell carcinoma: recent advances.
        Gastrointest Endosc. 2018; 88: 413-426

      Linked Article