Real-time artificial intelligence for detecting focal lesions and diagnosing neoplasms of the stomach by white-light endoscopy (with videos)

  • Author Footnotes
    ∗ Drs Wu, Xu, and Jiang contributed equally to this article.
    Lianlian Wu
    Footnotes
    ∗ Drs Wu, Xu, and Jiang contributed equally to this article.
    Affiliations
    Department of Gastroenterology, Renmin Hospital of Wuhan University, Wuhan, China

    Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China

    Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China
    Search for articles by this author
  • Author Footnotes
    ∗ Drs Wu, Xu, and Jiang contributed equally to this article.
    Ming Xu
    Footnotes
    ∗ Drs Wu, Xu, and Jiang contributed equally to this article.
    Affiliations
    Department of Gastroenterology, Renmin Hospital of Wuhan University, Wuhan, China

    Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China

    Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China
    Search for articles by this author
  • Author Footnotes
    ∗ Drs Wu, Xu, and Jiang contributed equally to this article.
    Xiaoda Jiang
    Footnotes
    ∗ Drs Wu, Xu, and Jiang contributed equally to this article.
    Affiliations
    Department of Gastroenterology, Renmin Hospital of Wuhan University, Wuhan, China

    Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China

    Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China
    Search for articles by this author
  • Xinqi He
    Affiliations
    Department of Gastroenterology, Renmin Hospital of Wuhan University, Wuhan, China

    Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China

    Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China
    Search for articles by this author
  • Heng Zhang
    Affiliations
    Department of Gastroenterology, Central Hospital of Wuhan, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, China
    Search for articles by this author
  • Yaowei Ai
    Affiliations
    Department of Gastroenterology, The People's Hospital of China Three Gorges University, The First People’s Hospital of Yichang, Yichang, China
    Search for articles by this author
  • Qiaoyun Tong
    Affiliations
    Department of Gastroenterology, Yichang Central People’s Hospital & Institute of Digestive Diseases, China Three Gorges University, Yichang, China
    Search for articles by this author
  • Peihua Lv
    Affiliations
    Spleen and Stomach Department, Jingmen Petrochemical Hospital, Jingmen, China
    Search for articles by this author
  • Bin Lu
    Affiliations
    Department of Gastroenterology, Xiaogan Central Hospital, Xiaogan, China
    Search for articles by this author
  • Mingwen Guo
    Affiliations
    Department of Gastroenterology, The People's Hospital of China Three Gorges University, The First People’s Hospital of Yichang, Yichang, China
    Search for articles by this author
  • Manling Huang
    Affiliations
    Department of Gastroenterology, Central Hospital of Wuhan, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, China
    Search for articles by this author
  • Liping Ye
    Affiliations
    Department of Gastroenterology, Taizhou Hospital of Zhejiang Province affiliated to Wenzhou Medical University, Linhai, China
    Search for articles by this author
  • Lei Shen
    Affiliations
    Department of Gastroenterology, Renmin Hospital of Wuhan University, Wuhan, China

    Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China

    Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China
    Search for articles by this author
  • Honggang Yu
    Correspondence
    Reprint requests: Honggang Yu or Lei Shen, Department of Gastroenterology, Renmin Hospital of Wuhan University, 99 Zhangzhidong Rd, Wuhan 430060, Hubei Province, China.
    Affiliations
    Department of Gastroenterology, Renmin Hospital of Wuhan University, Wuhan, China

    Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China

    Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China
    Search for articles by this author
  • Author Footnotes
    ∗ Drs Wu, Xu, and Jiang contributed equally to this article.
Published:September 18, 2021DOI:https://doi.org/10.1016/j.gie.2021.09.017

      Background and Aims

      White-light endoscopy (WLE) is the most pivotal tool to detect gastric cancer in an early stage. However, the skill among endoscopists varies greatly. Here, we aim to develop a deep learning–based system named ENDOANGEL-LD (lesion detection) to assist in detecting all focal gastric lesions and predicting neoplasms by WLE.

      Methods

      Endoscopic images were retrospectively obtained from Renmin Hospital of Wuhan University (RHWU) for the development, validation, and internal test of the system. Additional external tests were conducted in 5 other hospitals to evaluate the robustness. Stored videos from RHWU were used for assessing and comparing the performance of ENDOANGEL-LD with that of experts. Prospective consecutive patients undergoing upper endoscopy were enrolled from May 6, 2021 to August 2, 2021 in RHWU to assess clinical practice applicability.

      Results

      Over 10,000 patients undergoing upper endoscopy were enrolled in this study. The sensitivities were 96.9% and 95.6% for detecting gastric lesions and 92.9% and 91.7% for diagnosing neoplasms in internal and external patients, respectively. In 100 videos, ENDOANGEL-LD achieved superior sensitivity and negative predictive value for detecting gastric neoplasms from that of experts (100% vs 85.5% ± 3.4% [P = .003] and 100% vs 86.4% ± 2.8% [P = .002], respectively). In 2010 prospective consecutive patients, ENDOANGEL-LD achieved a sensitivity of 92.8% for detecting gastric lesions with 3.04 ± 3.04 false positives per gastroscopy and a sensitivity of 91.8% and specificity of 92.4% for diagnosing neoplasms.

      Conclusions

      Our results show that ENDOANGEL-LD has great potential for assisting endoscopists in screening gastric lesions and suspicious neoplasms in clinical work. (Clinical trial registration number: ChiCTR2100045963.)

      Abbreviations:

      AI (artificial intelligence), CNN (convolutional neural network), EGC (early gastric cancer), GC (gastric cancer), LD (lesion detection), M-IEE (magnifying image-enhanced endoscopy), NPV (negative predictive value), PPV (positive predictive value), RHWU (Renmin Hospital of Wuhan University), SD (standard deviation), WLE (white light endoscopy)
      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'

      Subscribe:

      Subscribe to Gastrointestinal Endoscopy
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect

      References

        • Karimi P.
        • Islami F.
        • Anandasabapathy S.
        • et al.
        Gastric cancer: descriptive epidemiology, risk factors, screening, and prevention.
        Cancer Epidemiol Biomark Prevent. 2014; 23: 700-713
        • Bray F.
        • Ferlay J.
        • Soerjomataram I.
        • et al.
        Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries.
        CA Cancer J Clin. 2018; 68: 394-424
        • Chen W.
        • Zheng R.
        • Baade P.D.
        • et al.
        Cancer statistics in China, 2015.
        CA Cancer J Clin. 2016; 66: 115-132
        • Laks S.
        • Meyers M.O.
        • Kim H.J.
        Surveillance for gastric cancer.
        Surg Clin North Am. 2017; 97: 317-331
        • Imagawa A.
        • Okada H.
        • Kawahara Y.
        • et al.
        Endoscopic submucosal dissection for early gastric cancer: results and degrees of technical difficulty as well as success.
        Endoscopy. 2006; 38: 987-990
        • Bisschops R.
        • Areia M.
        • Coron E.
        • et al.
        Performance measures for upper gastrointestinal endoscopy: a European Society of Gastrointestinal Endoscopy (ESGE) quality improvement initiative.
        Endoscopy. 2016; 48: 843-864
        • Pasechnikov V.
        • Chukov S.
        • Fedorov E.
        • et al.
        Gastric cancer: prevention, screening and early diagnosis.
        World J Gastroenterol. 2014; 20: 13842-13862
        • Banks M.
        • Graham D.
        • Jansen M.
        • et al.
        British Society of Gastroenterology guidelines on the diagnosis and management of patients at risk of gastric adenocarcinoma.
        Gut. 2019; 68: 1545-1575
        • Chiu P.W.Y.
        • Uedo N.
        • Singh R.
        • et al.
        An Asian consensus on standards of diagnostic upper endoscopy for neoplasia.
        Gut. 2019; 68: 186-197
        • Evans J.A.
        • Chandrasekhara V.
        • et al.
        • ASGE Standards of Practice Committee
        The role of endoscopy in the management of premalignant and malignant conditions of the stomach.
        Gastrointest Endosc. 2015; 82: 1-8
        • Rutter M.D.
        • Senore C.
        • Bisschops R.
        • et al.
        The European Society of Gastrointestinal endoscopy quality improvement initiative: developing performance measures.
        Endoscopy. 2016; 48: 81-89
        • Kaise M.
        Advanced endoscopic imaging for early gastric cancer.
        Best Pract Res Clin Gastroenterol. 2015; 29: 575-587
        • Litjens G.
        • Kooi T.
        • Bejnordi B.E.
        • et al.
        A survey on deep learning in medical image analysis.
        Med Image Anal. 2017; 42: 60-88
        • Repici A.
        • Badalamenti M.
        • Maselli R.
        • et al.
        Efficacy of real-time computer-aided detection of colorectal neoplasia in a randomized trial.
        Gastroenterology. 2020; 159: 512-520
        • Bisschops R.
        • East J.E.
        • Hassan C.
        • et al.
        Advanced imaging for detection and differentiation of colorectal neoplasia: European Society of Gastrointestinal Endoscopy (ESGE) guideline—update 2019.
        Endoscopy. 2019; 51: 1155-1179
        • Swager A.F.
        • van der Sommen F.
        • Klomp S.R.
        • et al.
        Computer-aided detection of early Barrett's neoplasia using volumetric laser endomicroscopy.
        Gastrointest Endosc. 2017; 86: 839-846
        • Wu L.
        • Zhang J.
        • Zhou W.
        • et al.
        Randomised controlled trial of WISENSE, a real-time quality improving system for monitoring blind spots during esophagogastroduodenoscopy.
        Gut. 2019; 68: 2161-2169
        • Li L.
        • Chen Y.
        • Shen Z.
        • et al.
        Convolutional neural network for the diagnosis of early gastric cancer based on magnifying narrow band imaging.
        Gastric Cancer. 2020; 23: 126-132
        • Hu H.
        • Gong L.
        • Dong D.
        • et al.
        Identifying early gastric cancer under magnifying narrow-band images with deep learning: a multicenter study.
        Gastrointest Endosc. 2020; 93: 1333-1341.e3
        • Horiuchi Y.
        • Hirasawa T.
        • Ishizuka N.
        • et al.
        Performance of a computer-aided diagnosis system in diagnosing early gastric cancer using magnifying endoscopy videos with narrow-band imaging (with videos).
        Gastrointest Endosc. 2020; 92: 856-865
        • Tang D.
        • Wang L.
        • Ling T.
        • et al.
        Development and validation of a real-time artificial intelligence-assisted system for detecting early gastric cancer: a multicentre retrospective diagnostic study.
        EBioMedicine. 2020; 62: 103146
        • Wu L.
        • Zhou W.
        • Wan X.
        • et al.
        A deep neural network improves endoscopic detection of early gastric cancer without blind spots.
        Endoscopy. 2019; 51: 522-531
        • An P.
        • Yang D.
        • Wang J.
        • et al.
        A deep learning method for delineating early gastric cancer resection margin under chromoendoscopy and white light endoscopy.
        Gastric Cancer. 2020; 23: 884-892
        • Hirasawa T.
        • Aoyama K.
        • Tanimoto T.
        • et al.
        Application of artificial intelligence using a convolutional neural network for detecting gastric cancer in endoscopic images.
        Gastric Cancer. 2018; 21: 653-660
        • Cho B.J.
        • Bang C.S.
        • Park S.W.
        • et al.
        Automated classification of gastric neoplasms in endoscopic images using a convolutional neural network.
        Endoscopy. 2019; 51: 1121-1129
        • Mori Y.
        • Kudo S.E.
        • Mohmed H.E.N.
        • et al.
        Artificial intelligence and upper gastrointestinal endoscopy: current status and future perspective.
        Dig Endosc. 2019; 31: 378-388
        • Wu L.
        • He X.
        • Liu M.
        • et al.
        Evaluation of the effects of an artificial intelligence system on endoscopy quality and preliminarily testing its performance on detecting early gastric cancer: a randomized controlled trial.
        Endoscopy. Epub. 2021; (Jan 11)
        • Redmon J.
        • Divvala S.
        • Girshick R.
        • et al.
        You only look once: unified, real-time object detection.
        2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, Nevada, USA. 2016; (p. 779-88)
        • He K.
        • Zhang X.
        • Ren S.
        • et al.
        Deep residual learning for image recognition.
        2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, Nevada, USA. 2016; (p. 770-8)
        • Yao K.
        The endoscopic diagnosis of early gastric cancer.
        J Ann Gastroenterol. 2013; 26: 11-22
        • Yoshimizu S.
        • Hirasawa T.
        • Horiuchi Y.
        • et al.
        Differences in upper gastrointestinal neoplasm detection rates based on inspection time and esophagogastroduodenoscopy training.
        Endosc Int Open. 2018; 6: E1190-E1197
        • Yao K.
        • Uedo N.
        • Muto M.
        • et al.
        Development of an e-learning system for teaching endoscopists how to diagnose early gastric cancer: basic principles for improving early detection.
        Gastric Cancer. 2017; 20: 28-38
        • Nakanishi H.
        • Doyama H.
        • Ishikawa H.
        • et al.
        Evaluation of an e-learning system for diagnosis of gastric lesions using magnifying narrow-band imaging: a multicenter randomized controlled study. Endoscopy.
        . 2017; 49: 957-967
        • Topol E.J.
        High-performance medicine: the convergence of human and artificial intelligence.
        Nat Med. 2019; 25: 44-56
        • Horiuchi Y.
        • Aoyama K.
        • Tokai Y.
        • et al.
        Convolutional neural network for differentiating gastric cancer from gastritis using magnified endoscopy with narrow band imaging.
        Dig Dis Sci. 2020; 65: 1355-1363
        • Ikenoyama Y.
        • Hirasawa T.
        • Ishioka M.
        • et al.
        Detecting early gastric cancer: comparison between the diagnostic ability of convolutional neural networks and endoscopists.
        Dig Endosc. 2021; 33: 141-150
        • White J.R.
        • Winter J.A.
        • Robinson K.
        Differential inflammatory response to Helicobacter pylori infection: etiology and clinical outcomes.
        J Inflamm Res. 2015; 8: 137-147

      References

        • Tsung-Yi L.
        • Michael M.
        • Serge B.
        • et al.
        Microsoft coco: common objects in context.
        in: European conference on computer vision. Springer, Cham, Switzerland2014: 740-755
        • Deng J.
        • Dong W.
        • Socher R.
        • et al.
        ImageNet: a large-scale hierarchical image database.
        2009 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Miami Beach, FL, USA. 2009; (p. 248-55)
        • Shao L.
        • Zhu F.
        • Li X.
        Transfer learning for visual categorization: a survey.
        IEEE Trans Neural Netw Learn Syst. 2015; 26: 1019-1034
      1. Martín A, Ashish A, Paul B, et al. Tensorflow: large-scale machine learning on heterogeneous distributed systems. arXiv 2016; arXiv:1603.04467.

        • Baldi P.
        • Sadowski P.
        The dropout learning algorithm.
        Artif Intell. 2014; 210: 78-122
      2. Cortes C, Mehryar M, Afshin R. L2 regularization for learning kernels. arXiv 2012; arXiv:1205.2653.

        • Prechelt L.
        Automatic early stopping using cross validation: quantifying the criteria.
        Neural Netw. 1998; 11: 761-767
        • Tanner M.A.
        • Wong W.H.
        The calculation of posterior distributions by data augmentation.
        J Am Stat Assoc. 1987; 82: 528-540
        • Wu L.
        • Zhang J.
        • Zhou W.
        • et al.
        Randomised controlled trial of WISENSE, a real-time quality improving system for monitoring blind spots during esophagogastroduodenoscopy.
        Gut. 2019; 68: 2161-2169