Automatic detection and classification of protruding lesions in wireless capsule endoscopy images based on a deep convolutional neural network

Published:February 18, 2020DOI:

      Background and Aims

      Protruding lesions of the small bowel vary in wireless capsule endoscopy (WCE) images, and their automatic detection may be difficult. We aimed to develop and test a deep learning–based system to automatically detect protruding lesions of various types in WCE images.


      We trained a deep convolutional neural network (CNN), using 30,584 WCE images of protruding lesions from 292 patients. We evaluated CNN performance by calculating the area under the receiver operating characteristic curve (AUC), sensitivity, and specificity, using an independent set of 17,507 test images from 93 patients, including 7507 images of protruding lesions from 73 patients.


      The developed CNN analyzed 17,507 images in 530.462 seconds. The AUC for detection of protruding lesions was 0.911 (95% confidence interval [Cl], 0.9069–0.9155). The sensitivity and specificity of the CNN were 90.7% (95% CI, 90.0%–91.4%) and 79.8% (95% CI, 79.0%–80.6%), respectively, at the optimal cut-off value of 0.317 for probability score. In a subgroup analysis of the category of protruding lesions, the sensitivities were 86.5%, 92.0%, 95.8%, 77.0%, and 94.4% for the detection of polyps, nodules, epithelial tumors, submucosal tumors, and venous structures, respectively. In individual patient analyses (n = 73), the detection rate of protruding lesions was 98.6%.


      We developed and tested a new computer-aided system based on a CNN to automatically detect various protruding lesions in WCE images. Patient-level analyses with larger cohorts and efforts to achieve better diagnostic performance are necessary in further studies.


      AI (artificial intelligence), AUC (area under the curve), CEST (capsule endoscopy structured terminology), CI (confidence interval), CNN (convolutional neural network), FAP (familial adenomatous polyposis), IoU (intersection over union), PS (probability score), ROC (receiver operating characteristic), SMT (submucosal tumors), SSD (single shot multibox detector), WCE (wireless capsule endoscopy)
      To read this article in full you will need to make a payment


      Subscribe to Gastrointestinal Endoscopy
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Iddan G.
        • Meron G.
        • Glukhovsky A.
        • et al.
        Wireless capsule endoscopy.
        Nature. 2000; 405 (417-17)
        • Costamagna G.
        • Shah S.
        • Riccioni M.
        • et al.
        A prospective trial comparing small bowel radiographs and video capsule endoscopy for suspected small bowel disease.
        Gastroenterology. 2002; 123: 999-1005
        • Omori T.
        • Hara T.
        • Sakasai S.
        • et al.
        Does the PillCam SB3 capsule endoscopy system improve image reading efficiency irrespective of experience? A pilot study.
        Endosc Int Open. 2018; 6: E669-E675
        • McAlindon M.E.
        • Ching H.-L.
        • Yung D.
        • et al.
        Capsule endoscopy of the small bowel.
        Ann Transl Med. 2016; 4: 369
        • Leenhardt R.
        • Vasseur P.
        • Li C.
        • et al.
        A neural network algorithm for detection of GI angiectasia during small-bowel capsule endoscopy.
        Gastrointest Endosc. 2019; 89: 189-194
        • Korman L.Y.
        • Delvaux M.
        • Gay G.
        • et al.
        Capsule endoscopy structured terminology (CEST): proposal of a standardized and structured terminology for reporting capsule endoscopy procedures.
        Endoscopy. 2005; 37: 951-959
        • Shah P.
        • Kendall F.
        • Khozin S.
        • et al.
        Artificial intelligence and machine learning in clinical development: a translational perspective.
        NPJ Digit Med. 2019; 2: 69
        • Misawa M.
        • Kudo S.E.
        • Mori Y.
        • et al.
        Artificial intelligence-assisted polyp detection for colonoscopy: initial experience.
        Gastroenterology. 2018; 154: 2027-2029
        • Aoki T.
        • Yamada A.
        • Aoyama K.
        • et al.
        Automatic detection of erosions and ulcerations in wireless capsule endoscopy images based on a deep convolutional neural network.
        Gastrointest Endosc. 2019; 89: 357-363 e2
        • Ding Z.
        • Shi H.
        • Zhang H.
        • et al.
        Gastroenterologist-level identification of small bowel diseases and normal variants by capsule endoscopy using a deep-learning model.
        Gastroenterology. 2019; 157: 1044-1054
        • Tsuboi A.
        • Oka S.
        • Aoyama K.
        • et al.
        Artificial intelligence using a convolutional neural network for automatic detection of small-bowel angioectasia in capsule endoscopy images.
        Dig Endosc. Epub. 2019 Aug 7;
      1. Liu W, Anguelov D, Erhan D, et al. SSD: single shot multibox detector. arXiv:1512.02325v5 [cs.CV] 2016. Available at: Accessed October 29, 2019.

      2. Yangqing J, Evan S, Jeff D, et al. Caffe: convolutional architecture for fast feature embedding. Available at: Accessed October 29, 2019.

        • Youden W.J.
        Index for rating diagnostic tests.
        Cancer. 1950; 3: 32-35
        • Vieira P.M.
        • Freitas N.R.
        • Valente J.
        • et al.
        Automatic detection of small bowel tumors in wireless capsule endoscopy images using ensemble learning.
        Med Phys. 2019; 47: 52-63
        • Iakovidis D.K.
        • Koulaouzidis A.
        Automatic lesion detection in capsule endoscopy based on color saliency: closer to an essential adjunct for reviewing software.
        Gastrointest Endosc. 2014; 80: 877-883
        • Al-Shebani Q.
        • Premaratne P.
        • McAndrew D.J.
        • et al.
        A frame reduction system based on a color structural similarity (CSS) method and Bayer images analysis for capsule endoscopy.
        Artif Intell Med. 2019; 94: 18-27
        • Ghosh T.
        • Fattah S.A.
        • Wahid K.A.
        CHOBS: color histogram of block statistics for automatic bleeding detection in wireless capsule endoscopy video.
        IEEE J Transl Eng Health Med. 2018; 6: 1800112
        • Hwang Y.
        • Park J.
        • Lim Y.J.
        • et al.
        Application of artificial intelligence in capsule endoscopy: where are we now?.
        Clin Endosc. 2018; 51: 547-551
        • Alaskar H.
        • Hussain A.
        • Al-Aseem N.
        • et al.
        Application of convolutional neural networks for automated ulcer detection in wireless capsule endoscopy images.
        Sensors (Basel). 2019; 19: E1265
        • Yuan Y.
        • Meng M.Q.
        Deep learning for polyp recognition in wireless capsule endoscopy images.
        Med Phys. 2017; 44: 1379-1389