Advertisement

The exceptional performance of deep learning for capsule endoscopy: Will such quality be maintained in clinical scenarios?

      Abbreviations:

      CE (capsule endoscopy), CI (confidence interval), CNN (convolutional neural network)
      Capsule endoscopy (CE) has revolutionized the investigation of various small-bowel abnormalities. About 10 devices fabricated by 5 companies are commercially available. As the capsule travels through the GI tract, thousands of pictures are automatically captured and transmitted to a recorder. Although the procedure is minimally invasive, physicians must review over 10,000 images per patient, which is obviously very time consuming. Physicians also fear the risk of oversight; any abnormality may be evident in only a few frames. Computer-aided diagnosis would be extremely helpful.
      Despite several attempts, reliability remains unacceptable. For example, the QuickView mode, originally used to remove uninformative CE images of the PillCam system, is not often used in practice because of unacceptably high miss rates for noteworthy abnormalities. Systems based on conventional machine-learning methods (eg, support vector machines, neural networks, or binary classifiers) are not yet commercially available, probably because they remain inaccurate. The difficulties include the relatively poor image quality caused by inaccurate focusing; light limitations; low resolution; the presence of bile, debris, and bubbles; and the fact that various types of abnormalities must be detected. Recently, state-of-the-art deep learning–based methods have significantly improved recognition performance in various medical fields and are expected to resolve the abovementioned problems in CE reading. Convolutional neural networks (CNNs) lead the field.
      In this systematic review with a meta-analysis in Gastrointestinal Endoscopy, Mohan et al
      • Mohan B.P.
      • Khan S.R.
      • Kassab L.L.
      • et al.
      High pooled performance of convolutional neural networks in computer-aided diagnosis of GI ulcers and/or hemorrhage on wireless capsule endoscopy images: a systematic review and meta-analysis.
      report a high pooled performance of CNN-based systems in terms of computer-aided diagnosis of GI ulcers and/or hemorrhage on CE images. From the 9 studies included in the final analysis, 20, 24, 23, 9, and 9 datasets were extracted to calculate accuracy, sensitivity, specificity, the positive predictive value, and the negative predictive value, respectively. All studies were retrospective in nature. The CNN systems exhibited a pooled accuracy of 95.4% (95% confidence interval [CI], 94.3-96.3), a sensitivity of 95.5% (95% CI, 94-96.5), a specificity of 95.8% (95% CI, 94.7-96.6), a positive predictive value of 95.8% (95% CI, 90.5-98.2), and a negative predictive value of 96.8% (95% CI, 94.9-98.1). The quality of the evidence seemed to be robust (heterogeneity was minimal) except for the positive predictive value. A strength of this study is that strict selection criteria were applied. Studies using non–CNN-based machine-learning algorithms and nonclinical redundant studies on the mathematic development and/or fine tuning of CNN algorithms were excluded. Therefore, the datasets were of high quality; expert endoscopists had confirmed the ground truths in most studies. Thus, the meta-analysis focused on CNN systems, and potential bias was reduced. Although this review found that CNN-based systems seemed promising in terms of CE diagnosis of GI ulcers and/or hemorrhage, the data must be interpreted with caution, as the authors indeed noted.
      First, the various types of small-bowel abnormalities were not adequately covered. The abstracted data pertained to ulcers, nonbleeding angioectasias, and hemorrhages identified on CE images. Although these are the most common small-bowel abnormalities, CE can also identify other serious findings, including protruding lesions such as small-bowel adenocarcinomas, lymphomas, and polyps. Recently, Saito et al
      • Saito H.
      • Aoki T.
      • Aoyama K.
      • et al.
      Automatic detection and classification of protruding lesions in wireless capsule endoscopy images based on a deep convolutional neural network.
      reported that a CNN system performed well when used to detect various protruding lesions in CE images that had been categorized by expert endoscopists using the capsule endoscopy structured terminology classification system.
      • Korman L.Y.
      • Delvaux M.
      • Gay G.
      • et al.
      Capsule endoscopy structured terminology (CEST): proposal of a standardized and structured terminology for reporting capsule endoscopy procedures.
      Second, most studies in this review article evaluated CNN performance under experimental conditions, thus, not in real-life clinical scenarios. In other words, the meta-analysis was based on only selected images, not full-length videos. Evaluation of videos would afford some advantages: (1) The CNN threshold could be optimized. The thresholds used to report sensitivity and specificity vary; image-based studies cannot optimize these for real-world CE reading; and (2) the clinical utility of a CNN system could be directly compared with that of other software. The cited authors mention that few CNN models have been compared with existing selection tools such as the QuickView mode, which is mounted on the RAPID reader software. We agree that comparative studies are needed; CNN systems are not necessarily better than existing tools in practice, even if such systems exhibit excellent performance at the level of selected images.
      Only a few studies have investigated the clinical utilities of CNN systems that explore full-length videos. To the best of our knowledge, 3 reports have discussed patient-level videos. Ding et al
      • Ding Z.
      • Shi H.
      • Zhang H.
      • et al.
      Gastroenterologist-level identification of small bowel diseases and normal variants by capsule endoscopy using a deep-learning model.
      and Aoki et al
      • Aoki T.
      • Yamada A.
      • Aoyama K.
      • et al.
      Clinical usefulness of a deep learning-based system as the first screening on small-bowel capsule endoscopy reading.
      compared 2 reading methods (endoscopist-alone and endoscopist’s evaluation after the first CNN screen). Both reports found that endoscopists’ reading time was reduced by the CNN-based systems without compromising the abnormality detection rates. Another recent report from Aoki et al

      Aoki T, Yamada A, Kato Y, et al. Automatic detection of various abnormalities in capsule endoscopy videos by a deep learning-based system: a multicenter study. Gastrointest Endosc. Epub 2020 May 15.

      compared the QuickView mode with a CNN system in terms of the detectability of various abnormalities; the CNN system was significantly better. The difference could be intriguing between the TOP 100 tool, which is mounted on the latest RAPID reader software (version 9.0) and CNN systems but has not yet been evaluated. Despite the promising results of these 3 reports, the limitations include small numbers of cases, the few CE systems tested, and/or a rather high false positive rate.
      Again, all published studies on CE-CNN systems were retrospective; we completely agree with the cited authors that prospective studies are essential. However, certain challenges are apparent. Many institutions must be prepared to spend time evaluating certain critical but rare conditions, including small-bowel cancers and lymphomas.
      This systematic review covered several types of CNN algorithms but did not compare them. A recent report in a clinical journal by Otani et al
      • Otani K.
      • Nakada A.
      • Kurose Y.
      • et al.
      Automatic detection of different types of small-bowel lesions on capsule endoscopy images using a newly developed deep convolutional neural network.
      revealed that a new CNN algorithm outperformed an earlier version; significant improvements to CNN systems thus remain possible.
      Some research groups have conducted their unique attempts. Yiftach et al

      Barash Y, Azaria L, Soffer S, et al. Ulcer severity grading in video-capsule images of Crohn's disease patients: an ordinal neural network solution. Gastrointest Endosc. Epub 2020 Jun 12.

      used a CNN system to perform detailed evaluations of patients with Crohn’s disease, not only to detect ulcers automatically. In an impressive article,

      Barash Y, Azaria L, Soffer S, et al. Ulcer severity grading in video-capsule images of Crohn's disease patients: an ordinal neural network solution. Gastrointest Endosc. Epub 2020 Jun 12.

      they developed a CNN system for the “automated severity grading” of Crohn’s disease ulcers evident on CE, although they evaluated only individual images (thus, not the entire length of the small bowel). A European group built a CE database termed computer-assisted diagnosis for capsule endoscopy
      • Leenhardt R.
      • Li C.
      • Le Mouel J.P.
      • et al.
      CAD-CAP: A 25,000-image database serving the development of artificial intelligence for capsule endoscopy.
      ; an international consensus in terms of nomenclature and semantic descriptions of small-bowel lesions has been achieved.
      • Leenhardt R.
      • Li C.
      • Koulaouzidis A.
      • et al.
      Nomenclature and semantic description of vascular lesions in small bowel capsule endoscopy: an international Delphi consensus statement.
      Such work will certainly widen the appeal of CNN systems.
      In summary, the cited systematic review of retrospective image-based studies showed that the pooled performance of CNN systems detecting ulcers and/or hemorrhage on CE images was high. The systems remain at the research stage but apparently show potential for daily use. We must consider how to use them and whether they can be trusted. A meta-analysis of works evaluating full-length videos is urgently needed.

      Disclosure

      All authors disclosed no financial relationships.

      References

        • Mohan B.P.
        • Khan S.R.
        • Kassab L.L.
        • et al.
        High pooled performance of convolutional neural networks in computer-aided diagnosis of GI ulcers and/or hemorrhage on wireless capsule endoscopy images: a systematic review and meta-analysis.
        Gastrointest Endosc. 2021; 93: 356-364
        • Saito H.
        • Aoki T.
        • Aoyama K.
        • et al.
        Automatic detection and classification of protruding lesions in wireless capsule endoscopy images based on a deep convolutional neural network.
        Gastrointest Endosc. 2020; 92: 144-151.e1
        • Korman L.Y.
        • Delvaux M.
        • Gay G.
        • et al.
        Capsule endoscopy structured terminology (CEST): proposal of a standardized and structured terminology for reporting capsule endoscopy procedures.
        Endoscopy. 2005; 37: 951-959
        • Ding Z.
        • Shi H.
        • Zhang H.
        • et al.
        Gastroenterologist-level identification of small bowel diseases and normal variants by capsule endoscopy using a deep-learning model.
        Gastroenterology. 2019; 157: 1044-1054.e5
        • Aoki T.
        • Yamada A.
        • Aoyama K.
        • et al.
        Clinical usefulness of a deep learning-based system as the first screening on small-bowel capsule endoscopy reading.
        Dig Endosc. 2020; 32: 585-591
      1. Aoki T, Yamada A, Kato Y, et al. Automatic detection of various abnormalities in capsule endoscopy videos by a deep learning-based system: a multicenter study. Gastrointest Endosc. Epub 2020 May 15.

        • Otani K.
        • Nakada A.
        • Kurose Y.
        • et al.
        Automatic detection of different types of small-bowel lesions on capsule endoscopy images using a newly developed deep convolutional neural network.
        Endoscopy. 2020; 52: 786-791
      2. Barash Y, Azaria L, Soffer S, et al. Ulcer severity grading in video-capsule images of Crohn's disease patients: an ordinal neural network solution. Gastrointest Endosc. Epub 2020 Jun 12.

        • Leenhardt R.
        • Li C.
        • Le Mouel J.P.
        • et al.
        CAD-CAP: A 25,000-image database serving the development of artificial intelligence for capsule endoscopy.
        Endosc Int Open. 2020; 8: E415-E420
        • Leenhardt R.
        • Li C.
        • Koulaouzidis A.
        • et al.
        Nomenclature and semantic description of vascular lesions in small bowel capsule endoscopy: an international Delphi consensus statement.
        Endosc Int Open. 2019; 7: E372-E379

      Linked Article