Advertisement

Deep learning–based pancreas segmentation and station recognition system in EUS: development and validation of a useful training tool (with video)

  • Jun Zhang
    Affiliations
    Department of Gastroenterology, Renmin Hospital of Wuhan University, Wuhan, China

    Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China

    Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China
    Search for articles by this author
  • Liangru Zhu
    Affiliations
    Department of Gastroenterology, Wuhan Union Hospital, Huazhong University of Science and Technology, Wuhan, China
    Search for articles by this author
  • Liwen Yao
    Affiliations
    Department of Gastroenterology, Renmin Hospital of Wuhan University, Wuhan, China

    Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China

    Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China
    Search for articles by this author
  • Xiangwu Ding
    Affiliations
    Department of Gastroenterology, Wuhan Puai Hospital, Wuhan, China
    Search for articles by this author
  • Di Chen
    Affiliations
    Department of Gastroenterology, Renmin Hospital of Wuhan University, Wuhan, China

    Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China

    Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China
    Search for articles by this author
  • Huiling Wu
    Affiliations
    Department of Gastroenterology, Renmin Hospital of Wuhan University, Wuhan, China

    Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China

    Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China
    Search for articles by this author
  • Zihua Lu
    Affiliations
    Department of Gastroenterology, Renmin Hospital of Wuhan University, Wuhan, China

    Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China

    Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China
    Search for articles by this author
  • Wei Zhou
    Affiliations
    Department of Gastroenterology, Renmin Hospital of Wuhan University, Wuhan, China

    Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China

    Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China
    Search for articles by this author
  • Lihui Zhang
    Affiliations
    Department of Gastroenterology, Renmin Hospital of Wuhan University, Wuhan, China

    Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China

    Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China
    Search for articles by this author
  • Ping An
    Affiliations
    Department of Gastroenterology, Renmin Hospital of Wuhan University, Wuhan, China

    Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China

    Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China
    Search for articles by this author
  • Bo Xu
    Affiliations
    Department of Gastroenterology, Wuhan Puai Hospital, Wuhan, China
    Search for articles by this author
  • Wei Tan
    Affiliations
    Department of Gastroenterology, Renmin Hospital of Wuhan University, Wuhan, China

    Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China

    Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China
    Search for articles by this author
  • Shan Hu
    Affiliations
    Research and Development Department, Wuhan EndoAngel Medical Technology Company, Wuhan, China
    Search for articles by this author
  • Fan Cheng
    Affiliations
    Department of Surgery, Renmin Hospital of Wuhan University, Wuhan, China
    Search for articles by this author
  • Honggang Yu
    Correspondence
    Reprint requests: Honggang Yu, Department of Gastroenterology, Renmin Hospital of Wuhan University, 99 Zhangzhidong Rd, Wuhan 430060, Hubei Province, China.
    Affiliations
    Department of Gastroenterology, Renmin Hospital of Wuhan University, Wuhan, China

    Key Laboratory of Hubei Province for Digestive System Disease, Renmin Hospital of Wuhan University, Wuhan, China

    Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision, Renmin Hospital of Wuhan University, Wuhan, China
    Search for articles by this author

      Background and Aims

      EUS is considered one of the most sensitive modalities for pancreatic cancer detection, but it is highly operator-dependent and the learning curve is steep. In this study, we constructed a system named BP MASTER (pancreaticobiliary master) for EUS training and quality control.

      Methods

      The standard procedure of pancreatic EUS was divided into 6 stations. We developed a station classification model and a pancreas/abdominal aorta/portal confluence segmentation model with 19,486 images and 2207 images, respectively. Then, we used 1920 images and 700 images for classification and segmentation internal validation, respectively. To test station recognition we used 396 videos clips. An independent data set containing 180 images was applied for comparing the performance between models and EUS experts. Seven hundred sixty-eight images from 2 other hospitals were used for external validation. A crossover study was conducted to test the system effect on reducing difficulty in ultrasonographics interpretation among trainees.

      Results

      The models achieved 94.2% accuracy in station classification and .836 dice in segmentation at internal validation. At external validation, the models achieved 82.4% accuracy in station classification and .715 dice in segmentation. For the video test, the station classification model achieved a per-frame accuracy of 86.2%. Compared with EUS experts, the models achieved 90.0% accuracy in classification and .77 and .813 dice in blood vessel and pancreas segmentation, which is comparable with that of experts. In the crossover study, trainee station recognition accuracy improved from 67.2% to 78.4% (95% confidence interval, .058-1.663; P < .01).

      Conclusions

      The BP MASTER system has the potential to play an important role in shortening the pancreatic EUS learning curve and improving EUS quality control in the future.

      Graphical abstract

      Abbreviations:

      BP MASTER (pancreaticobiliary master), DCNN (deep convolutional neural network), GPU (graphics processing unit), IoU (intersection over union)
      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'

      Subscribe:

      Subscribe to Gastrointestinal Endoscopy
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect

      References

        • Henrikson N.B.
        • Aiello Bowles E.J.
        • Blasi P.R.
        • et al.
        Screening for pancreatic cancer: updated evidence report and systematic review for the US Preventive Services Task Force.
        JAMA. 2019; 322: 445-454
        • Goggins M.
        • Overbeek K.A.
        • Brand R.
        • et al.
        Management of patients with increased risk for familial pancreatic cancer: updated recommendations from the International Cancer of the Pancreas Screening (CAPS) Consortium.
        Gut. 2020; 69: 7-17
        • Canto M.I.
        • Hruban R.H.
        • Fishman E.K.
        • et al.
        Frequent detection of pancreatic lesions in asymptomatic high-risk individuals.
        Gastroenterology. 2012; 142: 796-804
        • Harinck F.
        • Konings I.C.A.W.
        • Kluijt I.
        • et al.
        A multicentre comparative prospective blinded analysis of EUS and MRI for screening of pancreatic cancer in high-risk individuals.
        Gut. 2016; 65: 1505-1513
        • Wani S.
        • Keswani R.N.
        • Petersen B.
        • et al.
        Training in EUS and ERCP: standardizing methods to assess competence.
        Gastrointest Endosc. 2018; 87: 1371-1382
        • Committee E.F.S.
        • Yamao K.
        • Irisawa A.
        • et al.
        Standard imaging techniques of endoscopic ultrasound-guided fine-needle aspiration using a curved linear array echoendoscope.
        Dig Endosc. 2007; 19: S180-S205
        • Irisawa A.
        • Yamao K.
        Curved linear array EUS technique in the pancreas and biliary tree: focusing on the stations.
        Gastrointest Endosc. 2009; 69: S84-S89
        • Cho C.M.
        Training in endoscopy: endoscopic ultrasound.
        Clin Endosc. 2017; 50: 340-344
        • Schmidhuber J.
        Deep learning in neural networks: an overview.
        Neural Netw. 2015; 61: 85-117
        • Wu L.
        • Zhang J.
        • Zhou W.
        • et al.
        Randomised controlled trial of WISENSE, a real-time quality improving system for monitoring blind spots during esophagogastroduodenoscopy.
        Gut. 2019; 68: 2161-2169
        • Gong D.
        • Wu L.
        • Zhang J.
        • et al.
        Detection of colorectal adenomas with a real-time computer-aided system (ENDOANGEL): a randomised controlled study.
        Lancet Gastroenterol Hepatol. Epub 2020 Jan 22;
        • Săftoiu A.
        • Vilmann P.
        • Dietrich C.F.
        • et al.
        Quantitative contrast-enhanced harmonic EUS in differential diagnosis of focal pancreatic masses (with videos).
        Gastrointest Endosc. 2015; 82: 59-69
        • Kuwahara T.
        • Hara K.
        • Mizuno N.
        • et al.
        Usefulness of deep learning analysis for the diagnosis of malignancy in intraductal papillary mucinous neoplasms of the pancreas.
        Clin Translat Gastroenterol. 2019; 10: 1-8
        • Ginès A.
        • Lightdale C.J.
        How I do a diagnostic EUS.
        Endoscopy. 2019; 51: 973-975
        • Palazzo L.
        How to perform EUS in the pancreaticobiliary area.
        Min Med. 2014; 105: 371-389
        • Giovannini M.
        Normal linear echoanatomy.
        Techn Gastrointest Endosc. 2000; 3: 124-135
        • Shao L.
        • Zhu F.
        • Li X.
        Transfer learning for visual categorization: a survey.
        IEEE Trans Neural Netw Learn Syst. 2014; 26: 1019-1034
      1. Abadi M, Barham P, Chen J, et al. Tensorflow: a system for large-scale machine learning. In 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI ’16). Available at: https://www.usenix.org/system/files/conference/osdi16/osdi16-abadi.pdf. Accessed July 13, 2020.

        • Baldi P.
        • Sadowski P.
        The dropout learning algorithm.
        Artific Intell. 2014; 210: 78-122
        • Prechelt L.
        Automatic early stopping using cross validation: quantifying the criteria.
        Neural Netw. 1998; 11: 761-767
        • Liaw A.
        • Wiener M.
        Classification and regression by random forest.
        R News. 2002; 2: 18-22
        • Wani S.
        • Keswani R.
        • Hall M.
        • et al.
        A prospective multicenter study evaluating learning curves and competence in endoscopic ultrasound and endoscopic retrograde cholangiopancreatography among advanced endoscopy trainees: the Rapid Assessment of Trainee Endoscopy Skills study.
        Clin Gastroenterol Hepatol. 2017; 15: 1758
        • Wani S.
        • Keswani R.N.
        • Han S.
        • et al.
        Competence in endoscopic ultrasound and endoscopic retrograde cholangiopancreatography, from training through independent practice.
        Gastroenterology. 2018; 155: 1483-1494
        • Wong J.Y.
        • Kongkam P.
        • Ho K.Y.
        Training in endoscopic ultrasonography: an Asian perspective.
        Digestive Endoscopy. 2017; 29: 512-516
        • Wani S.
        • Hall M.
        • Keswani R.N.
        • et al.
        Variation in aptitude of trainees in endoscopic ultrasonography, based on cumulative sum analysis.
        Clin Gastroenterol Hepatol. 2015; 13: 1318-1325
        • Wani S.
        • Han S.
        • Simon V.
        • et al.
        Setting minimum standards for training in EUS and ERCP: results from a prospective multicenter study evaluating learning curves and competence among advanced endoscopy trainees.
        Gastrointest Endosc. 2019; 89: 1160-1168.e9

      Linked Article

      • Erratum
        Gastrointestinal EndoscopyVol. 93Issue 3
        • Preview
          In the article, “Deep learning–based pancreas segmentation and station recognition system in EUS: development and validation of a useful training tool (with video),” which was published in the October 2020 issue (Gastrointest Endosc 2020;92:874-85), the following authors contributed equally to the article:
        • Full-Text
        • PDF