Advertisement

Artificial intelligence for polyp characterization: Don’t save on your competence!

      Abbreviations:

      AI (artificial intelligence), CRC (colorectal cancer)
      Colonoscopy screening comes with a significant cost. If you sum up the costs due to health care personnel, endoscopy technology, facilities, and histopathology, colonoscopy is among the most expensive diagnostic procedures, with estimates comparable with whole-body CT or magnetic resonance imaging. When these costs are projected at population level, the absolute magnitude is exceptional, inasmuch as it is estimated to correspond to an annual gross expenditure of more than U.S. $775 million.
      • Mori Y.
      • Kudo S.-e.
      • East J.E.
      • et al.
      Cost savings in colonoscopy with artificial intelligence-aided polyp diagnosis: an add-on analysis of a clinical trial (with video).
      It could be argued that this is unfair. Colonoscopy represents one of the most effective preventive interventions. To prevent or downstage colorectal cancer (CRC) also means to save money, namely, the expenses of those involved in surgery, chemoradiotherapy, and palliative treatment. Isn’t it true that all the savings in CRC-related costs are enough to offset all the direct and indirect cost of screening colonoscopy?
      • Zhong G.-C.
      • Sun W.-P.
      • Wan L.
      • et al.
      Efficacy and cost-effectiveness of fecal immunochemical test versus colonoscopy in colorectal cancer screening: a systematic review and meta-analysis.
      Unexpectedly, what appeared to be a costly procedure is suddenly converted to one of the most cost-effective preventive strategies!
      There is, however, a temporal window between the costs and the savings. Although costs are to be paid now, the savings will be delayed by 10 to 20 years because of the slow natural history of the adenoma-carcinoma sequence. Costs are certain, but the savings may be shadowed by unexpected factors, such as death by competing causes and/or incidence of postcolonoscopy CRC resulting from a missed lesion or incomplete resection. This is why the long-term benefit of colonoscopy must be discounted because “a bird in the hand is worth 2 in the bush.”
      Optical diagnosis of diminutive polyps remains by far the most attractive intervention for an immediate saving on the costs of screening colonoscopy. Because of their high prevalence, these lesions disproportionately account for most of the histopathology costs, representing nearly 10% of the whole colonoscopy cost in the United States.
      • Mori Y.
      • Kudo S.-e.
      • East J.E.
      • et al.
      Cost savings in colonoscopy with artificial intelligence-aided polyp diagnosis: an add-on analysis of a clinical trial (with video).
      In addition, the pathologic relevance is questionable inasmuch as it is mostly represented by either nonadvanced adenomas or indolent hyperplastic polyps. In addition, the role of pathology as reference standard has been questioned because of the possibility of inadequate orientation or insufficient sampling of the lesion.
      • Shahidi N.
      • Rex D.K.
      • Kaltenbach T.
      • et al.
      Use of endoscopic impression, artificial intelligence, and pathologist interpretation to resolve discrepancies between endoscopy and pathology analyses of diminutive colorectal polyps.
      Endoscopy classifications, based on the use of blue-light imaging, showed a high accuracy in the in-vivo prediction of histologic diagnosis, generating cost-saving strategies like “leave in situ” and “resect and discard.” Disappointingly, their implementation in a community setting failed because of their lower than expected accuracy and excessive interoperator variability.
      • Wang L.M.
      • East J.E.
      Diminutive polyp cancers and the DISCARD strategy: Much ado about nothing or the end of the affair?.
      Will artificial intelligence (AI) revitalize the expected savings from optical diagnosis? By modeling the results of a breakthrough trial, Mori et al
      • Mori Y.
      • Kudo S.-e.
      • East J.E.
      • et al.
      Cost savings in colonoscopy with artificial intelligence-aided polyp diagnosis: an add-on analysis of a clinical trial (with video).
      estimated in $85 million the potential annual savings of using the high accuracy of EndoBRAIN, a machine-learning system compliant with the criteria of Preservation and Incorporation of Valuable Endoscopic Innovation II, to implement a leave-in-situ strategy for rectosigmoid hyperplastic polyps ≤5 mm.
      • Mori Y.
      • Kudo S.-e.
      • East J.E.
      • et al.
      Cost savings in colonoscopy with artificial intelligence-aided polyp diagnosis: an add-on analysis of a clinical trial (with video).
      The value of this estimate is in the new opportunity to replicate an adequate standard of optical diagnosis at the community level, irrespectively of the level of experience. This was nicely shown in the original trial, where the performance of EndoBRAIN was similar between expert and nonexpert endoscopists. The fact that Mori et al
      • Mori Y.
      • Kudo S.-e.
      • East J.E.
      • et al.
      Cost savings in colonoscopy with artificial intelligence-aided polyp diagnosis: an add-on analysis of a clinical trial (with video).
      limited their financial analysis to the leave-in-situ strategy also increases the reliability of their estimate, because it is by far more endoscopist friendly than the resect-and-discard strategy. Third, when the $125 saving per colonoscopy is combined with a negative predictive value of 95.2%, the tradeoff would be a convincing saving of over $5000 per 1 adenoma misclassified as hyperplastic.
      • Mori Y.
      • Kudo S.-e.
      • East J.E.
      • et al.
      Cost savings in colonoscopy with artificial intelligence-aided polyp diagnosis: an add-on analysis of a clinical trial (with video).
      Will such savings be offset by the costs of AI implementation? In comparison with previous innovations in the field of optical diagnosis, such as advanced endoscopy imaging or magnification, the estimated AI cost of $10 per colonoscopy is negligible. This was somewhat expected when it is considered that AI systems are simple real-time software that do not require any adaptations of the original endoscopic image. Of note, the initial investment in AI appeared to be completely compensated by its savings in pathology costs, further supporting a widespread implementation of such technology.
      • Mori Y.
      • Kudo S.-e.
      • East J.E.
      • et al.
      Cost savings in colonoscopy with artificial intelligence-aided polyp diagnosis: an add-on analysis of a clinical trial (with video).
      It could be argued that the limited availability of endocytoscope, adopted by Mori et al,
      • Mori Y.
      • Kudo S.-e.
      • East J.E.
      • et al.
      Cost savings in colonoscopy with artificial intelligence-aided polyp diagnosis: an add-on analysis of a clinical trial (with video).
      would prevent the generalizability of such an estimate. However, as is also fairly suggested by the same authors, accuracy values similar to those shown by EndoBRAIN have been reproduced with deep learning systems that are easily adapted to nonmagnified endoscopes.
      • Byrne M.F.
      • Chapados N.
      • Soudan F.
      • et al.
      Real-time differentiation of adenomatous and hyperplastic diminutive colorectal polyps during analysis of unaltered videos of standard colonoscopy using a deep learning model.
      Will AI-triggered savings suddenly eliminate all the barriers preventing optical diagnosis to be used in clinical practice? Differently from detection, the use of AI for polyp characterization implies a higher cognitive skill possessed by the endoscopist. In the case of detection, the endoscopist, irrespective of the level of expertise, will consistently confirm or not the lesion flagged by AI with a high level of confidence, as simply related to a morphologic or color change of the mucosa. By contrast, only an endoscopist competent in optical diagnosis will be able to accept or refuse the AI characterization with a high level of confidence based on a complex gestaltic analysis of the surface and vessel features. Nonexpert endoscopists will passively exploit the AI prediction without any chance to challenge it with a high degree of confidence, generating the risk of a higher level of automation of the diagnostic process. If the choice is between a diagnosis by an expert pathologist and an automatic AI prediction merely certified by an inexpert endoscopist, health systems will remain reluctant to implement the leave-in-situ strategy, irrespective of the magnitude of the financial savings. Who will be finally responsible for an incorrect diagnosis: the endoscopist, the software developer, the health systems, or all of them together?
      Only a rigorously sequential algorithm between AI prediction and endoscopist’s confidence may fasten its incorporation into clinical practice. The leave-in-situ strategy should be limited to polyps that are both predicted as hyperplastic by AI and confirmed with a high level of confidence by competent endoscopists. In the case of any discrepancy or low level of confidence, histologic analysis should be required. This argues against using AI accuracy to bypass a suboptimal competence in optical diagnosis. The formal recommendations that “the optical diagnosis…can be performed only by experienced endoscopists who are adequately trained and audited” is not challenged by AI.
      • Bisschops R.
      • East J.E.
      • Hassan C.
      • et al.
      Advanced imaging for detection and differentiation of colorectal neoplasia: European Society of Gastrointestinal Endoscopy (ESGE) Guideline - Update 2019.
      Competence in optical diagnosis, acquired with a structured curriculum,
      • Bisschops R.
      • East J.E.
      • Hassan C.
      • et al.
      Advanced imaging for detection and differentiation of colorectal neoplasia: European Society of Gastrointestinal Endoscopy (ESGE) Guideline - Update 2019.
      is the prerequisite, not the final outcome of AI implementation!
      In conclusion, the breakthrough clinical benefit coming from AI-based polyp detection and characterization may be expected to be fully supported by the saving in useless polypectomies, pathologic examinations, and prevention of postcolonoscopy CRC. However, this must trigger a timely improvement of the endoscopists’ standard, especially in polyp characterization, because what can be self-paying will never be accepted to be also self-driving.

      Disclosure

      Dr Hassan and Dr Repici are consultants for Fujifilm and Medtronic. The other author disclosed no financial relationships.

      References

        • Mori Y.
        • Kudo S.-e.
        • East J.E.
        • et al.
        Cost savings in colonoscopy with artificial intelligence-aided polyp diagnosis: an add-on analysis of a clinical trial (with video).
        Gastrointest Endosc. 2020; 92: 905-911.e1
        • Zhong G.-C.
        • Sun W.-P.
        • Wan L.
        • et al.
        Efficacy and cost-effectiveness of fecal immunochemical test versus colonoscopy in colorectal cancer screening: a systematic review and meta-analysis.
        Gastrointest Endosc. 2020; 91: 684-697.e15
        • Shahidi N.
        • Rex D.K.
        • Kaltenbach T.
        • et al.
        Use of endoscopic impression, artificial intelligence, and pathologist interpretation to resolve discrepancies between endoscopy and pathology analyses of diminutive colorectal polyps.
        Gastroenterology. 2020; 158: 783-785.e1
        • Wang L.M.
        • East J.E.
        Diminutive polyp cancers and the DISCARD strategy: Much ado about nothing or the end of the affair?.
        Gastrointest Endosc. 2015; 82: 385-388
        • Byrne M.F.
        • Chapados N.
        • Soudan F.
        • et al.
        Real-time differentiation of adenomatous and hyperplastic diminutive colorectal polyps during analysis of unaltered videos of standard colonoscopy using a deep learning model.
        Gut. 2019; 68: 94-100
        • Bisschops R.
        • East J.E.
        • Hassan C.
        • et al.
        Advanced imaging for detection and differentiation of colorectal neoplasia: European Society of Gastrointestinal Endoscopy (ESGE) Guideline - Update 2019.
        Endoscopy. 2019; 51: 1155-1179

      Linked Article