Advertisement

Regulatory considerations for artificial intelligence technologies in GI endoscopy

      Artificial intelligence (AI) technologies in clinical medicine have become the subject of intensive investigative efforts and popular attention. In domains ranging from pathology to radiology, AI has demonstrated the potential to improve clinical performance and efficiency. In gastroenterology, AI has been applied on multiple fronts, with particular progress seen in the areas of computer-aided polyp detection (CADe) and computer-aided polyp diagnosis (CADx), to assist gastroenterologists during colonoscopy. As clinical evidence accrues for CADe and CADx, our attention must also turn toward the unique challenges that this new wave of technologies represent for the U.S. Food and Drug Administration and other regulatory agencies, who are tasked with protecting public health by ensuring the safety of medical devices. In this review, we describe the current regulatory pathways for AI tools in gastroenterology and the expected evolution of these pathways.

      Abbreviations:

      AI (artificial intelligence), CADe (computer-aided detection), CADx (computer-aided diagnosis), FDA (U.S. Food and Drug Administration), IMDRF (International Medical Device Regulators Forum), Pre-Cert (Precertification Pilot program), SaMD (software as a medical device)
      Artificial intelligence (AI), which refers to computer technology designed to mimic human intelligence, has emerged as a promising tool for tackling a wide variety of challenges in clinical medicine.
      • Topol E.J.
      High-performance medicine: the convergence of human and artificial intelligence.
      A subset of AI is machine learning, a technique in which computers use data to perform tasks without explicit instruction. In the last 5 years, numerous applications of AI have been developed and approved for clinical medicine, including identification of diabetic retinopathy and diagnosis of cutaneous malignancy.
      • Ting D.S.W.
      • Cheung C.Y.-L.
      • Lim G.
      • et al.
      Development and validation of a deep learning system for diabetic retinopathy and related eye diseases using retinal images from multiethnic populations with diabetes.
      ,
      • Esteva A.
      • Kuprel B.
      • Novoa R.A.
      • et al.
      Dermatologist-level classification of skin cancer with deep neural networks.
      AI technologies have proven especially promising in improving clinical performance during GI endoscopy. Over the past decade, numerous groups have developed novel computer-aided polyp detection (CADe) and computer-aided polyp diagnosis (CADx) systems, using endoscopic still images and video data.
      • Byrne M.F.
      • Chapados N.
      • Soudan F.
      • et al.
      Real-time differentiation of adenomatous and hyperplastic diminutive colorectal polyps during analysis of unaltered videos of standard colonoscopy using a deep learning model.
      • Repici A.
      • Hassan C.
      • Radaelli F.
      • et al.
      Accuracy of narrow-band imaging in predicting colonoscopy surveillance intervals and histology of distal diminutive polyps: results from a multicenter, prospective trial.
      • Misawa M.
      • Kudo S.-E.
      • Mori Y.
      • et al.
      Artificial intelligence-assisted polyp detection for colonoscopy: initial experience.
      • Urban G.
      • Tripathi P.
      • Alkayali T.
      • et al.
      Deep learning localizes and identifies polyps in real time with 96% accuracy in screening colonoscopy.
      In the past 3 years, several prospective clinical trials examining CADx and CADe systems have demonstrated the real-world feasibility of these technologies. In 2018, Mori et al
      • Mori Y.
      • Kudo S.-E.
      • Misawa M.
      • et al.
      Real-time use of artificial intelligence in identification of diminutive polyps during colonoscopy: a prospective study.
      performed a single-center, open-label prospective study examining the performance of a deep learning CADx system, Endobrain, to accurately diagnose diminutive rectosigmoid polyps. The EndoBrain CADx technology subsequently received regulatory approval by the Pharmaceuticals and Medical Devices Agency, a regulatory body in Japan.
      Cybernet Systems Co, Ltd
      EndoBRAIN—artificial intelligence system that supports optical diagnosis of colorectal polyps—was approved by PMDA (Pharmaceuticals and Medical Devices Agency), a regulatory body in Japan.
      Soon after, Wang et al
      • Wang P.
      • Berzin T.M.
      • Brown J.R.G.
      • et al.
      Real-time automatic detection system increases colonoscopic polyp and adenoma detection rates: a prospective randomised controlled study.
      published the first randomized clinical trial examining the role of CADe in colonoscopy, which demonstrated a significant increase in adenoma detection rate (29.1% vs 20.3%, P < .001), largely because of increased detection of diminutive adenomas. More recently Wang et al
      • Wang P.
      • Liu X.
      • Berzin T.M.
      • et al.
      Effect of a deep-learning computer-aided detection system on adenoma detection during colonoscopy (CADe-DB trial): a double-blind randomised study.
      published a double-blinded, randomized clinical trial demonstrating similar results. It is worth noting that current CADe literature has largely shown improved detection of diminutive adenomas, the clinical benefit of which remains an area of controversy.
      • Lieberman D.
      • Sullivan B.A.
      • Hauser E.R.
      • et al.
      Baseline colonoscopy findings associated with 10-year outcomes in a screening cohort undergoing colonoscopy surveillance.
      Larger CADe trials will be important to determine whether CADe can enhance detection of higher risk sessile lesions in the right-sided colon segment, which are less common but of higher clinical relevance.
      Additional randomized trials are underway evaluating a wide range of CADx and CADe technologies in GI endoscopy, and clear pathways are developing around how to produce high-quality research in this area.
      • Vinsard D.G.
      • Mori Y.
      • Misawa M.
      • et al.
      Quality assurance of computer-aided detection and diagnosis in colonoscopy.
      ,
      • Liu X.
      • Faes L.
      • Calvert M.J.
      • et al.
      Extension of the CONSORT and SPIRIT statements.
      As these technologies become commercially available, it will also be important to track practical outcomes including endoscopy efficiency and overall costs. These systems will likely require significant upfront costs, but if they maintain sufficient performance characteristics, they may allow for advances in diagnosis and treatment. For instance, the technologies could facilitate implementation of the “resect and discard” paradigm; the American Society for Gastrointestinal Endoscopy has proposed that if a technology can cross the diagnostic threshold of ≥90% negative predictive value for adenomatous histology, then small polyps may be resected and discarded without histologic confirmation.
      • Rex D.K.
      • Kahi C.
      • O’Brien M.
      • et al.
      The American Society for Gastrointestinal Endoscopy PIVI (Preservation and Incorporation of Valuable Endoscopic Innovations) on real-time endoscopic assessment of the histology of diminutive colorectal polyps.
      As the performance capabilities of AI for GI endoscopy continue to improve and evolve, there is a parallel need for evolution of regulatory pathways that address the unique opportunities and challenges of applying AI technologies to the clinical care of patients. To prepare for this future, we describe the current regulatory framework and anticipated modifications for AI technologies. In particular, we focus on the regulatory trajectories for CADx and CADe, the 2 AI applications with the most robust published clinical data for GI endoscopy.

      Approval process for AI technologies

      The U.S. Food and Drug Administration (FDA) has proposed that AI-based software be assessed along 2 primary axes. The first axis is focused on risk. The approach to categorizing algorithms from low to high risk is based on a framework established by the International Medical Device Regulators Forum (IMDRF), outlined in the next section below. The second axis relates to the spectrum between “locked” algorithms, which have fixed performance characteristics, and those designed to learn and evolve autonomously.

      Risk categorization

      The IMDRF, a collaborative group of international regulatory officials, has proposed 4 different risk categories for “software as a medical device” (SaMD), each with a different set of requirements for assessing the scientific and clinical validity of the technology.

      IMDRF SaMD Working Group. Software as a medical device (SaMD): key definitions. Available at: http://www.imdrf.org/docs/imdrf/final/technical/imdrf-tech-131209-samd-key-definitions-140901.pdf. Accessed March 26, 2020.

      ,
      IMDRF SaMD Working Group
      Software as a medical device (SaMD): clinical evaluation.
      The risk categories range from I.i to IV.i and are defined by both the significance of the information provided by the SaMD (eg, treat or diagnose, drive clinical management, inform clinical management) and the state of the condition it treats (eg, critical, serious, nonserious) (Table 1).
      IMDRF SaMD Working Group
      Software as a medical device (SaMD): clinical evaluation.
      Table 1SaMD categories, ranging from I.i to IV.i, are defined by the state of the healthcare situation and the significance of the information provided, as proposed by the IMDRF
      Adapted from documents produced by IMDRF SaMD Working Group.
      IMDRF SaMD Working Group
      Software as a medical device (SaMD): clinical evaluation.
      ,

      IMDRF SaMD Working Group. “Software as a medical device”: possible framework for risk categorization and corresponding considerations. Available at: http://www.imdrf.org/docs/imdrf/final/technical/imdrf-tech-140918-samd-framework-risk-categorization-141013.pdf. Accessed April 1, 2020.

      State of healthcare situation or conditionSignificance of information provided by SaMD to healthcare decision
      Treat or diagnoseDrive clinical managementInform clinical management
      CriticalIV.iIII.iII.i
      SaMD that analyzes cerebrospinal fluid spectroscopy to diagnose tuberculosis meningitisSaMD tracks and flags suspect skin lesions that have a higher likelihood to progress to cancerSaMD that allows user to send photos of skin lesions to their doctor
      SeriousIII.iiII.iiI.ii
      Radiation treatment planning system to be used in life-threatening oncologic diagnosesSaMD that analyzes heart rate data intended for a clinician as an aid in diagnosis of arrhythmiaSaMD that stores historical blood pressure information for a healthcare provider's later review
      NonseriousII.iiiI.iiiI.i
      SaMD that calculates a risk score for developing heart diseaseSaMD that analyzes radiographs to aid clinicians in the diagnosis of Osgood-Schlatter diseaseSaMD that analyzes images of the eye to guide diagnostic actions for astigmatism
      IMDRF, International Medical Device Regulators Forum; SaMD, software as a medical device.
      Within gastroenterology, CADe and CADx technologies for detecting and diagnosing colon polyps have not yet been classified. For both technologies, attention must be paid to the critical FDA distinction of whether a technology “drives” clinical management or merely “informs” clinical management. The term “inform clinical management” refers to software that can “inform of options for treating, diagnosing, preventing, or mitigating a disease or condition.”
      IMDRF SaMD Working Group
      Software as a medical device (SaMD): clinical evaluation.
      The phrase “drive clinical management” refers to software that may “aid in diagnosis by analyzing relevant information to help predict risk of a disease or condition or as an aid to making a definitive diagnosis.”
      IMDRF SaMD Working Group
      Software as a medical device (SaMD): clinical evaluation.
      We suspect that most CADe and CADx technologies will be considered technologies that “drive clinical management” and therefore will likely be assigned to IMDRF class I or II risk categories. Risk category III may be applied selectively to certain CADx technologies that aim to differentiate dysplasia from cancer. For instance, a technology designed to help guide decisions regarding endoscopic ablation versus endoscopic or surgical resection in dysplastic Barrett’s esophagus would likely be category III.

      Capacity for change

      Software technologies exist on a spectrum of adaptability. At one extreme are AI technologies, which have the ability to learn from data in real time. This feature of rapid and autonomous learning means that the clinical performance of these technologies, for instance the accuracy of CADe polyp detection, could evolve and improve continuously during ongoing clinical use. In contrast, “locked” algorithms provide a consistent output from the same inputs over time. A middle ground also exists. Even if the technology does not specifically incorporate autonomous learning, most software undergoes improvements resulting in iterative versions. Under the current FDA guidelines, these improvements from autonomous learning and iterative versions would trigger review, because they may affect performance, inputs, or intended use of the software.

      U.S. Food and Drug Administration. Proposed regulatory framework for modifications to artificial intelligence/machine learning (AI/ML)-based software as a medical device (SaMD)—discussion paper and request for feedback. Available at: https://www.fda.gov/media/122535/download. Accessed April 2, 2020.

      Although there have now been several prospective trials studying CADe/CADx in the field of gastroenterology, none of these technologies is currently cleared in the United Sates.
      • Mori Y.
      • Kudo S.-E.
      • Misawa M.
      • et al.
      Real-time use of artificial intelligence in identification of diminutive polyps during colonoscopy: a prospective study.
      ,
      • Wang P.
      • Berzin T.M.
      • Brown J.R.G.
      • et al.
      Real-time automatic detection system increases colonoscopic polyp and adenoma detection rates: a prospective randomised controlled study.
      ,
      • Wang P.
      • Liu X.
      • Berzin T.M.
      • et al.
      Effect of a deep-learning computer-aided detection system on adenoma detection during colonoscopy (CADe-DB trial): a double-blind randomised study.
      ,
      • Klare P.
      • Sander C.
      • Prinzen M.
      • et al.
      Automated polyp detection in the colorectum: a prospective study (with videos).
      Internationally, one CADx platform has secured approval in Japan and one CADe platform has been approved in the European Union.
      Cybernet Systems Co, Ltd
      EndoBRAIN—artificial intelligence system that supports optical diagnosis of colorectal polyps—was approved by PMDA (Pharmaceuticals and Medical Devices Agency), a regulatory body in Japan.
      ,
      Medtronic launches the first artificial intelligence system for colonoscopy at United European Gastroenterology Week 2019.
      Both of these tools, however, use “locked” algorithms. We anticipate that the first algorithms available for CADe or CADx in the field of gastroenterology in the United States will also be locked algorithms but will have the potential for planned, iterative version updates to improve performance over time. We expect that AI tools incorporating continuous learning could be ready for clinical use in gastroenterology within the next 5 years.

      FDA Regulations

      Regulation today

      The FDA’s current approval process for SaMD is derived from its approval process for medical devices. In a similar fashion to the risk categories defined for software, the FDA defines 3 risk categories for medical devices: class I (lowest risk), class II (moderate risk), and class III (highest risk).
      U.S. Food and Drug Administration
      Learn if a medical device has been cleared by FDA for marketing. FDA.
      According to their classification, devices are subject to general or special controls. General controls are provisions that apply to all medical devices and include registration and branding. Special controls are regulatory requirements that are usually device-specific and include performance standards and postmarket surveillance. Class I devices, such as examination gloves, are subject to only general controls, and 95% of these are exempt from the regulatory process. Class II devices, such as noninvasive blood pressure monitors, are subject to general controls and special controls. Finally, class III devices, such as artificial heart valves, usually sustain or support life or are implanted and require the highest level of evidence for safety and efficacy before approval.
      After device risk classification, the next step in the FDA approval process is to complete a premarket submission, which varies according to the risk and novelty of a given device. Most class II devices require a premarket notification, also known as a 510(k). In a 510(k), the device sponsor must demonstrate that their device is “substantially equivalent” to an existing device that has already been approved by the FDA.
      U.S. Food and Drug Administration
      How to study and market your device. FDA.
      If no predicate exists, sponsors of class II devices must apply for de novo classification, a more extensive process, that requires data demonstrating the safety and efficacy of the device.
      U.S. Food and Drug Administration
      Evaluation of automatic class III designation (de novo) summaries.
      Many of the AI-based SaMDs that have received FDA approval proceeded through either the 510(k) pathway or the de novo pathway.
      U.S. Food and Drug Administration
      Evaluation of automatic class III designation (de novo) summaries.
      For instance, OsteoDetect, an AI-based software that analyzes radiographs to assists clinicians in the diagnosis of distal radial fractures, completed the de novo pathway.
      U.S. Food and Drug Administration
      FDA permits marketing of artificial intelligence algorithm for aiding providers in detecting wrist fractures. FDA.
      Subsequently, ProFound AI, an AI-based software that assists in the detection of breast cancer, used OsteoDetect as a predicate device in its 510(k).
      • Voelker R.
      Diagnosing fractures with AI.
      Manufacturers of medical devices often make modifications and updates to their devices after obtaining approval. To provide guidance on when modifications require additional review, the FDA released a document entitled “Deciding When to Submit a 510(k) for a Change to an Existing Device” in 2017, which recommends that for devices initially reviewed and cleared under a 510(k) notification, a new premarket submission would be needed for software modifications that (1) introduce a new risk or modify an existing risk resulting in the potential for significant harm, (2) change risk controls to prevent significant harm, or (3) significantly change clinical functionality or performance specifications.
      U.S. Food and Drug Administration
      Deciding when to submit a 510(k) for a change to an existing device.
      Under the current FDA framework, we would expect CADe and CADx tools to be classified as class II devices because they are moderate risk and complex in design but are not implantable or used to sustain life. Similar to OsteoDetect and ProFound AI, we anticipate that the first generation of CADe and CADx tools will proceed through the de novo classification approval pathway and subsequent generations will likely only require a 510(k). CADe and CADx tools that are modified to add new clinical functionality would likely require an additional 510(k) submission.

      Regulation going forward

      The current FDA system described above is practical for software technologies with locked algorithms. However, AI algorithms that are rapidly iterative or those that incorporate continuous learning represent a more complex challenge for regulatory agencies, because the current regulatory processes do not match the natural evolution of these tools.
      To address these challenges, the FDA has proposed a new system of regulation for AI technologies. In its Digital Health Innovation Action Plan, the FDA outlined a Software Precertification (Pre-Cert) Pilot program to provide a streamlined regulatory model for SaMD, focused on AI technologies that rely on continuous learning and adaptation.
      U.S. Food and Drug Administration
      Digital health innovation action plan.
      This regulatory approach aims to evaluate the software developer rather than the just the software itself. The FDA will first screen for companies that exhibit a robust culture of quality and organizational excellence and are committed to longitudinal monitoring. The FDA expects that software products from these precertified companies will reliably meet the standard of safety and efficacy throughout their development, and thus modifications to SaMD over time will proceed through a streamlined review.
      U.S. Food and Drug Administration
      Digital health innovation action plan.
      The Pre-Cert regulatory model begins with the evaluation of the software company.
      U.S. Food and Drug Administration
      Developing a software precertification program: a working model.
      The FDA has established 2 levels of company classification under the Pre-Cert process. A level 1 Pre-Cert applies to organizations that have demonstrated excellence in 5 key areas (product quality, patient safety, clinical responsibility, cybersecurity responsibility, and proactive culture) but do not have a track record of delivering SaMD. A level 2 Pre-Cert applies to organizations that have demonstrated excellence in the 5 key areas and also have a track record of delivering SaMD.
      U.S. Food and Drug Administration
      Developing a software precertification program: a working model.
      After a company is classified, the SaMD it produces is evaluated and assigned into an IMDRF risk category.
      The FDA Digital Health proposal suggests that the required level of review for SaMD will take into account the Pre-Cert level of the organization, the IMDRF risk category, and the classification of the device as an initial product or iteration (Table 2). For changes to existing devices, the FDA additionally determines whether the proposed modification is a “minor change” or a “major change.” All these data points will help determine whether software/algorithm modifications can proceed without review or instead go forward with a “streamlined” review. The streamlined review process will focus on understanding the product, software functions, intended use, analytical performance, clinical performance, and safety measures.
      U.S. Food and Drug Administration
      Developing a software precertification program: a working model.
      Although the review will contain elements of the current premarket submission (eg, 510(k), de novo classification), the FDA will leverage an organization’s precertification to distill the review process to only those elements that are specific to the product.
      Table 2Level of review required for SaMD technologies is defined by the risk category and stage of review
      Adapted from U.S. Food and Drug Administration “Developing a Software Precertification Program.”
      U.S. Food and Drug Administration
      Developing a software precertification program: a working model.
      IMDRF risk categorizationLevel of review for level 1 and level 2 precertified organizations’ SaMD
      CategoryDescriptionInitial productMajor changesMinor changes
      IVCritical × diagnose/treatSRSRNo review
      III.iCritical × driveL1, SR
      III.iiSerious × diagnose/treatL2, no review
      II.iiSerious × driveL1, SR
      II.iiiNonserious × diagnose/treatL2, no reviewNo review
      II.iCritical × inform
      I.iiiNonserious × driveNo review
      I.iiSerious × inform
      I.iNonserious × inform
      IMDRF, International Medical Device Regulators Forum; SaMD, software as a medical device; SR, streamlined review.
      The final component of the Pre-Cert program is the collection of real-world data regarding health, user experience, and product performance. The FDA will use these data to monitor ongoing safety and efficacy, support modifications of performance claims, provide input to initial precertification, and provide feedback to the FDA.
      U.S. Food and Drug Administration
      Developing a software precertification program: a working model.
      This latter requirement is not one to be glossed over lightly in the world of GI endoscopy. Radiology, unlike gastroenterology, has a long tradition of centralized quality monitoring and industry standards for storing and sharing of imaging information. Examples include the Digital Imaging and Communications in Medicine (DICOM) standard developed in the 1980s by the American College of Radiology and the National Electrical Manufacturers Association.
      National Electrical Manufacturers Association
      History—DICOM standard.
      Similar efforts in gastroenterology, potentially led by professional societies, will be required so the field of GI endoscopy can develop data science infrastructures that are sufficient to support the ongoing safety and efficacy monitoring that will be required for AI algorithms.
      The FDA has already begun testing the Pre-Cert program in 2019 with 9 companies.
      U.S. Food and Drug Administration
      Digital health software precertification (pre-cert) program.
      This proposed regulatory framework represents an important step by the FDA toward addressing the potential regulatory burdens associated with nonlocked AI technology, in particular for CADe and CADx tools in gastroenterology. The program should enable the FDA to provide more agile regulation of new SaMD and modifications to existing SaMD across the total life cycle of products.

      International approval process

      Regulators around the world have also recognized the need for a new regulatory system to account for the unique features of AI-based SaMD. Countries including China, the European Union, Japan, Australia, the United Arab Emirates, and Canada have all initiated efforts to develop policies tailored for SaMD.

      Therapeutic Goods Administration. Consultation: regulation of software, including software as a medical device (SaMD). Available at: https://www.tga.gov.au/sites/default/files/consultation-regulation-software-including-software-medical-device-samd.pdf. Accessed April 2, 2020.

      Department of Health. Policy on use of artificial intelligence (AI) in the healthcare sector of the Emirate of Abu Dhabi. Available at: https://doh.gov.ae/-/media/E9C1470A575146B18015DEBE57E47F8D.ashx. Accessed April 2, 2020.

      • Harvey H.
      How to get clinical AI tech approved by regulators.
      Canadian Institutes of Health Research Government of Canada
      Introduction of artificial intelligence and machine learning in medical devices.
      ChinaMed Device, LLC
      NMPA (CFDA) final guideline on AI-aided software: propels China to the leadership of AI applications.

      Pharmaceuticals and Medical Devices Agency. Guidance for evaluation of artificial intelligence–assisted medical imaging systems for clinical diagnosis. Available at: http://dmd.nihs.go.jp/jisedai/tsuuchi/Guidance_for_evaluation_of_AI_assisted_systems.pdf. Accessed April 2, 2020.

      Although each country and governing body has unique processes for the approval and regulation of software technologies, many of them share the same core principles: designation of risk, review of clinical evidence to demonstrate safety and efficacy, and evolving practices to incorporate new types of rapidly evolving software technologies. The FDA’s efforts in this field appear to be the most mature among the international community, and thus we expect the framework discussed above will serve as a model for many regulators.

      Conclusion

      There has been a tremendous growth in AI-based tools to support the practice of GI endoscopy. For these tools to be safely and effectively integrated into clinical practice, regulatory agencies must be able to evaluate and monitor these technologies using new approaches that are both robust and practical. The FDA in particular is making clear efforts to develop effective regulatory pathways to accomplish these goals. Because of the technical advances and clinical trial progress already occurring for colon polyp CADe and CADx, we expect the field of gastroenterology will play a leading role in charting the near future of AI in clinical medicine.

      References

        • Topol E.J.
        High-performance medicine: the convergence of human and artificial intelligence.
        Nat Med. 2019; 25: 44-56
        • Ting D.S.W.
        • Cheung C.Y.-L.
        • Lim G.
        • et al.
        Development and validation of a deep learning system for diabetic retinopathy and related eye diseases using retinal images from multiethnic populations with diabetes.
        JAMA. 2017; 318: 2211-2223
        • Esteva A.
        • Kuprel B.
        • Novoa R.A.
        • et al.
        Dermatologist-level classification of skin cancer with deep neural networks.
        Nature. 2017; 542: 115-118
        • Byrne M.F.
        • Chapados N.
        • Soudan F.
        • et al.
        Real-time differentiation of adenomatous and hyperplastic diminutive colorectal polyps during analysis of unaltered videos of standard colonoscopy using a deep learning model.
        Gut. 2019; 68: 94-100
        • Repici A.
        • Hassan C.
        • Radaelli F.
        • et al.
        Accuracy of narrow-band imaging in predicting colonoscopy surveillance intervals and histology of distal diminutive polyps: results from a multicenter, prospective trial.
        Gastrointest Endosc. 2013; 78: 106-114
        • Misawa M.
        • Kudo S.-E.
        • Mori Y.
        • et al.
        Artificial intelligence-assisted polyp detection for colonoscopy: initial experience.
        Gastroenterology. 2018; 154: 2027-2029
        • Urban G.
        • Tripathi P.
        • Alkayali T.
        • et al.
        Deep learning localizes and identifies polyps in real time with 96% accuracy in screening colonoscopy.
        Gastroenterology. 2018; 155: 1069-1078
        • Mori Y.
        • Kudo S.-E.
        • Misawa M.
        • et al.
        Real-time use of artificial intelligence in identification of diminutive polyps during colonoscopy: a prospective study.
        Ann Intern Med. 2018; 169: 357-366
        • Cybernet Systems Co, Ltd
        EndoBRAIN—artificial intelligence system that supports optical diagnosis of colorectal polyps—was approved by PMDA (Pharmaceuticals and Medical Devices Agency), a regulatory body in Japan.
        (Available at:)
        • Wang P.
        • Berzin T.M.
        • Brown J.R.G.
        • et al.
        Real-time automatic detection system increases colonoscopic polyp and adenoma detection rates: a prospective randomised controlled study.
        Gut. 2019; 68: 1813-1819
        • Wang P.
        • Liu X.
        • Berzin T.M.
        • et al.
        Effect of a deep-learning computer-aided detection system on adenoma detection during colonoscopy (CADe-DB trial): a double-blind randomised study.
        Lancet Gastroenterol Hepatol. 2020; 5: 343-351
        • Lieberman D.
        • Sullivan B.A.
        • Hauser E.R.
        • et al.
        Baseline colonoscopy findings associated with 10-year outcomes in a screening cohort undergoing colonoscopy surveillance.
        Gastroenterology. 2020; 158: 862-874
        • Vinsard D.G.
        • Mori Y.
        • Misawa M.
        • et al.
        Quality assurance of computer-aided detection and diagnosis in colonoscopy.
        Gastrointest Endosc. 2019; 90: 55-63
        • Liu X.
        • Faes L.
        • Calvert M.J.
        • et al.
        Extension of the CONSORT and SPIRIT statements.
        Lancet. 2019; 394: 1225
        • Rex D.K.
        • Kahi C.
        • O’Brien M.
        • et al.
        The American Society for Gastrointestinal Endoscopy PIVI (Preservation and Incorporation of Valuable Endoscopic Innovations) on real-time endoscopic assessment of the histology of diminutive colorectal polyps.
        Gastrointest Endosc. 2011; 73: 419-422
      1. IMDRF SaMD Working Group. Software as a medical device (SaMD): key definitions. Available at: http://www.imdrf.org/docs/imdrf/final/technical/imdrf-tech-131209-samd-key-definitions-140901.pdf. Accessed March 26, 2020.

        • IMDRF SaMD Working Group
        Software as a medical device (SaMD): clinical evaluation.
        (Available at:)
      2. U.S. Food and Drug Administration. Proposed regulatory framework for modifications to artificial intelligence/machine learning (AI/ML)-based software as a medical device (SaMD)—discussion paper and request for feedback. Available at: https://www.fda.gov/media/122535/download. Accessed April 2, 2020.

        • Klare P.
        • Sander C.
        • Prinzen M.
        • et al.
        Automated polyp detection in the colorectum: a prospective study (with videos).
        Gastrointest Endosc. 2019; 89: 576-582
      3. Medtronic launches the first artificial intelligence system for colonoscopy at United European Gastroenterology Week 2019.
        (Available at:)
        • U.S. Food and Drug Administration
        Learn if a medical device has been cleared by FDA for marketing. FDA.
        (Available at:)
        • U.S. Food and Drug Administration
        How to study and market your device. FDA.
        (Available at:)
        • U.S. Food and Drug Administration
        Evaluation of automatic class III designation (de novo) summaries.
        (Available at:)
        • U.S. Food and Drug Administration
        FDA permits marketing of artificial intelligence algorithm for aiding providers in detecting wrist fractures. FDA.
        (Available at:)
        • Voelker R.
        Diagnosing fractures with AI.
        JAMA. 2018; 320: 23
        • U.S. Food and Drug Administration
        Deciding when to submit a 510(k) for a change to an existing device.
        (Available at:)
        • U.S. Food and Drug Administration
        Digital health innovation action plan.
        (Available at:)
        • U.S. Food and Drug Administration
        Developing a software precertification program: a working model.
        (Available at:)
        • National Electrical Manufacturers Association
        History—DICOM standard.
        (Available at:)
        https://www.dicomstandard.org/history/
        Date accessed: April 10, 2020
        • U.S. Food and Drug Administration
        Digital health software precertification (pre-cert) program.
        (Available at:)
      4. Therapeutic Goods Administration. Consultation: regulation of software, including software as a medical device (SaMD). Available at: https://www.tga.gov.au/sites/default/files/consultation-regulation-software-including-software-medical-device-samd.pdf. Accessed April 2, 2020.

      5. Department of Health. Policy on use of artificial intelligence (AI) in the healthcare sector of the Emirate of Abu Dhabi. Available at: https://doh.gov.ae/-/media/E9C1470A575146B18015DEBE57E47F8D.ashx. Accessed April 2, 2020.

        • Harvey H.
        How to get clinical AI tech approved by regulators.
        (Available at:)
        • Canadian Institutes of Health Research Government of Canada
        Introduction of artificial intelligence and machine learning in medical devices.
        (Available at:)
        https://cihr-irsc.gc.ca/e/51459.html
        Date accessed: April 2, 2020
        • ChinaMed Device, LLC
        NMPA (CFDA) final guideline on AI-aided software: propels China to the leadership of AI applications.
        (Available at:)
      6. Pharmaceuticals and Medical Devices Agency. Guidance for evaluation of artificial intelligence–assisted medical imaging systems for clinical diagnosis. Available at: http://dmd.nihs.go.jp/jisedai/tsuuchi/Guidance_for_evaluation_of_AI_assisted_systems.pdf. Accessed April 2, 2020.

      7. IMDRF SaMD Working Group. “Software as a medical device”: possible framework for risk categorization and corresponding considerations. Available at: http://www.imdrf.org/docs/imdrf/final/technical/imdrf-tech-140918-samd-framework-risk-categorization-141013.pdf. Accessed April 1, 2020.