Advertisement

The path to quality colonoscopy continues after graduation

      Abbreviations:

      ADR (adenoma detection rate), CIR (cecal intubation rate), DIP (drop in colonoscopy performance), GIQuIC (GI quality improvement consortium), PCC (provisional colonoscopy certification)
      The effort over the past decade to improve quality in colonoscopy was marked by several essential steps forward. These included definition of quality indicators,
      • Faulx A.L.
      • Lightdale J.R.
      • Acosta R.D.
      • et al.
      Guidelines for privileging, credentialing, and proctoring to perform GI endoscopy.

      Faigel DO, Baron TH, Lewis B, et al. Ensuring competence in endoscopy: ASGE taskforce on ensuring competency in endoscopy and ACG executive and practical management committees.

      the development and validation of objective skills assessment tools such as the American Society for Gastrointestinal Endoscopy assessment of competency in endoscopy tool, the direct observation of procedural skills (DOPS),
      • Barton J.R.
      • Corbett S.
      • van der Vleuten C.P.
      The validity and reliability of a direct observation of procedural skills assessment tool: assessing colonoscopic skills of senior endoscopists.
      • Sedlack R.E.
      • Coyle W.J.
      • Obstein K.L.
      • et al.
      ASGE’s assessment of competency in endoscopy evaluation tools for colonoscopy and EGD.
      and the investment in infrastructure to track and benchmark performance for key indicators such as the GI quality improvement consortium (GIQuIC)

      Pike IM. GIQuIC: more than a decade of endoscopic improvement through observation. Healio Gastroenterol April 2018.

      and the Joint Advisory Group on GI Endoscopy - Endoscopy Training System eportfolio in the United Kingdom.
      • Mehta T.
      • Dowler K.
      • McKaig B.C.
      • et al.
      Development and roll out of the JETS e-portfolio: a web based electronic portfolio for endoscopists.
      Parallel to this effort to measure the quality of colonoscopy performance of practitioners so as to improve patient outcome was the application of these principles to the trainee learning process. The endpoints of training or competency were defined as the ability of the trainee to independently perform each specific task being assessed at a performance level or target that was deemed acceptable relative to benchmark performance data in the practicing community. This remains a work in progress while endoscopic education evolves into true competency-based training, replete with milestones and replacement of subjective assessment with objective summative measurement.
      Even when this minimum level of skill for independent practice is achieved, how do trainees perform once they complete their supervised training? In this issue of Gastrointestinal Endoscopy, Siau et al
      • Siau K.
      • Hodson J.
      • Valori R.M.
      • et al.
      Performance indicators in colonoscopy after certification for independent practice: outcomes and predictors of competence.
      demonstrate that many fellows experience an initial drop in colonoscopy performance (DIP) after graduation that rebounds over their first 100 cases.
      In this study, the authors compare the colonoscopy performance measures of a large cohort of trainees before and after they achieve provisional colonoscopy certification (PCC). To achieve PCC, the trainees must achieve a cecal intubation rate (CIR) of at least 90% and undergo objective skill assessment, including the use of the direct observation of procedural skills tool. During the first 50 cases after attainment of PCC, with 78% of colonoscopies performed without trainer assistance or observation, the authors tracked the fellows’ outcome measures because a significant portion experienced a decline to below 90% CIR along with increased use of sedatives and associated lower levels of patient comfort. The authors observed that the 2 predictors of minimizing this posttraining DIP were a very high CIR before PCC (>98%) and having a gastroenterologist (rather than a nonmedical endoscopist) as the trainer. Despite the DIP, the entire cohort recovered the PCC level of CIR by the end of the first 100 cases after training.
      This study raises several important issues, both specific to the United Kingdom and more broadly. Partial or provisional certification is not formalized in training programs in the United States, although it is fairly common for more-senior fellows to function more independently with lessoversight from attending gastroenterologists. In some programs, senior fellows may even have a more-direct role in the supervision and training of their more-junior peers. Supervision, as in the United Kingdom study, may take several forms, including less-direct oversight (“available in the endoscopy unit”), more direct observational supervision with or without verbal coaching, and more focused “hands-on” teaching involving manipulation of the instrument during the trainee’s procedure, particularly earlier in training or during a more challenging procedure.
      When is it appropriate for a trainee to perform independent colonoscopy? The decision of where to set the bar for provisional or full competency ought to be the achievement of minimal target levels of technical and cognitive competency set for community practice based on actual benchmarking performance data. For colonoscopy in the United States, these targets for quality indicators are well set forth.
      • Faulx A.L.
      • Lightdale J.R.
      • Acosta R.D.
      • et al.
      Guidelines for privileging, credentialing, and proctoring to perform GI endoscopy.

      Faigel DO, Baron TH, Lewis B, et al. Ensuring competence in endoscopy: ASGE taskforce on ensuring competency in endoscopy and ACG executive and practical management committees.

      • Sedlack R.E.
      • Coyle W.J.
      • Obstein K.L.
      • et al.
      ASGE’s assessment of competency in endoscopy evaluation tools for colonoscopy and EGD.
      However, it remains difficult to collect meaningful data on quality indicators for current trainees. For example, to assess a meaningful adenoma detection rate (ADR) value with narrow confidence intervals, one should have data on at least 100 screening colonoscopies. When a supervising physician is present, how fastidious can the supervising physician and fellow be to ensure that the measured ADR distinguishes between that achieved by the trainee independently of the attending’s contribution in supervising the procedures? CIR is perhaps far easier to track. In our opinion, the failure of a trainee to achieve CIR as a quality indicator standard should be a barrier to initial full independent practice after graduation—perhaps even with data showing that they are likely to achieve this with time. The granting of PCC in this study relied heavily on CIR as self-reported by the trainees, and it is noteworthy that the polyp detection rate demonstrated during the first 100 cases after PCC was comparable with the community standard, providing support for the validity of using the PCC to mark a transition with regard to trainee supervision.
      However, the fact that a DIP occurred to levels below acceptable standards during the first 50 cases in 18% of the study participants implies that PCC is aptly termed “provisional.” A key take-home message of this article by Siau et al
      • Siau K.
      • Hodson J.
      • Valori R.M.
      • et al.
      Performance indicators in colonoscopy after certification for independent practice: outcomes and predictors of competence.
      is the need to continue the process of skills assessment and performance measurement started during training into the first year of practice, and possibly beyond. It seems that tied to this must be an action plan to protect patients, identify DIPs when they occur, and provide backup supervision as needed.
      The authors identify 2 factors that may help predict who might need such backup support during early independent practice. But indeed, should only trainees at risk for a decline in performance monitor their outcomes after PCC or graduation? It seems that if outcome measurements by use of a self-reporting eportfolio or by more objective means tied to electronic procedure reporting are feasible, then the preferable way to ensure quality colonoscopy is to have all recent trainees track their performance.
      Closely tied to this line of reasoning is the question of maintenance of skill and privileges. The same infrastructure that would allow monitoring of recent graduates could one day be applied as part of the biennial recredentialing process in a move to track outcomes prospectively over time. Credentialing for endoscopy remains a controversial topic of debate in the United States, although some assessment and benchmarking tools required to make it feasible, at least for colonoscopy, already exist.
      There are a few limitations to the present study. First, we have no data as to when the CIR and other metrics tested reach a plateau after recovering from the initial decline (DIP), or how the polyp detection rate might improve during the first few years of independent practice. Should the process of periodic peer review of graduates with PCC status end once CIR alone stays above the 90% threshold? Or should the process of colonoscopy performance measurement continue indefinitely, as more than 4200 endoscopists are now doing in the United States every year using GIQuIC? If so, the point of this article is not when to stop tracking but rather the fact that tracking marks and perhaps facilitates ongoing quality improvement. Indeed, the quality indicator targets are intended as minimum performance flagposts based on both what is feasible and what can lead to improved patient outcomes. Improving to greater-than-target values of ADR, for example, have been associated with decreased colorectal cancer mortality.
      • Corley D.A.
      • Jensen C.D.
      • Marks A.R.
      • et al.
      Adenoma detection rate and risk of colorectal cancer and death.
      Although tracking outcomes may indeed benefit all endoscopists and their patients, we agree with the authors that their work demonstrates that early graduates have a particular need to adopt this practice.
      Why does performance decline after PCC for some trainees? These data do not provide a clear answer. Case selection during trainee endoscopy sessions may be affected by both patient preferences and faculty input. Gradual tackling of more challenging cases by young graduates may be more a factor in a decline in successful metrics for individual procedures that are more complex, more therapeutic, or both, such as ERCP and EMR.
      What we still do not know is whether the act of measuring outcome during the first year of practice contributes as a formative tool to the ongoing improvement of the post-PCC trainees as they overcome the initial DIP. This research raises the question whether an endoscopist with a DIP after the first year would have improved with time and experience as much or as quickly if the endoscopist’s key performance indicators not been tracked.
      The authors also acknowledge the limitation of self-reported data using the electronic portfolio. Confirmatory investigation in the United States should be quite feasible with use of the well-established GIQuIC registry.
      The finding in this article of a lower incidence in the DIP when the trainee was mentored by a specialist in gastroenterology raises the question as to who might be the best instructors for trainees, regardless of the trainee’s own specialty. Ideally, trainees will learn from the faculty most skilled in both performing and teaching colonoscopy whenever feasible.
      The particular case volume of colonoscopy during pre-PCC training in this study may also limit the generalizability of the findings. In the United States, where gastroenterology fellows typically perform far more than the mean number of 265 cases over their standard 3-year fellowship training, it is an open question whether a similar DIP would be observed in the first 50 or so cases after graduation. For others who go on to perform colonoscopy with a provisional certification analogous to that used in the United Kingdom after far fewer supervised cases, but supplemented by use of endoscopic simulators and formal curricula, the type of early practice performance tracking with available backup support appears to be critically important.
      It will be important to delve deeper into the benefit shown here by Siau et al
      • Siau K.
      • Hodson J.
      • Valori R.M.
      • et al.
      Performance indicators in colonoscopy after certification for independent practice: outcomes and predictors of competence.
      of ongoing self-scrutiny and performance tracking in the first year of practice. If this principle is true for colonoscopy, it should be even more applicable to certain therapeutic procedures with higher risks of adverse events and wide variations in performance metrics, such as ERCP and mucosal resection. Native papilla cannulation rates of at least 90% is a quality target for ERCP
      • Baron T.H.
      • Petersen B.T.
      • Mergener K.
      • et al.
      Quality indicators for endoscopic retrograde pancreatography.
      • Adler D.G.
      • Lieb J.G.
      • Cohen J.
      • et al.
      Quality indicators for ERCP.
      and recent data point to deficiencies in many graduates of advanced fourth-year endoscopy programs to achieve this target, in the setting of limited case numbers for supervised practice during training.
      • Wani S.
      • Keswani R.
      • Hall M.
      • et al.
      A prospective multicenter study evaluating learning curves and competence in endoscopic ultrasound and endoscopic retrograde cholangiopancreatography among advanced endoscopy trainees: the rapid assessment of trainee endoscopy skills study.
      In such circumstances, it is even more important to promote tracking of procedure outcomes after graduation to provide opportunities for mentoring early in practice and to encourage those who need it to take advantage of such support.
      Overall, this research supports a 4-step trajectory for training new colonoscopists:
      • 1.
        Training to achieve defined quality parameters for a given procedure
      • 2.
        Tracking serial performance during training to guide and possibly accelerate training
      • 3.
        Undergoing comprehensive skill assessment using validated tools before training is completed
      • 4.
        Following such procedural assessments with actual practice performance data to confirm competency
      Unlike the Greek goddess of wisdom, Athena, who emerged fully formed from the forehead of Zeus, endoscopists must follow an uphill and sometimes undulating path on the way to competent high-quality performance. Siau et al
      • Siau K.
      • Hodson J.
      • Valori R.M.
      • et al.
      Performance indicators in colonoscopy after certification for independent practice: outcomes and predictors of competence.
      illustrate well how we must continually work to improve even after we are given the green light to practice on our own.

      Disclosure

      Dr Poppers is a consultant/educator for Olympus. The other author disclosed no financial relationships relevant to this publication.

      References

        • Faulx A.L.
        • Lightdale J.R.
        • Acosta R.D.
        • et al.
        Guidelines for privileging, credentialing, and proctoring to perform GI endoscopy.
        Gastrointest Endosc. 2017; 85: 274-281
      1. Faigel DO, Baron TH, Lewis B, et al. Ensuring competence in endoscopy: ASGE taskforce on ensuring competency in endoscopy and ACG executive and practical management committees.

        • Barton J.R.
        • Corbett S.
        • van der Vleuten C.P.
        The validity and reliability of a direct observation of procedural skills assessment tool: assessing colonoscopic skills of senior endoscopists.
        Gastrointest Endosc. 2012; 75: 591-597
        • Sedlack R.E.
        • Coyle W.J.
        • Obstein K.L.
        • et al.
        ASGE’s assessment of competency in endoscopy evaluation tools for colonoscopy and EGD.
        Gastrointest Endosc. 2014; 79: 1-7
      2. Pike IM. GIQuIC: more than a decade of endoscopic improvement through observation. Healio Gastroenterol April 2018.

        • Mehta T.
        • Dowler K.
        • McKaig B.C.
        • et al.
        Development and roll out of the JETS e-portfolio: a web based electronic portfolio for endoscopists.
        Frontline Gastroenterol. 2011; 2: 35-42
        • Siau K.
        • Hodson J.
        • Valori R.M.
        • et al.
        Performance indicators in colonoscopy after certification for independent practice: outcomes and predictors of competence.
        Gastrointest Endosc. 2019; 89: 482-492.e2
        • Corley D.A.
        • Jensen C.D.
        • Marks A.R.
        • et al.
        Adenoma detection rate and risk of colorectal cancer and death.
        N Engl J Med. 2014; 370: 1298-1306
        • Baron T.H.
        • Petersen B.T.
        • Mergener K.
        • et al.
        Quality indicators for endoscopic retrograde pancreatography.
        Am J Gastroenterol. 2006; 101: 892-897
        • Adler D.G.
        • Lieb J.G.
        • Cohen J.
        • et al.
        Quality indicators for ERCP.
        Gastrointest Endosc. 2015; 81: 54-66
        • Wani S.
        • Keswani R.
        • Hall M.
        • et al.
        A prospective multicenter study evaluating learning curves and competence in endoscopic ultrasound and endoscopic retrograde cholangiopancreatography among advanced endoscopy trainees: the rapid assessment of trainee endoscopy skills study.
        Clin Gastroenterol Hepatol. 2017; 15: 1758-1767

      Linked Article