Effect of individualized feedback on learning curves in EGD and colonoscopy: a cluster randomized controlled trial

Published:November 09, 2019DOI:

      Background and Aims

      Gastroenterology fellowships need to ensure that trainees achieve competence in upper endoscopy (EGD) and colonoscopy. Because the impact of structured feedback remains unknown in endoscopy training, this study compared the effect of structured feedback with standard feedback on trainee learning curves for EGD and colonoscopy.


      In this multicenter, cluster, randomized controlled trial, trainees received either individualized quarterly learning curves or feedback standard to their fellowship. Assessment was performed in all trainees using the Assessment of Competency in Endoscopy tool on 5 consecutive procedures after every 25 EGDs and colonoscopies. Individual learning curves were created using cumulative sum (CUSUM) analysis. The primary outcome was the mean CUSUM score in overall technical and overall cognitive skills.


      In all, 13 programs including 132 trainees participated. The intervention arm (6 programs, 51 trainees) contributed 558 EGD and 600 colonoscopy assessments. The control arm (7 programs, 81 trainees) provided 305 EGD and 468 colonoscopy assessments. For EGD, the intervention arm (–.7 [standard deviation {SD}, 1.3]) had a superior mean CUSUM score in overall cognitive skills compared with the control arm (1.6 [SD, .8], P = .03) but not in overall technical skills (intervention, –.26 [SD, 1.4]; control, 1.76 [SD, .7]; P = .06). For colonoscopy, no differences were found between the 2 arms in overall cognitive skills (intervention, –.7 [SD, 1.3]; control, .7 [SD, 1.3]; P = .95) or overall technical skills (intervention, .1 [SD, 1.5]; control, –.1 [SD, 1.5]; P = .77).


      Quarterly feedback in the form of individualized learning curves did not affect learning curves for EGD and colonoscopy in a clinically meaningful manner. (Clinical trial registration number: NCT02891304.)

      Graphical abstract


      ACE (Assessment of Competency in Education), ACGME (Accreditation Council for Graduate Medical Education), CUSUM (cumulative sum), NAS (Next Accreditation System)
      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Gastrointestinal Endoscopy
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Nasca T.J.
        • Philibert I.
        • Brigham T.
        • et al.
        The next GME accreditation system—rationale and benefits.
        N Engl J Med. 2012; 366: 1051-1056
        • Nasca T.J.
        • Weiss K.B.
        • Bagian J.P.
        • et al.
        The accreditation system after the "next accreditation system.".
        Acad Med. 2014; 89: 27-29
        • Sedlack R.E.
        • Coyle W.J.
        • Obstein K.L.
        • et al.
        ASGE's assessment of competency in endoscopy evaluation tools for colonoscopy and EGD.
        Gastrointest Endosc. 2014; 79: 1-7
        • Adler D.G.
        • Bakis G.
        • Coyle W.J.
        • et al.
        Principles of training in GI endoscopy.
        Gastrointest Endosc. 2012; 75: 231-235
        • Sedlack R.E.
        • Coyle W.J.
        Assessment of competency in endoscopy: establishing and validating generalizable competency benchmarks for colonoscopy.
        Gastrointest Endosc. 2016; 83: 516-523
        • Spier B.J.
        • Benson M.
        • Pfau P.R.
        • et al.
        Colonoscopy training in gastroenterology fellowships: determining competence.
        Gastrointest Endosc. 2010; 71: 319-324
        • Ward S.T.
        • Mohammed M.A.
        • Walt R.
        • et al.
        An analysis of the learning curve to achieve competency at colonoscopy using the JETS database.
        Gut. 2014; 63: 1746-1754
        • van de Ridder J.M.
        • Stokking K.M.
        • McGaghie W.C.
        • et al.
        What is feedback in clinical education?.
        Med Educ. 2008; 42: 189-197
        • Branch Jr., W.T.
        • Paranjape A.
        Feedback and reflection: teaching methods for clinical settings.
        Acad Med. 2002; 77: 1185-1188
        • Richardson B.K.
        Acad Emerg Med. 2004; 11: e1-e5
        • Education ACfGM
        Common Program Requirements.
        2019 (Available at: Accessed July 2, 2019)
        • Kluger A.N.
        • DeNisi A.
        The effects of feedback interventions on performance: a historical review, a meta-analysis, and a preliminary feedback intervention theory.
        Psychol Bull. 1996; 119: 254-284
        • Anderson J.
        The future of gastroenterology training: instruction in technical skills.
        Frontline Gastroenterol. 2012; 3: i13-i18
        • Dilly C.K.
        • Sewell J.L.
        How to give feedback during endoscopy training.
        Gastroenterology. 2017; 153: 632-636
        • Gastroenterology ACo
        GI Fellowship Program Information. Volume 2018.
        American College of Gastroenterology, 2018 (Available at: Accessed June 11, 2019)
        • Miller A.T.
        • Sedlack R.E.
        Competency in esophagogastroduodenoscopy: a validated tool for assessment and generalizable benchmarks for gastroenterology fellows.
        Gastrointest Endosc. 2019; 90: 613-620
        • Ten Cate O.
        • Hart D.
        • Ankel F.
        • et al.
        Entrustment decision making in clinical training.
        Acad Med. 2016; 91: 191-198
      1. Han S, Obuch JC, Duloy AM, et al. A prospective multicenter study evaluating endoscopy competence among gastroenterology trainees in the era of the next accreditation system. Acad Med. Epub 2019 Jul 16.

        • Harris P.A.
        • Taylor R.
        • Thielke R.
        • et al.
        Research electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support.
        J Biomed Inform. 2009; 42: 377-381
        • Wani S.
        • Cote G.A.
        • Keswani R.
        • et al.
        Learning curves for EUS by using cumulative sum analysis: implications for American Society for Gastrointestinal Endoscopy recommendations for training.
        Gastrointest Endosc. 2013; 77: 558-565
        • Wani S.
        • Hall M.
        • Keswani R.N.
        • et al.
        Variation in aptitude of trainees in endoscopic ultrasonography, based on cumulative sum analysis.
        Clin Gastroenterol Hepatol. 2015; 13: 1318-1325
        • Wani S.
        • Hall M.
        • Wang A.Y.
        • et al.
        Variation in learning curves and competence for ERCP among advanced endoscopy trainees by using cumulative sum analysis.
        Gastrointest Endosc. 2016; 83: 711-719
        • Wani S.
        • Keswani R.
        • Hall M.
        • et al.
        A prospective multicenter study evaluating learning curves and competence in endoscopic ultrasound and endoscopic retrograde cholangiopancreatography among advanced endoscopy trainees: the rapid assessment of trainee endoscopy skills study.
        Clin Gastroenterol Hepatol. 2017; 15: 1758-1767
        • Wani S.
        • Keswani R.N.
        • Han S.
        • et al.
        Competence in endoscopic ultrasound and endoscopic retrograde cholangiopancreatography, from training through independent practice.
        Gastroenterology. 2018; 155: 1483-1494
        • Ende J.
        Feedback in clinical medical education.
        JAMA. 1983; 250: 777-781
        • Thuraisingam A.I.
        • MacDonald J.
        • Shaw I.S.
        Insights into endoscopy training: a qualitative study of learning experience.
        Med Teach. 2006; 28: 453-459
        • Dolan B.M.
        • Yialamas M.A.
        • McMahon G.T.
        A randomized educational intervention trial to determine the effect of online education on the quality of resident-delivered care.
        J Grad Med Educ. 2015; 7: 376-381
        • Gupta S.
        • Alladina J.
        • Heaton K.
        • et al.
        A randomized trial of an intervention to improve resident-fellow teaching interactions on the wards.
        BMC Med Educ. 2016; 16: 276
        • Peery A.F.
        • Crockett S.D.
        • Murphy C.C.
        • et al.
        Burden and cost of gastrointestinal, liver, and pancreatic diseases in the United States: update 2018.
        Gastroenterology. 2019; 156: 254-272
        • Sewell J.L.
        • Boscardin C.K.
        • Young J.Q.
        • et al.
        Measuring cognitive load during procedural skills training with colonoscopy as an exemplar.
        Med Educ. 2016; 50: 682-692
        • Sewell J.L.
        • Boscardin C.K.
        • Young J.Q.
        • et al.
        Learner, patient, and supervisor features are associated with different types of cognitive load during procedural skills training: implications for teaching and instructional design.
        Acad Med. 2017; 92: 1622-1631
        • Mann K.
        • van der Vleuten C.
        • Eva K.
        • et al.
        Tensions in informed self-assessment: how the desire for feedback and reticence to collect and use it can conflict.
        Acad Med. 2011; 86: 1120-1127
        • Telio S.
        • Ajjawi R.
        • Regehr G.
        The "educational alliance" as a framework for reconceptualizing feedback in medical education.
        Acad Med. 2015; 90: 609-614
        • Walsh C.M.
        • Ling S.C.
        • Wang C.S.
        • et al.
        Concurrent versus terminal feedback: It may be better to wait.
        Acad Med. 2009; 84: S54-S57
        • Wani S.
        • Keswani R.N.
        • Han S.
        • et al.
        Competence in endoscopic ultrasound and endoscopic retrograde cholangiopancreatography, from training through independent practice.
        Gastroenterology. 2018; 155: 1483-1494
        • Rose S.
        • Fix O.K.
        • Shah B.J.
        • et al.
        Entrustable professional activities for gastroenterology fellowship training.
        Gastroenterology. 2014; 147: 233-242
        • Hauer K.E.
        • Ten Cate O.
        • Boscardin C.
        • et al.
        Understanding trust as an essential element of trainee supervision and learning in the workplace.
        Adv Health Sci Educ Theory Pract. 2014; 19: 435-456