The Use of Formative Assessment in Postgraduate Urology Training: A Systematic Review

Rehan N Khan 1, Nadeem A Siddiqui 2

Affiliations

doi: 10.7759/cureus.27162

Abstract

Formative assessment is an essential component of surgical training. However, it is not usually a mandatory component in postgraduate curricula. The purpose of this study is to identify and evaluate how formative assessments are integrated into postgraduate urology training in programs across the globe. This study consisted of a systemic review to see how formative assessments are being implemented in various urology programs globally. A total of 427 articles were identified for the literature review. Of these, only 10 were included and critically appraised. These studies explored various techniques for exploration of formative assessments in urology training programs, which included established tools, such as portfolio reviews, and direct observations of procedure skills (DOPS); novel tools, including the Dutch urology practical skills (D-UPS) program and Ottawa surgical competency operating room evaluation (O-SCORE); and curricular models. Nine of the 10 articles favored their potential utility in formative assessments. Current literature involving formative assessments in postgraduate urology programs is scarce, and available resources have a high heterogeneity between them. More structured formative assessments need to be incorporated into surgical training programs, and affiliated training institutions should be encouraged to integrate them into their curricula.

Keywords: formative assessments; formative feedback; medical education; post-graduate training; urology.

Conflict of interest statement

The authors have declared that no competing interests exist.

Figures

Figure 1. PRISMA flow diagram PRISMA: Preferred Reporting Items for Systematic Reviews and Meta-Analyses.

Figure 1. PRISMA flow diagram PRISMA: Preferred Reporting Items for Systematic Reviews and Meta-Analyses.

Similar articles

The Usefulness of Procedure-Based Assessments in Postgraduate Surgical Training Within the Intercollegiate Surgical Curriculum Programme; A Scoping Review.

Mayne A, Wilson L, Kennedy N.J Surg Educ. 2020 Sep-Oct;77(5):1227-1235. doi: 10.1016/j.jsurg.2020.03.005. Epub 2020 May 22.PMID: 32451310 Review.

Training and Assessment in Urology: Perspective From a Third World Country.

Khan RN, Sultan AS, Siddiqui N.Cureus. 2022 May 25;14(5):e25341. doi: 10.7759/cureus.25341. eCollection 2022 May.PMID: 35774670 Free PMC article. Review.

Effectiveness of Direct Observation of Procedural Skills (DOPS) in Postgraduate Training in Urology at Institute of Kidney Diseases, Peshawar.

Ali L, Ali S, Orakzai N, Ali N.J Coll Physicians Surg Pak. 2019 Jun;29(6):516-519. doi: 10.29271/jcpsp.2019.06.516.PMID: 31133147

Improving surgical training by identifying the most common feedback trainees receive for index cases.

Maizels M, Liu D, Yerkes E, Meade P, Biondi M, Sutherland R.J Pediatr Urol. 2018 Dec;14(6):577-583. doi: 10.1016/j.jpurol.2018.07.026. Epub 2018 Aug 28.PMID: 30270102

Feedback and formative assessment in Competency by Design The experience of residents and supervisors within a urology training program.

Aubé-Peterkin M, Ehlebracht A, Petrella F.Can Urol Assoc J. 2023 Apr;17(4):94-100. doi: 10.5489/cuaj.8103.PMID: 36486176 Free PMC article.

KMEL References

https://pubmed.ncbi.nlm.nih.gov/

References

  1.  
    1. Debriefing as formative assessment: closing performance gaps in medical education. Rudolph JW, Simon R, Raemer DB, Eppich WJ. Acad Emerg Med. 2008;15:1010–1016. - PubMed
  2.  
    1. Differential effects of two types of formative assessment in predicting performance of first-year medical students. Krasne S, Wimmers PF, Relan A, Drake TA. Adv Health Sci Educ Theory Pract. 2006;11:155–171. - PubMed
  3.  
    1. Formative assessment: how am I doing? Rolfe I, McPherson J. Lancet. 1995;345:837–839. - PubMed
  4.  
    1. Formative assessment and academic achievement in pre-graduate students of health sciences. Carrillo-de-la-Peña MT, Baillès E, Caseras X, Martínez A, Ortet G, Pérez J. Adv Health Sci Educ Theory Pract. 2009;14:61–67. - PubMed
  5.  
    1. Assessment and learning: differences and relationships between formative and summative assessment. Harlen W, James M. https://doi.org/10.1080/0969594970040304 Assess Educ: Princ Policy Pract. 1997;4:365–379.
  6.  
    1. Formative assessment: assessment is for self-regulated learning. Clark I. Educational Psychology Review. 2012;24:205–249.
  7.  
    1. Formative assessment: revisiting the territory. Sadler DR. https://doi.org/10.1080/0969595980050104 Assess Educ: Princ Policy Pract. 1998;5:77–84.
  8.  
    1. Developing the theory of formative assessment. Black P, Wiliam D. https://doi.org/10.1007/s11092-008-9068-5 Educational Assessment, Evaluation and Accountability (formerly: Journal of Personnel Evaluation in Education) 2009;21:5–31.
  9.  
    1. The concept of formative assessment. Boston C. https://doi.org/10.7275/kmcq-dj31 Practical Assessment, Research, and Evaluation. 2002;8:9.
  10.  
    1. Setting standards for the assessment of operative competence. Beard JD. Eur J Vasc Endovasc Surg. 2005;30:215–218. - PubMed
  11.  
    1. Use of the operative logbook to monitor trainee progress, and evaluate operative supervision provided by accredited training posts. Beasley SW, McBride C, Pearson ML. Surgeon. 2011;9:0–5. - PubMed
  12.  
    1. Development and evaluation of a new formative assessment of surgical performance. Davies RM, Hadfield-Law L, Turner PG. https://doi.org/10.1016/j.jsurg.2018.02.004. J Surg Educ. 2018;75:1309–1316. - PubMed
  13.  
    1. Getting lost in translation? Workplace based assessments in surgical training. Ali JM. Surgeon. 2013;11:286–289. - PubMed
  14.  
    1. Acceptability and feasibility of mini-clinical evaluation exercise as a formative assessment tool for workplace-based assessment for surgical postgraduate students. Joshi MK, Singh T, Badyal DK. J Postgrad Med. 2017;63:100–105. - PMC - PubMed
  15.  
    1. What evidence is there for the use of workplace-based assessment in surgical training? Shalhoub J, Vesey AT, Fitzgerald JE. J Surg Educ. 2014;71:906–915. - PubMed
  16.  
    1. How good a surgeon are you? - standardized formative assessment of surgical competence for ophthalmology residents in training. Honavar SG. Indian J Ophthalmol. 2017;65:777–778. - PMC - PubMed
  17.  
    1. Formative assessment in undergraduate medical education: concept, implementation and hurdles. Rauf A, Shamim MS, Aly SM, Chundrigar T, Alam SN. http://www.jpma.org.pk/article-details/5756. J Pak Med Assoc. 2014;64:72–75. - PubMed
  18.  
    1. Welcome to the preferred reporting items for systematic reviews and meta-analyses (PRISMA) website! [ Jul; 2022 ];https://prisma-statement.org/ PRISMA. 2015 1:2021.
  19.  
    1. Program director perceptions of usefulness of the accreditation council for graduate medical education milestones system for urology resident evaluation. Sebesta EM, Cooper KL, Badalato GM. https://doi.org/10.1016/j.urology.2018.10.042. Urology. 2019;124:28–32. - PubMed
  20.  
    1. Development and implementation of a mobile version of the O-SCORE assessment tool and case log for competency-based assessment in urology residency training: an initial assessment of utilization and acceptance among residents and faculty. Fitzpatrick R, Paterson NR, Watterson J, Seabrook C, Roberts M. Can Urol Assoc J. 2019;13:45–50. - PMC - PubMed
  21.  
    1. Non-technical skills for urological surgeons (NoTSUS): development and evaluation of curriculum and assessment scale. Aydın A, Griffin CM, Brunckhorst O, et al. https://doi.org/10.1007/s00345-020-03406-6. World J Urol. 2021;39:2231–2237. - PMC - PubMed
  22.  
    1. On line clinical reasoning assessment with Script Concordance test in urology: results of a French pilot study. Sibert L, Darmoni SJ, Dahamna B, Hellot MF, Weber J, Charlin B. BMC Med Educ. 2006;6:45. - PMC - PubMed
  23.  
    1. High acceptability of a newly developed urological practical skills training program. de Vries AH, van Luijk SJ, Scherpbier AJ, Hendrikx AJ, Koldewijn EL, Wagner C, Schout BM. BMC Urol. 2015;15:93. - PMC - PubMed
  24.  
    1. Toward an objective assessment of technical skills: a national survey of surgical program directors in Saudi Arabia. Alkhayal A, Aldhukair S, Alselaim N, et al. Adv Med Educ Pract. 2012;3:97–104. - PMC - PubMed
  25.  
    1. Effectiveness of direct observation of procedural skills (DOPS) in postgraduate training in urology at institute of kidney diseases, Peshawar. Ali L, Ali S, Orakzai N, Ali N. J Coll Physicians Surg Pak. 2019;29:516–519. - PubMed
  26.  
    1. Quality of feedback using workplace based assessments in urological training. Ali AS, Bussey M, O'flynn KJ, Eardley I. Br J Med Surg Urol. 2012;5:39–43.
  27.  
    1. Assessing clinical reasoning skills using script concordance test (SCT) and extended matching questions (EMQs): a pilot for urology trainees. Nazim SM, Talati JJ, Pinjani S, Biyabani SR, Ather MH, Norcini JJ. J Adv Med Educ Prof. 2019;7:7–13. - PMC - PubMed
  28.  
    1. The Ottawa surgical competency operating room evaluation (O-SCORE): a tool to assess surgical competence. Gofton WT, Dudek NL, Wood TJ, Balaa F, Hamstra SJ. Acad Med. 2012;87:1401–1407. - PubMed
  29.  
    1. Continued validation of the O-SCORE (Ottawa surgical competency operating room evaluation): use in the simulated environment. MacEwan MJ, Dudek NL, Wood TJ, Gofton WT. Teach Learn Med. 2016;28:72–79. - PubMed
  30.  
    1. 4-year curriculum for urology residency training. McDougall EM, Watters TJ, Clayman RV. J Urol. 2007;178:2540–2544. - PubMed
  31.  
    1. Work-based assessments: making the transition from participation to engagement. Swayamprakasam AP, Segaran A, Allery L. JRSM Open. 2014;26:2042533313515861. - PMC - PubMed
  32.  
    1. Workplace-based assessments in Wessex: the first 6 months. Babu KS, Htike MM, Cleak VE. Psychiatric Bulletin. 2009;33:474–478.
  33.  
    1. Workplace-based assessment: survey of psychiatric trainees in Wales. Menon S, Winston M, Sullivan G. Psychiatric Bulletin. 2009;33:468–474.
  34.  
    1. Key attributes of a modern surgical trainer: perspectives from consultants and trainees in the United kingdom. Nisar PJ, Scott HJ. J Surg Educ. 2011;68:202–208. - PubMed
  35.  
    1. Training in minimally invasive surgery in urology: European Association of Urology/International Consultation of Urological Diseases consultation. van der Poel H, Brinkman W, van Cleynenbreugel B, et al. BJU Int. 2016;117:515–530. - PubMed
  36.  
    1. Effectiveness of direct observation of procedural skills (DOPS) as an assessment tool in surgery. Khaliq T. http://www.pafmj.org/index.php/PAFMJ/article/view/1664 PAFMJ. 2014;31:626–629.
  37.  
    1. Use of portfolios as a learning and assessment tool in a surgical practical session of urology during undergraduate medical training. Amsellem-Ouazana D, Pee DV, Godin V. https://doi.org/10.1080/01421590600628100. Med Teach. 2006;28:356–359. - PubMed
  38.  
    1. Is the script concordance test a valid instrument for assessment of intraoperative decision-making skills? Meterissian S, Zabolotny B, Gagnon R, Charlin B. Am J Surg. 2007;193:248–251. - PubMed
  39.  
    1. Threats to validity in the use and interpretation of script concordance test scores. Lineberry M, Kreiter CD, Bordage G. Med Educ. 2013;47:1175–1183. - PubMed
  40.  
    1. Non-technical skills for surgeons (NOTSS) [ Jul; 2022 ];https://www.rcsed.ac.uk/professional-support-development-resources/learn... Surgeons of Edinburgh. 2021

Page Navigation