25 April 2024 in Annals of Surgery
Objective: To assess the quality of narrative feedback given to surgical residents during the first 5 years of competency-based medical education implementation. Background: Competency-based medical education requires ongoing formative assessments and feedback on learners’ performance. Methods: We conducted a retrospective cross-sectional study using assessments of entrustable professional activities (EPAs) in the Surgical Foundations curriculum at Queen’s University from 2017 to 2022. Two raters independently evaluated the quality of narrative feedback using the Quality of Assessment of Learning score (0–5). Results: A total of 3900 EPA assessments were completed over 5 years. Of assessments, 57% (2229/3900) had narrative feedback documented with a mean Quality of Assessment of Learning score of 2.16 ± 1.49. Of these, 1614 (72.4%) provided evidence about the resident’s performance, 951 (42.7%) provided suggestions for improvement, and 499/2229 (22.4%) connected suggestions to the evidence. There was no meaningful change in narrative feedback quality over time (r = 0.067, P = 0.002). Variables associated with lower quality of narrative feedback include: attending role (2.04 ± 1.48) compared with the medical student (3.13 ± 1.12, P < 0.001) and clinical fellow (2.47 ± 1.54, P < 0.001), concordant specialties between the assessor and learner (2.06 ± 1.50 vs 2.21 ± 1.49, P = 0.025), completion of the assessment 1 month or more after the encounter versus 1 week (1.85 ± 1.48 vs 2.23 ± 1.49, P < 0.001), and resident entrusted versus not entrusted to perform the assessed EPA (2.13 ± 1.45 vs 2.35 ± 1.66; P = 0.008). The quality of narrative feedback was similar for assessments completed under direct and indirect observation (2.18 ± 1.47 vs 2.06 ± 1.54; P = 0.153). Conclusions: Just over half of the EPA assessments of surgery residents contained narrative feedback with overall fair quality. There was no meaningful change in the quality of feedback over 5 years. These findings prompt future research and faculty development.
5 January 2024 in Perspectives on Medical Education
In competency-based medical education (CBME), which is being embraced globally, the patient-learner-educator encounter occurs in a highly complex context which contributes to a wide range of assessment outcomes. Current and historical barriers to considering context in assessment include the existing post-positivist epistemological stance that values objectivity and validity evidence over the variability introduced by context. This is most evident in standardized testing. While always critical to medical education the impact of context on assessment is becoming more pronounced as many aspects of training diversify. This diversity includes an expanding interest beyond individual trainee competence to include the interdependency and collective nature of clinical competence and the growing awareness that medical education needs to be co-produced among a wider group of stakeholders. In this Eye Opener, we wish to consider: 1) How might we best account for the influence of context in the clinical competence assessment of individuals in medical education? and by doing so, 2) How could we usher in the next era of assessment that improves our ability to meet the dynamic needs of society and all its stakeholders? The purpose of this Eye Opener is thus two-fold. First, we conceptualize - from a variety of viewpoints, how we might address context in assessment of competence at the level of the individual learner. Second, we present recommendations that address how to approach implementation of a more contextualized competence assessment.
16 December 2025 in Journal of Surgical Education
To investigate (1) the quantity and quality of verbal feedback provided to residents in surgery and anesthesiology in the operating room (OR), (2) the relationship between provided verbal feedback and assigned entrustment scores; and (3) residents' perceptions of usefulness of verbal feedback they received in the OR for a specific entrustable professional activity (EPA). This was a prospective observational study. An academic tertiary care hospital in Kingston, Ontario, Canada. Twenty residents (8 anesthesiology, 8 general surgery, 4 orthopedic surgery) and 23 attending physicians (11 anesthesiology, 9 general surgery, 3 orthopedic surgery) from Kingston Health Sciences Centre participated voluntarily. Participants were recruited using convenience sampling and no compensation was provided. We recorded 1180 verbal feedback events across 122 hours of OR interactions for 50 attending physician-resident dyads. Verbal feedback events focused predominantly on analyzing performance (57%) and fostering learner agency (20%). Verbal feedback was rated by residents as "extremely useful" or "very useful" over 70% of the time. Verbal feedback events that addressed a greater number of FQI domains were perceived to be more useful by residents. There were no significant correlations between verbal feedback event quantity or quality and entrustment scores on EPAs. Attending physicians frequently provide residents with verbal feedback on their performance in the OR; however, the quality of this feedback is variable. Future research should focus on developing strategies to capture and utilize verbal feedback as assessment for learning within CBME.