Background : Direct observation of clinical skills is a cornerstone of competency-based education and training. Ensuring direct observation in a consistent fashion has been a significant challenge for residency programs. Objective : The purpose of this study was to evaluate the effects of a novel evaluation system, designed to achieve ongoing direct observation of residents, examine changes in resident observation practices, and understand faculty attitudes toward direct observation and the evaluation system. Methods : Internal medicine residents on an ambulatory block rotation participated in a new evaluation system, which replaced a single end-of-rotation summative evaluation with 9 formative evaluations based on direct observation. Faculty received training in direct observation and use of the forms, and residents were given responsibility to collect 9 observations per rotation. Faculty members contacted residents at the beginning and middle of the rotation to ensure completion of the observations. Residents and faculty also completed postrotation surveys to gauge the impact of the new system. Results : A total of 507 patient encounters were directly observed, and 52 of 57 (91%) residents completed all 9 observations. Residents reported considerably more direct observation than prior to the intervention, and most reported changes to their clinical skills based on faculty feedback. Faculty reported improvements in their attitudes, increased their use of direct observation, and preferred the new system to the old one. Conclusions : A novel evaluation system replacing summative evaluations with multiple formative evaluations based on direct observation was successful in achieving high rates of observations, and improving faculty attitudes toward direct observation.
1 August 2023 in Journal of Graduate Medical Education
Background Despite growing interest in incorporating holistic review within residency admissions, implementation by residency programs remains challenging. Objective To incorporate holistic review into the internal medicine residency program at the University of Wisconsin and to report initial feasibility and acceptability data. Methods During the 2020-2021 application cycle, residency stakeholders performed a consensus-driven process to identify highly valued applicant attributes. We used a holistic review process to identify the presence of these attributes among applicants and updated our rank list algorithm to incorporate these attributes. We modified our interview screening criteria and rank list algorithm to de-emphasize other metrics. We surveyed stakeholders to assess time required for this process and compared our final rank list to what it would have been using our prior system. Results The final list of 10 prioritized applicant attributes included extraordinary leadership, community service, and grit, among others. Among 25 matched residents, 8 (32%) were recognized to have exceptional achievement within one of these 10 attributes. Four members of the incoming intern class (16%) would have been in a rank position lower than our historical matched resident cutoff had they not received additional points for these attributes. Faculty reported that holistic review of applications took an additional 3.8 minutes on average. It was felt that current application materials limit the ability to implement a fully holistic review. Conclusions The addition of holistic review to our residency admissions process was achieved using a consensus-driven approach and showed favorable feasibility and acceptability data.
Junior faculty are often called upon to deliver high-stakes large-group presentations. Training in the skills needed to do this effectively is often lacking. We devised a 1.25 h coaching program. The coach analyzed a practice run of the presentation using a locally developed assessment tool. Areas covered included public speaking skills, promoting learner understanding and retention, creating a dynamic learning climate, and optimal use of slides. COVID-19 necessitated a switch to virtual coaching, and we studied the impact of virtual vs. in-person coaching. We added two additional coaches and studied the transferability of the coaching component. There was high uptake of the offered coaching. Participant surveys showed improved comfort levels with large-group presentations; had a sense that their presentation skills had improved; showed an increased likelihood of volunteering for future speaking opportunities; and were likely to recommend the program. Comparisons between virtual and in-person coaching showed no statistical difference, and there was little difference between the original coach and the subsequent two coaches. Qualitative assessments revealed broad areas in which faculty felt the program had been most impactful. The coaching program was well-received and resulted in concrete positive changes in presenter behavior. Conducting the coaching in a virtual manner may increase the feasibility of the intervention, as would expanding the coach pool.
The tears came without warning, as an image appeared in my head, unbidden, of one of my final days with Grumpa. My family and I had come to Kewanee a few months prior to my grandfather's deterioration. Living in the city had been getting to be too much: three kids with another one planned, shoveling out the car after another Chicago blizzard, Cubs fans shouting outside our door in a postgame delirium, tiny backyard—little Ben's home runs over the fence becoming very routine, which does take away some of the excitement. The suburbs meant doubling the commute and doubling the mortgage. Were there any other options? We finally considered Kewanee—population 12 000, Hog Capital of the World, where my mom had been born and raised, where my elderly grandparents still lived. Two and a half hours from Chicago, nestled among the cornfields, it seemed to us quiet, and safe, and simple.