Michael J. Cullen

Michael J. Cullen

Auto-generated profile, based on publicly available information

Michael J. Cullen

Graduate Medical Education, University of Minnesota Medical School

Research Focus & Metrics

Academic Career Length

34Years (1993-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
19 May 2020 in Teaching and Learning in Medicine
Verified
Journal Article
Closed access
MCMichael J. Cullen
Michael J. Cullen
CZCharlene Zhang
Charlene Zhang
BMBrittany Marcus-Blank
Brittany Marcus-Blank
JBJonathan P. Braman
Jonathan P. Braman
ETEzgi Tiryaki
Ezgi Tiryaki
JAJohn S. Andrews
John S. Andrews
Construct: We investigated whether a situational judgment test (SJT) designed to measure professionalism in physicians predicts residents’ performance on (a) Accreditation Council for Graduate Medical Education (ACGME) competencies and (b) a multisource professionalism assessment (MPA). Background: There is a consensus regarding the importance of assessing professionalism and interpersonal and communication skills in medical students, residents, and practicing physicians. Nonetheless, these noncognitive competencies are not well measured during medical education selection processes. One promising method for measuring these noncognitive competencies is the SJT. In a typical SJT, respondents are presented with written or video-based scenarios and asked to make choices from a set of alternative courses of action. Interpersonally oriented SJTs are commonly used for selection to medical schools in the United Kingdom and Belgium and for postgraduate selection of trainees to medical practice in Belgium, Singapore, Canada, and Australia. However, despite international evidence suggesting that SJTs are useful predictors of in-training performance, end-of-training performance, supervisory ratings of performance, and clinical skills licensing objective structured clinical examinations, the use of interpersonally oriented SJTs in residency settings in the United States has been infrequently investigated. The purpose of this study was to investigate whether residents’ performance on an SJT designed to measure professionalism-related competencies—conscientiousness, integrity, accountability, aspiring to excellence, teamwork, stress tolerance, and patient-centered care—predicts both their current and future performance as residents on two important but conceptually distinct criteria: ACGME competencies and the MPA. Approach: We developed an SJT to measure seven dimensions of professionalism. During calendar year 2017, 21 residency programs from 2 institutions administered the SJT. We conducted analyses to determine the validity of SJT and USMLE scores in predicting milestone performance in ACGME core competency domains and the MPA in June 2017 and 3 months later in September 2017 for the MPA and 1 year later, in June 2018, for ACGME domains. Results: At both periods, the SJT score predicted overall ACGME milestone performance (r = .13 and .17, respectively; p < .05) and MPA performance (r = .19 and .21, respectively; p < .05). In addition, the SJT predicted ACGME patient care, systems-based practice, practice-based learning and improvement, interpersonal and communication skills, and professionalism competencies (r = .16, .15, .15, .17, and .16, respectively; p < .05) 1 year later. The SJT score contributed incremental validity over USMLE scores in predicting overall ACGME milestone performance (ΔR = .07) 1 year later and MPA performance (ΔR = .05) 3 months later. Conclusions: SJTs show promise as a method for assessing noncognitive attributes in residency program applicants. The SJT’s incremental validity to the USMLE series in this study underscores the importance of moving beyond these standardized tests to a more holistic review of candidates that includes both cognitive and noncognitive measures.

27 September 2016 in Medical Teacher
Verified
Journal Article
Closed access
MCMichael J. Cullen
Michael J. Cullen
MKMojca R. Konia
Mojca R. Konia
EBEmily C. Borman-Shoap
Emily C. Borman-Shoap
JBJonathan P. Braman
Jonathan P. Braman
ETEzgi Tiryaki
Ezgi Tiryaki
JAJohn S. Andrews
John S. Andrews
Introduction: Professionalism is a key component of medical education and training. However, there are few tools to aid educators in diagnosing unprofessional behavior at an early stage. The purpose of this study was to employ policy capturing methodology to develop two empirically validated checklists for identifying professionalism issues in early-career physicians. Method: In a series of workshops, a professionalism competency model containing 74 positive and 70 negative professionalism behaviors was developed and validated. Subsequently, 23 subject matter experts indicated their level of concern if each negative behavior occurred 1, 2, 3, 4, or 5 or more times during a six-month period. These ratings were used to create a “brief” and “extended” professionalism checklist for monitoring physician misconduct. Results: This study confirmed the subjective impression that some unprofessional behaviors are more egregious than others. Fourteen negative behaviors (e.g. displaying obvious signs of substance abuse) were judged to be concerning if they occurred only once, whereas many others (e.g. arriving late for conferences) were judged to be concerning only when they occurred repeatedly. Discussion: Medical educators can use the professionalism checklists developed in this study to aid in the early identification and subsequent remediation of unprofessional behavior in medical students and residents.

1 March 2022 in Journal of Surgical Education
Verified
Journal Article
Closed access
JWJillian Wothe
Jillian Wothe
LBLaura Bosacker
Laura Bosacker
HNHarika Nalluri
Harika Nalluri
MCMichael J. Cullen
Michael J. Cullen
MBMelissa E. Brunsvold
Melissa E. Brunsvold

24 May 2022 in Academic Medicine
Verified
Journal Article
Closed access
MCMichael J. Cullen
Michael J. Cullen
CZCharlene Zhang
Charlene Zhang
PSPaul R. Sackett
Paul R. Sackett
KTKrima Thakker
Krima Thakker
JYJohn Q. Md Young
John Q. Md Young
Purpose: To determine whether overall situational judgment test (SJT) scores are associated with programs’ clinical competency committee (CCC) ratings of trainee professionalism, any concerning behavior, and concerning behavior requiring active remediation at 2 time periods. Method: In fall 2019, trainees from 17 U.S. programs (16 residency, 1 fellowship) took an online 15-scenario SJT developed to measure 7 dimensions of professionalism. CCC midyear and year-end (6 months and 1 year following SJT completion, respectively) professionalism scores and concern ratings were gathered for academic year 2019–2020. Analyses were conducted to determine whether overall SJT scores related to overall professionalism ratings, trainees displaying any concerns, and trainees requiring active remediation at both time periods. Results: Overall SJT scores correlated positively with midyear and year-end overall professionalism ratings ( r = .21 and .14, P < .001 and = .03, respectively). Holding gender and race/ethnicity constant, a 1 standard deviation (SD) increase in overall SJT score was associated with a .20 SD increase in overall professionalism ratings at midyear ( P = .005) and a .22 SD increase at year-end ( P = .001). Holding gender and race/ethnicity constant, a 1 SD increase in overall SJT score decreased the odds of a trainee displaying any concerns by 37% (odds ratio [OR] 95% confidence interval [CI]: [.44, .87], P = .006) at midyear and 34% (OR 95% CI: [.46, .95], P = .025) at year-end and decreased the odds of a trainee requiring active remediation by 51% (OR 95% CI: [.25, .90], P = .02) at midyear. Conclusions: Overall SJT scores correlated positively with midyear and year-end overall professionalism ratings and were associated with whether trainees exhibited any concerning behavior at midyear and year-end and whether trainees needed active remediation at midyear. Future research should investigate whether other potential professionalism measures are associated with concerning trainee behavior.

22 September 2023 in Academic Medicine
Verified
Journal Article
Closed access
MCMichael J. Cullen
Michael J. Cullen
YZYou Zhou
You Zhou
PSPaul R. Sackett
Paul R. Sackett
TMTaj Mustapha
Taj Mustapha
JHJessica Hane
Jessica Hane
SPSusan M. Pruett
Susan M. Pruett
Purpose: To examine whether gender differences exist in medical trainees’ (residents’ and fellows’) evaluations of faculty at a number of clinical departments. Method: The authors conducted a single institution (University of Minnesota Medical School) retrospective cohort analysis of 5,071 trainee evaluations of 447 faculty (for which trainee and faculty gender information was available) completed between July 1, 2019, and June 30, 2022. The authors developed and employed a 17-item measure of clinical teaching effectiveness, with 4 dimensions: overall teaching effectiveness, role modeling, facilitating knowledge acquisition, and teaching procedures. Using both between- and within-subject samples, they conducted analyses to examine gender differences among the trainees making ratings (rater effects), the faculty receiving ratings (ratee effects), and whether faculty ratings differed by trainee gender (interaction effects). Results: There was a statistically significant rater effect for the overall teaching effectiveness and facilitating knowledge acquisition dimensions (B = –0.28 and –0.14, 95% CI: [–0.35, –0.21] and [–0.20, –0.09], respectively, P < .001, medium corrected effect sizes between –0.34 and –0.54); female trainees rated male and female faculty lower than male trainees on both dimensions. There also was a statistically significant ratee effect for the overall teaching effectiveness and role modeling dimensions (B = –0.09 and –0.08, 95% CI: [–0.16, –0.02] and [–0.13, –0.04], P = .01 and < .001, respectively, small to medium corrected effect sizes between –0.16 and –0.44); female faculty were rated lower than male faculty on both dimensions. There was not a statistically significant interaction effect. Conclusions: Female trainees rated faculty lower than male trainees and female faculty were rated lower than male faculty on 2 teaching dimensions each. The authors encourage researchers to continue to examine the reasons for the evaluation differences observed and how implicit bias interventions might help to address them.

26 May 2021 in International Journal of Selection and Assessment
Verified
Journal Article
Closed access
PDF file
CZCharlene Zhang
Charlene Zhang
MCMichael J. Cullen
Michael J. Cullen
PSPaul R. Sackett
Paul R. Sackett