Jocelyn Lockyer

Jocelyn Lockyer

Auto-generated profile, based on publicly available information

Jocelyn Lockyer

University of Calgary

Research Focus & Metrics

Academic Career Length

45Years (1982-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
3 June 2017 in Medical Teacher
Verified
Journal Article
Closed access
JLJocelyn Lockyer
Jocelyn Lockyer
CCCarol Carraccio
Carol Carraccio
MCMing-Ka Chan
Ming-Ka Chan
DHDanielle Hart
Danielle Hart
SSSydney Smee
Sydney Smee
OCOn Behalf Of The Icbme Collaborators
On Behalf Of The Icbme Collaborators
The meaningful assessment of competence is critical for the implementation of effective competency-based medical education (CBME). Timely ongoing assessments are needed along with comprehensive periodic reviews to ensure that trainees continue to progress. New approaches are needed to optimize the use of multiple assessors and assessments; to synthesize the data collected from multiple assessors and multiple types of assessments; to develop faculty competence in assessment; and to ensure that relationships between the givers and receivers of feedback are appropriate. This paper describes the core principles of assessment for learning and assessment of learning. It addresses several ways to ensure the effectiveness of assessment programs, including using the right combination of assessment methods and conducting careful assessor selection and training. It provides a reconceptualization of the role of psychometrics and articulates the importance of a group process in determining trainees’ progress. In addition, it notes that, to reach its potential as a driver in trainee development, quality care, and patient safety, CBME requires effective information management and documentation as well as ongoing consideration of ways to improve the assessment system.

1 January 2007 in Journal of Continuing Education in the Health Professions
Verified
Journal Article
Closed access
PDF file
MMMaliheh Mansouri
Maliheh Mansouri
JLJocelyn Lockyer
Jocelyn Lockyer
Introduction: We undertook a meta‐analysis of the Continuing Medical Education (CME) outcome literature to examine the effect of moderator variables on physician knowledge, performance, and patient outcomes. Methods: A literature search of MEDLINE and ERIC was conducted for randomized controlled trials and experimental design studies of CME outcomes in which physicians were a major group. CME moderator variables included the types of intervention, the types and number of participants, time, and the number of intervention sessions held over time. Results: Thirty‐one studies met the eligibility criteria, generating 61 interventions. The overall sample‐size weighted effect size for all 61 interventions was r = 0.28 (0.18). The analysis of CME moderator variables showed that active and mixed methods had medium effect sizes (r = 0.33 [0.33], r = 0.33 [0.26], respectively), and passive methods had a small effect size (r = 0.20 [0.16], confidence interval 0.15, 0.26). There was a positive correlation between the effect size and the length of the interventions (r = 0.33) and between multiple interventions over time (r = 0.36). There was a negative correlation between the effect size and programs that involved multiple disciplines (r = −0.18) and the number of participants (r = −0.13). The correlation between the effect size and the length of time for outcome assessment was negative (r = −0.31). Discussion: The meta‐analysis suggests that the effect size of CME on physician knowledge is a medium one; however, the effect size is small for physician performance and patient outcome. The examination of moderator variables shows there is a larger effect size when the interventions are interactive, use multiple methods, and are designed for a small group of physicians from a single discipline.

6 April 2011 in Advances in Health Sciences Education
Verified
Journal Article
Open access
PDF file
KEKevin W. Eva
Kevin W. Eva
HAHeather Armson
Heather Armson
EHEric Holmboe
Eric Holmboe
JLJocelyn Lockyer
Jocelyn Lockyer
ELElaine Loney
Elaine Loney
JSJoan Sargeant
Joan Sargeant
Self-appraisal has repeatedly been shown to be inadequate as a mechanism for performance improvement. This has placed greater emphasis on understanding the processes through which self-perception and external feedback interact to influence professional development. As feedback is inevitably interpreted through the lens of one’s self-perceptions it is important to understand how learners interpret, accept, and use feedback (or not) and the factors that influence those interpretations. 134 participants from 8 health professional training/continuing competence programs were recruited to participate in focus groups. Analyses were designed to (a) elicit understandings of the processes used by learners and physicians to interpret, accept and use (or not) data to inform their perceptions of their clinical performance, and (b) further understand the factors (internal and external) believed to influence interpretation of feedback. Multiple influences appear to impact upon the interpretation and uptake of feedback. These include confidence, experience, and fear of not appearing knowledgeable. Importantly, however, each could have a paradoxical effect of both increasing and decreasing receptivity. Less prevalent but nonetheless important themes suggested mechanisms through which cognitive reasoning processes might impede growth from formative feedback. Many studies have examined the effectiveness of feedback through variable interventions focused on feedback delivery. This study suggests that it is equally important to consider feedback from the perspective of how it is received. The interplay observed between fear, confidence, and reasoning processes reinforces the notion that there is no simple recipe for the delivery of effective feedback. These factors should be taken into account when trying to understand (a) why self-appraisal can be flawed, (b) why appropriate external feedback is vital (yet can be ineffective), and (c) why we may need to disentangle the goals of performance improvement from the goals of improving self-assessment.

1 May 2016 in Academic Medicine
Verified
Journal Article
Closed access
CCCarol Carraccio
Carol Carraccio
RERobert Englander
Robert Englander
EMElaine Van Melle
Elaine Van Melle
OCOlle Ten Cate
Olle Ten Cate
JLJocelyn Lockyer
Jocelyn Lockyer
ICInternational Competency-Based Medical Education Collaborators
International Competency-Based Medical Education Collaborators
The International Competency-Based Medical Education (ICBME) Collaborators have been working since 2009 to promote understanding of competency-based medical education (CBME) and accelerate its uptake worldwide. This article presents a charter, supported by a literature-based rationale, which is meant to provide a shared mental model of CBME that will serve as a path forward in its widespread implementation. At a 2013 summit, the ICBME Collaborators laid the groundwork for this charter. Here, the fundamental principles of CBME and professional responsibilities of medical educators in its implementation process are described. The authors outline three fundamental principles: (1) Medical education must be based on the health needs of the populations served; (2) the primary focus of education and training should be the desired outcomes for learners rather than the structure and process of the educational system; and (3) the formation of a physician should be seamless across the continuum of education, training, and practice. Building on these principles, medical educators must demonstrate commitment to teaching, assessing, and role modeling the range of identified competencies. In the clinical setting, they must provide supervision that balances patient safety with the professional development of learners, being transparent with stakeholders about level of supervision needed. They must use effective and efficient assessment strategies and tools for basing transition decisions on competence rather than time in training, empowering learners to be active participants in their learning and assessment. Finally, advancing CBME requires program evaluation and research, faculty development, and a collaborative approach to realize its full potential.

1 July 2010 in Academic Medicine
Verified
Journal Article
Closed access
JSJoan Sargeant
Joan Sargeant
HAHeather Armson
Heather Armson
BCBen Chesluk
Ben Chesluk
TDTimothy Dornan
Timothy Dornan
KEKevin Eva
Kevin Eva
CVCees van der Vleuten
Cees van der Vleuten
To determine how learners and physicians engaged in various structured interventions to inform self-assessment, how they perceived and used self-assessment in clinical learning and practice, and the components and processes comprising informed self-assessment and factors that influence these. This was a qualitative study guided by principles of grounded theory. Using purposive sampling, eight programs were selected in Canada, the United States, the United Kingdom, the Netherlands, and Belgium, representing low, medium, and high degrees of structure/rigor in self-assessment activities. In 2008, 17 focus groups were conducted with 134 participants (53 undergraduate learners, 32 postgraduate learners, 49 physicians). Focus-group transcripts were analyzed interactively and iteratively by the research team to identify themes and compare and confirm findings. Informed self-assessment appeared as a flexible, dynamic process of accessing, interpreting, and responding to varied external and internal data. It was characterized by multiple tensions arising from complex interactions among competing internal and external data and multiple influencing conditions. The complex process was evident across the continuum of medical education and practice. A conceptual model of informed self-assessment emerged. Central challenges to informing self-assessment are the dynamic interrelationships and underlying tensions among the components comprising self-assessment. Realizing this increases understanding of why self-assessment accuracy seems frequently unreliable. Findings suggest the need for attention to the varied influencing conditions and inherent tensions to progress in understanding self-assessment, how it is informed, and its role in self-directed learning and professional self-regulation. Informed self-assessment is a multidimensional, complex construct requiring further research.

1 December 2015 in Academic Medicine
Verified
Journal Article
Closed access
JSJoan Sargeant
Joan Sargeant
JLJocelyn Lockyer
Jocelyn Lockyer
KMKaren Mann
Karen Mann
EHEric Holmboe
Eric Holmboe
ISIvan Md Silver
Ivan Md Silver
MPMary Mba Power
Mary Mba Power
Purpose To develop and conduct feasibility testing of an evidence-based and theory-informed model for facilitating performance feedback for physicians so as to enhance their acceptance and use of the feedback. Method To develop the feedback model (2011–2013), the authors drew on earlier research which highlights not only the factors that influence giving, receiving, accepting, and using feedback but also the theoretical perspectives which enable the understanding of these influences. The authors undertook an iterative, multistage, qualitative study guided by two recognized research frameworks: the UK Medical Research Council guidelines for studying complex interventions and realist evaluation. Using these frameworks, they conducted the research in four stages: (1) modeling, (2) facilitator preparation, (3) model feasibility testing, and (4) model refinement. They analyzed data, using content and thematic analysis, and used the findings from each stage to inform the subsequent stage. Results Findings support the facilitated feedback model, its four phases—build relationship, explore reactions, explore content, coach for performance change (R2C2)—and the theoretical perspectives informing them. The findings contribute to understanding elements that enhance recipients’ engagement with, acceptance of, and productive use of feedback. Facilitators reported that the model made sense and the phases generally flowed logically. Recipients reported that the feedback process was helpful and that they appreciated the reflection stimulated by the model and the coaching. Conclusions The theory- and evidence-based reflective R2C2 Facilitated Feedback Model appears stable and helpful for physicians in facilitating their reflection on and use of formal performance assessment feedback.