Background: As the adoption of artificial intelligence (AI) in health care increases, it will become increasingly crucial to involve health care professionals (HCPs) in developing, validating, and implementing AI-enabled technologies. However, because of a lack of AI literacy, most HCPs are not adequately prepared for this revolution. This is a significant barrier to adopting and implementing AI that will affect patients. In addition, the limited existing AI education programs face barriers to development and implementation at various levels of medical education. Objective: With a view to informing future AI education programs for HCPs, this scoping review aims to provide an overview of the types of current or past AI education programs that pertains to the programs’ curricular content, modes of delivery, critical implementation factors for education delivery, and outcomes used to assess the programs’ effectiveness. Methods: After the creation of a search strategy and keyword searches, a 2-stage screening process was conducted by 2 independent reviewers to determine study eligibility. When consensus was not reached, the conflict was resolved by consulting a third reviewer. This process consisted of a title and abstract scan and a full-text review. The articles were included if they discussed an actual training program or educational intervention, or a potential training program or educational intervention and the desired content to be covered, focused on AI, and were designed or intended for HCPs (at any stage of their career). Results: Of the 10,094 unique citations scanned, 41 (0.41%) studies relevant to our eligibility criteria were identified. Among the 41 included studies, 10 (24%) described 13 unique programs and 31 (76%) discussed recommended curricular content. The curricular content of the unique programs ranged from AI use, AI interpretation, and cultivating skills to explain results derived from AI algorithms. The curricular topics were categorized into three main domains: cognitive, psychomotor, and affective. Conclusions: This review provides an overview of the current landscape of AI in medical education and highlights the skills and competencies required by HCPs to effectively use AI in enhancing the quality of care and optimizing patient outcomes. Future education efforts should focus on the development of regulatory strategies, a multidisciplinary approach to curriculum redesign, a competency-based curriculum, and patient-clinician interaction.

Feedback and debriefing are experience-informed dialogues upon which experiential models of learning often depend. Efforts to understand each have largely been independent of each other, thus splitting them into potentially problematic and less productive factions. Given their shared purpose of improving future performance, the authors asked whether efforts to understand these dialogues are, for theoretical and pragmatic reasons, best advanced by keeping these concepts unique or whether some unifying conceptual framework could better support educational contributions and advancements in medical education. The authors identified seminal works and foundational concepts to formulate a purposeful review and analysis exploring these dialogues’ theoretical roots and their manifestations. They considered conceptual and theoretical details within and across feedback and debriefing literatures and traced developmental paths to discover underlying and foundational conceptual approaches and theoretical similarities and differences. Findings suggest that each of these strategies was derived from distinct theoretical roots, leading to variations in how they have been studied, advanced, and enacted; both now draw on multiple (often similar) educational theories, also positioning themselves as ways of operationalizing similar educational frameworks. Considerable commonality now exists; those studying and advancing feedback and debriefing are leveraging similar cognitive and social theories to refine and structure their approaches. As such, there may be room to merge these educational strategies as learning conversations because of their conceptual and theoretical consistency. Future scholarly work should further delineate the theoretical, educational, and practical relevance of integrating feedback and debriefing.

Objective. The aim of this study was to develop and critically appraise a global rating scale (GRS) for the assessment of individual paramedic clinical competence at the entry-to-practice level. Methods. The development phase of this study involved task analysis by experts, contributions from a focus group, and a modified Delphi process using a national expert panel to establish evidence of content validity. The critical appraisal phase had two raters apply the GRS, developed in the first phase, to a series of sample performances from three groups: novice paramedic students (group 1), paramedic students at the entry-to-practice level (group 2), and experienced paramedics (group 3). Using data from this process, we examined the tool's reliability within each group and tested the discriminative validity hypothesis that higher scores would be associated with higher levels of training and experience. Results. The development phase resulted in a seven-dimension, seven-point adjectival GRS. The two independent blinded raters scored 81 recorded sample performances (n = 25 in group 1, n = 33 in group 2, n = 23 in group 3) using the GRS. For groups 1, 2, and 3, respectively, interrater reliability reached 0.75, 0.88, and 0.94. Intrarater reliability reached 0.94 and the internal consistency ranged from 0.53 to 0.89. Rater differences contributed 0–5.7% of the total variance. The GRS scores assigned to each group increased with level of experience, both using the overall rating (means = 2.3, 4.1, 5.0; p < 0.001) and considering each dimension separately. Applying a modified borderline group method, 54.9% of group 1, 13.4% of group 2, and 2.9% of group 3 were below the cut score. Conclusion. The results of this study provide evidence that the scores generated using this scale can be valid for the purpose of making decisions regarding paramedic clinical competence. Key words: educational measurement; clinical competence; licensure; certification; paramedics; global rating scale; rating scales

Paramedicine is a rapidly evolving health profession with increasing responsibilities and contributions to healthcare. This rapid growth has left the profession with unclear professional and clinical boundaries. Existing defining frameworks may no longer align with the practice of paramedicine or expectations of the public. The purpose of this study was to explore the roles paramedics in Canada are to embody and that align with or support the rapid and ongoing evolution of the profession. We used a concurrent mixed methods study design involving a focused discourse analysis (i.e., analysis of language used to describe paramedics and paramedicine) of peer reviewed and grey literature (Phase 1) and in-depth one-on-one semi-structured interviews with key informants in Canadian paramedicine (Phase 2). Data from both methods were analyzed simultaneously throughout and after being merged using inductive thematic analysis. Saturation was reached after 99 national and international grey and peer reviewed publications and 20 in depth interviews with stakeholders representing six provinces, seven different service/agency types, 11 operational roles and seven provider roles. After merging both data sets three framing concepts, six roles and four crosscutting themes emerged that may be significant to both present-day practice and aspirational. Framing concepts, which provide context, include variable contexts or practice, embedded relationships and a health and social continuum. Roles include clinician, health and social advocate, team member, educator, professional and reflective practitioner. Crosscutting themes including patient safety, adaptability, compassion and communication appear to exist in all roles. The paramedic profession is experiencing a shift that appears to deviate or at least place a tension on traditional views or models of practice. Underlying and evolving notions of practice are resulting in intended or actual clinical and professional boundaries that may require the profession to re-think how it is defined and/or shaped. Until these framing concepts, roles and crosscutting themes are fully understood, tested and operationalized, tensions between guiding frameworks and actual or intended practice may persist.

Theory: Assessment of clinical competence is a complex cognitive task with many mental demands often imposed on raters unintentionally. We were interested in whether this burden might contribute to well-described limitations in assessment judgments. In this study we examine the effect on indicators of rating quality of asking raters to (a) consider multiple competencies and (b) attend to multiple issues. In addition, we explored the cognitive strategies raters engage when asked to consider multiple competencies simultaneously. Hypotheses: We hypothesized that indications of rating quality (e.g., interrater reliability) would decline as the number of dimensions raters are expected to consider increases. Method: Experienced faculty examiners rated prerecorded clinical performances within a 2 (number of dimensions) × 2 (presence of distracting task) × 3 (number of videos) factorial design. Half of the participants were asked to rate 7 dimensions of performance (7D), and half were asked to rate only 2 (2D). The second factor involved the requirement (or lack thereof) to rate the performance of actors participating in the simulation. We calculated the interrater reliability of the scores assigned and counted the number of relevant behaviors participants identified as informing their ratings. Second, we analyzed data from semistructured posttask interviews to explore the rater strategies associated with rating under conditions designed to broaden raters' focus. Results: Generalizability analyses revealed that the 2D group achieved higher interrater reliability relative to the 7D group (G = .56 and .42, respectively, when the average of 10 raters is calculated). The requirement to complete an additional rating task did not have an effect. Using the 2 dimensions common to both groups, an analysis of variance revealed that participants who were asked to rate only 2 dimensions identified more behaviors of relevance to the focal dimensions than those asked to rate 7 dimensions: procedural skill = 36.2%, 95% confidence interval (CI) [32.5, 40.0] versus 23.5%, 95% CI [20.8, 26.3], respectively; history gathering = 38.6%, 95% CI [33.5, 42.9] versus 24.0%, 95% CI [21.1, 26.9], respectively; ps < .05. During posttask interviews, raters identified many sources of cognitive load and idiosyncratic cognitive strategies used to reduce cognitive load during the rating task. Conclusions: As intrinsic rating demands increase, indicators of rating quality decline. The strategies that raters engage when asked to rate many dimensions simultaneously are varied and appear to yield idiosyncratic efforts to reduce cognitive effort, which may affect the degree to which raters make judgments based on comparable information.