Journal of Medical Internet Research - International Scientific Journal for Medical Research, Information and Communication on the Internet #Preprint #PeerReviewMe: Warning: This is a unreviewed preprint. Readers are warned that the document has not been peer-reviewed by expert/patient reviewers or an academic editor, may contain misleading claims, and is likely to undergo changes before final publication, if accepted, or may have been rejected/withdrawn. Readers with interest and expertise are encouraged to sign up as peer-reviewer, if the paper is within an open peer-review period. Please cite this preprint only for review purposes or for grant applications and CVs (if you are the author). Background: Virtual Reality (VR) is a technology that allows the user to explore and manipulate computer-generated real or artificial three-dimensional (3D) multimedia sensory environments in real time to gain practical knowledge that can be used in clinical practice. Objective: The aim of this systematic review was to evaluate the effectiveness of virtual reality (VR) for educating health professions and improving their knowledge, cognitive skills, attitudes and satisfaction. Methods: We performed a systematic review of the effectiveness of virtual reality in pre- and post-registration health professions education following the gold standard Cochrane methodology. We searched seven databases from the year 1990 up to August 2017. No language restrictions were applied. We included randomized controlled trials and cluster-randomized trials. We independently selected studies, extracted data and assessed risk of bias, then compared the information in pairs. We contacted study authors for additional information if necessary. All pooled analyses were based on random-effects models. We used the GRADE approach to rate the quality of the body of evidence. Results: Thirty-one studies (N = 2407) were included. Meta-analysis of eight trials found that VR slightly improves post-intervention knowledge scores when compared with traditional learning (SMD = 0.44, 95% CI: 0.18, 0.69, I2 = 49%, N = 603) or to other types of digital education such as online or digital education (SMD = 0.43, 95% CI: 0.07, 0.79, I2 = 78%, N = 608, low certainty evidence). Another meta-analysis of four trials found that VR improves health professions cognitive skills when compared with traditional learning (SMD = 1.12, 95% CI: 0.81, 1.43, I2 = 0%, N = 235). Two studies compared VR to other forms of digital education on skills, favoring the VR group (SMD = 0.50, 95% CI: 0.32, 0.69, I2 = 0%, N = 467, moderate effect size, low certainty evidence). The findings for attitudes and satisfaction were mixed and inconclusive. No studies reported patient-related outcomes, behavior change or unintended/adverse effects of the VR. Overall, the certainty of evidence according to the GRADE criteria ranged from low to moderate. We downgraded our certainty of the evidence primarily because of the risk of bias and/or inconsistency. Conclusions: We found evidence suggesting that virtual reality improves post-intervention knowledge and skills outcomes of health professions when compared to traditional education or other types of digital education such as online or offline digital education. The findings on other outcomes are limited. Future research should evaluate more on the effectiveness of immersive and interactive forms of VR and evaluate other outcomes such as attitude, satisfaction, cost-effectiveness and clinical practice or behavior change.

Background: Conversational agents, also known as chatbots, are computer programs designed to simulate human text or verbal conversations. They are increasingly used in a range of fields, including health care. By enabling better accessibility, personalization, and efficiency, conversational agents have the potential to improve patient care. Objective: This study aimed to review the current applications, gaps, and challenges in the literature on conversational agents in health care and provide recommendations for their future research, design, and application. Methods: We performed a scoping review. A broad literature search was performed in MEDLINE (Medical Literature Analysis and Retrieval System Online; Ovid), EMBASE (Excerpta Medica database; Ovid), PubMed, Scopus, and Cochrane Central with the search terms “conversational agents,” “conversational AI,” “chatbots,” and associated synonyms. We also searched the gray literature using sources such as the OCLC (Online Computer Library Center) WorldCat database and ResearchGate in April 2019. Reference lists of relevant articles were checked for further articles. Screening and data extraction were performed in parallel by 2 reviewers. The included evidence was analyzed narratively by employing the principles of thematic analysis. Results: The literature search yielded 47 study reports (45 articles and 2 ongoing clinical trials) that matched the inclusion criteria. The identified conversational agents were largely delivered via smartphone apps (n=23) and used free text only as the main input (n=19) and output (n=30) modality. Case studies describing chatbot development (n=18) were the most prevalent, and only 11 randomized controlled trials were identified. The 3 most commonly reported conversational agent applications in the literature were treatment and monitoring, health care service support, and patient education. Conclusions: The literature on conversational agents in health care is largely descriptive and aimed at treatment and monitoring and health service support. It mostly reports on text-based, artificial intelligence–driven, and smartphone app–delivered conversational agents. There is an urgent need for a robust evaluation of diverse health care conversational agents’ formats, focusing on their acceptability, safety, and effectiveness.

Background: Digital health technologies can be key to improving health outcomes, provided health care workers are adequately trained to use these technologies. There have been efforts to identify digital competencies for different health care worker groups; however, an overview of these efforts has yet to be consolidated and analyzed. Objective: The review aims to identify and study existing digital health competency frameworks for health care workers and provide recommendations for future digital health training initiatives and framework development. Methods: A literature search was performed to collate digital health competency frameworks published from 2000. A total of 6 databases including gray literature sources such as OpenGrey, ResearchGate, Google Scholar, Google, and websites of relevant associations were searched in November 2019. Screening and data extraction were performed in parallel by the reviewers. The included evidence is narratively described in terms of characteristics, evolution, and structural composition of frameworks. A thematic analysis was also performed to identify common themes across the included frameworks. Results: In total, 30 frameworks were included in this review, a majority of which aimed at nurses, originated from high-income countries, were published since 2016, and were developed via literature reviews, followed by expert consultations. The thematic analysis uncovered 28 digital health competency domains across the included frameworks. The most prevalent domains pertained to basic information technology literacy, health information management, digital communication, ethical, legal, or regulatory requirements, and data privacy and security. The Health Information Technology Competencies framework was found to be the most comprehensive framework, as it presented 21 out of the 28 identified domains, had the highest number of competencies, and targeted a wide variety of health care workers. Conclusions: Digital health training initiatives should focus on competencies relevant to a particular health care worker group, role, level of seniority, and setting. The findings from this review can inform and guide digital health training initiatives. The most prevalent competency domains identified represent essential interprofessional competencies to be incorporated into health care workers’ training. Digital health frameworks should be regularly updated with novel digital health technologies, be applicable to low- and middle-income countries, and include overlooked health care worker groups such as allied health professionals.

Background: Effective communication skills are essential in diagnosis and treatment processes and in building the doctor-patient relationship. Objective: Our aim was to evaluate the effectiveness of digital education in medical students for communication skills development. Broadly, we assessed whether digital education could improve the quality of future doctors’ communication skills. Methods: We performed a systematic review and searched seven electronic databases and two trial registries for randomized controlled trials (RCTs) and cluster RCTs (cRCTs) published between January 1990 and September 2018. Two reviewers independently screened the citations, extracted data from the included studies, and assessed the risk of bias. We also assessed the quality of evidence using the Grading of Recommendations, Assessment, Development, and Evaluations assessment (GRADE). Results: We included 12 studies with 2101 medical students, of which 10 were RCTs and two were cRCTs. The digital education included online modules, virtual patient simulations, and video-assisted oral feedback. The control groups included didactic lectures, oral feedback, standard curriculum, role play, and no intervention as well as less interactive forms of digital education. The overall risk of bias was high, and the quality of evidence ranged from moderate to very low. For skills outcome, meta-analysis of three studies comparing digital education to traditional learning showed no statistically significant difference in postintervention skills scores between the groups (standardized mean difference [SMD]=–0.19; 95% CI –0.9 to 0.52; I2=86%, N=3 studies [304 students]; small effect size; low-quality evidence). Similarly, a meta-analysis of four studies comparing the effectiveness of blended digital education (ie, online or offline digital education plus traditional learning) and traditional learning showed no statistically significant difference in postintervention skills between the groups (SMD=0.15; 95% CI –0.26 to 0.56; I2=86%; N=4 studies [762 students]; small effect size; low-quality evidence). The additional meta-analysis of four studies comparing more interactive and less interactive forms of digital education also showed little or no difference in postintervention skills scores between the two groups (SMD=0.12; 95% CI: –0.09 to 0.33; I2=40%; N=4 studies [893 students]; small effect size; moderate-quality evidence). For knowledge outcome, two studies comparing the effectiveness of blended online digital education and traditional learning reported no difference in postintervention knowledge scores between the groups (SMD=0.18; 95% CI: –0.2 to 0.55; I2=61%; N=2 studies [292 students]; small effect size; low-quality evidence). The findings on attitudes, satisfaction, and patient-related outcomes were limited or mixed. None of the included studies reported adverse outcomes or economic evaluation of the interventions. Conclusions: We found low-quality evidence showing that digital education is as effective as traditional learning in medical students’ communication skills training. Blended digital education seems to be at least as effective as and potentially more effective than traditional learning for communication skills and knowledge. We also found no difference in postintervention skills between more and less interactive forms of digital education. There is a need for further research to evaluate the effectiveness of other forms of digital education such as virtual reality, serious gaming, and mobile learning on medical students’ attitude, satisfaction, and patient-related outcomes as well as the adverse effects and cost-effectiveness of digital education.

Journal of Medical Internet Research - International Scientific Journal for Medical Research, Information and Communication on the Internet #Preprint #PeerReviewMe: Warning: This is a unreviewed preprint. Readers are warned that the document has not been peer-reviewed by expert/patient reviewers or an academic editor, may contain misleading claims, and is likely to undergo changes before final publication, if accepted, or may have been rejected/withdrawn. Readers with interest and expertise are encouraged to sign up as peer-reviewer, if the paper is within an open peer-review period. Please cite this preprint only for review purposes or for grant applications and CVs (if you are the author). Background: The use of digital education in problem-based learning, i.e. digital problem-based (DPBL) learning is increasingly employed in healthcare professions’ education. DPBL includes purely digitally delivered as well as blended problem-based learning, in which digital and face-to-face learning are combined. Objective: The aim of this review is to evaluate the effectiveness of digital problem-based learning (DPBL) in improving healthcare professionals (HCPs)' knowledge, skills, attitudes and satisfaction. Methods: We used gold standard Cochrane methods to conduct a systematic review of randomized controlled trials (RCTs). Results: Nine RCTs involving 890 pre-registration HCPs were included. Digital technology was mostly employed for presentation of problems. In three studies, PBL was fully delivered online. Digital technology modalities spanned online learning, offline learning, virtual reality, and virtual patients. The control groups consisted of traditional PBL and traditional learning. The pooled analysis of seven studies comparing digital PBL to traditional PBL reported little or no difference in post-intervention knowledge outcomes (SMD = 0.19, 95% CI: 0.00-0.38). The pooled analysis of three studies comparing DPBL to traditional learning favored of DPBL (SMD = 0.67, 95% CI: 0.14-1.19). For skills, the pooled analysis of two studies comparing DPBL to traditional learning favored DPBL (SMD = 0.30, 95% CI: 0.07-0.54). Findings on attitudes and satisfaction outcomes were mixed. Conclusions: Our findings suggest that DPBL is as effective as traditional PBL and more effective than traditional learning in improving knowledge. DPBL may be more effective than traditional learning in improving skills. Further studies should evaluate the use of digital technology for delivery of other PBL components as well as PBL overall.

Background: Medical schools worldwide are accelerating the introduction of digital health courses into their curricula. The COVID-19 pandemic has contributed to this swift and widespread transition to digital health and education. However, the need for digital health competencies goes beyond the COVID-19 pandemic because they are becoming essential for the delivery of effective, efficient, and safe care. Objective: This review aims to collate and analyze studies evaluating digital health education for medical students to inform the development of future courses and identify areas where curricula may need to be strengthened. Methods: We carried out a scoping review by following the guidance of the Joanna Briggs Institute, and the results were reported in accordance with the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews) guidelines. We searched 6 major bibliographic databases and gray literature sources for articles published between January 2000 and November 2019. Two authors independently screened the retrieved citations and extracted the data from the included studies. Discrepancies were resolved by consensus discussions between the authors. The findings were analyzed using thematic analysis and presented narratively. Results: A total of 34 studies focusing on different digital courses were included in this review. Most of the studies (22/34, 65%) were published between 2010 and 2019 and originated in the United States (20/34, 59%). The reported digital health courses were mostly elective (20/34, 59%), were integrated into the existing curriculum (24/34, 71%), and focused mainly on medical informatics (17/34, 50%). Most of the courses targeted medical students from the first to third year (17/34, 50%), and the duration of the courses ranged from 1 hour to 3 academic years. Most of the studies (22/34, 65%) reported the use of blended education. A few of the studies (6/34, 18%) delivered courses entirely digitally by using online modules, offline learning, massive open online courses, and virtual patient simulations. The reported courses used various assessment approaches such as paper-based assessments, in-person observations, and online assessments. Most of the studies (30/34, 88%) evaluated courses mostly by using an uncontrolled before-and-after design and generally reported improvements in students’ learning outcomes. Conclusions: Digital health courses reported in literature are mostly elective, focus on a single area of digital health, and lack robust evaluation. They have diverse delivery, development, and assessment approaches. There is an urgent need for high-quality studies that evaluate digital health education.