Study aims and objectives. This study aims to evaluate the accuracy of medical knowledge in the most advanced LLMs (GPT-4o, GPT-4, Gemini 1.5 Pro, and Claude 3 Opus) as of 2024. It is the first to evaluate these LLMs using a non-English medical licensing exam. The insights from this study will guide educators, policymakers, and technical experts in the effective use of AI in medical education and clinical diagnosis. Authors inputted 790 questions from Japanese National Medical Examination into the chat windows of the LLMs to obtain responses. Two authors independently assessed the correctness. Authors analyzed the overall accuracy rates of the LLMs and compared their performance on image and non-image questions, questions of varying difficulty levels, general and clinical questions, and questions from different medical specialties. Additionally, authors examined the correlation between the number of publications and LLMs' performance in different medical specialties. GPT-4o achieved highest accuracy rate of 89.2% and outperformed the other LLMs in overall performance and each specific category. All four LLMs performed better on non-image questions than image questions, with a 10% accuracy gap. They also performed better on easy questions compared to normal and difficult ones. GPT-4o achieved a 95.0% accuracy rate on easy questions, marking it as an effective knowledge source for medical education. Four LLMs performed worst on "Gastroenterology and Hepatology" specialty. There was a positive correlation between the number of publications and LLM performance in different specialties. GPT-4o achieved an overall accuracy rate close to 90%, with 95.0% on easy questions, significantly outperforming the other LLMs. This indicates GPT-4o's potential as a knowledge source for easy questions. Image-based questions and question difficulty significantly impact LLM accuracy. "Gastroenterology and Hepatology" is the specialty with the lowest performance. The LLMs' performance across medical specialties correlates positively with the number of related publications.
28 April 2023 in Journal of Communication in Healthcare
During the prolonged COVID-19 pandemic, health communication researchers and practitioners should be aware of the unintended effects of message fatigue. Message fatigue is a motivational state caused by repeated and prolonged exposure to similar health-related messages that induces resistance to health behaviors. Messages encouraging COVID-19 vaccination tend to focus on scientific evidence and efficacy information. However, prolonged exposure to similarly framed repeated pro-COVID-19 vaccination messages may cause message fatigue, generate psychological reactance, and lead to ineffective persuasive outcomes. Scholars of message fatigue argue that health communication practitioners should select a less common frame to reduce fatigue responses and increase favorable attitudes toward message recommendations. Entering the second year since COVID-19 vaccination has begun, to reduce message fatigue, future pro-COVID-19 vaccination communication should increase the diversity of messages different than the frequently used types. This opinion piece proposes alternative dissemination of cognitive, affective, narrative, and non-narrative pro-COVID-19 vaccination messages.
We reviewed studies that examined the anticipated affects associated with human papillomavirus (HPV) vaccination to identify gaps in the literature and the currently available practice implications for encouraging HPV vaccination. We systematically searched MEDLINE, the Cumulative Index to Nursing and Allied Health Literature, PsycINFO, PsycArticles, Academic Search Complete, Scopus, and Web of Science to find English articles that quantitatively and qualitatively examined anticipated affects associated with HPV vaccination. A total of twenty-one studies were identified. Seventeen studies examined the anticipated inaction regret (i.e., not being vaccinated). Most of the included studies reported that anticipated inaction regret had a significantly positive association with HPV vaccination outcomes, such as vaccination behavior, intention, willingness, and acceptability. Furthermore, seven studies reported that anticipated inaction regret had a significantly positive and stronger association with vaccination outcomes than cognitive beliefs, such as vaccine effectiveness and safety, and perceived susceptibility and severity. The present review indicated that the stronger the participants’ anticipated inaction regret, the more likely they were to receive the HPV vaccine. Messages targeting the anticipated affect may be as effective as or more effective than messages targeting cognitive beliefs in encouraging HPV vaccination among people. However, most of the studies included in the present review adopted a cross-sectional design with vaccination intention and willingness as outcomes. Therefore, future studies should examine the influence of anticipated affects on the utilization of HPV vaccines using experimental designs to accumulate stronger evidence.
In recent years, researchers have discussed the introduction of an evolutionary perspective into public health and health behavior research. We aimed to examine the effects of messages that target the fundamental human motive of kin care on HPV vaccination recommendations among mothers with daughters, based on an evolutionary theoretical approach. This study consisted of a three-arm parallel-group single-blinded randomized controlled study. A web-based survey was conducted from 7 to 8 October 2021 in Japan. Mothers with daughters (n = 969) were randomly assigned either to a group that received an intervention message that targeted the fundamental motive of kin care, or that targeted the fundamental motive of disease avoidance, or a control message. Intention to have daughter(s) receive HPV vaccination was assessed both before and right after reading the messages. A one-way ANOVA with Tukey’s or Games–Howell test was conducted. An intervention message targeting the fundamental motive of kin care and disease avoidance significantly increased intention of vaccination versus a control message (p < 0.001, respectively). There was no significant difference between the two intervention groups. The evolutionary theoretical approach that focuses on fundamental human motives may have the potential to extend the communication strategy for HPV vaccination recommendations. Health professionals may be recommended to deliver messages that target the fundamental motive of kin care as well as messages about the susceptibility and severity of cervical cancer and vaccine efficacy (e.g., “Getting cervical cancer can prevent childbirth. To protect your daughter and your future grandchildren, get your daughter vaccinated against HPV”). However, the present study only evaluated HPV vaccination intentions in Japanese mothers with daughters. Future studies should evaluate vaccination behavior in a wider range of subjects to confirm that the evolutionary theoretical approach promotes HPV vaccination.