Background/Objectives: The use of artificial intelligence (AI) in education is dynamically growing, and models such as ChatGPT show potential in enhancing medical education. In Poland, to obtain a medical diploma, candidates must pass the Medical Final Examination, which consists of 200 questions with one correct answer per question, is administered in Polish, and assesses students’ comprehensive medical knowledge and readiness for clinical practice. The aim of this study was to determine how ChatGPT-3.5 handles questions included in this exam. Methods: This study considered 980 questions from five examination sessions of the Medical Final Examination conducted by the Medical Examination Center in the years 2022–2024. The analysis included the field of medicine, the difficulty index of the questions, and their type, namely theoretical versus case-study questions. Results: The average correct answer rate achieved by ChatGPT for the five examination sessions hovered around 60% and was lower (p < 0.001) than the average score achieved by the examinees. The lowest percentage of correct answers was in hematology (42.1%), while the highest was in endocrinology (78.6%). The difficulty index of the questions showed a statistically significant correlation with the correctness of the answers (p = 0.04). Questions for which ChatGPT-3.5 provided incorrect answers had a lower (p < 0.001) percentage of correct responses. The type of questions analyzed did not significantly affect the correctness of the answers (p = 0.46). Conclusions: This study indicates that ChatGPT-3.5 can be an effective tool for assisting in passing the final medical exam, but the results should be interpreted cautiously. It is recommended to further verify the correctness of the answers using various AI tools.
Background: Use of Large Language Models (LLMs) as a health information source is on the rise. As the amount of people willing to self-diagnose with the tools’ help is increasing, verifying the correctness of generated responses becomes extremely important. Methods: Four LLMs (ChatGPT, Copilot, DeepSeek, Gemini) were chosen for the analysis. All models were asked 90 multiple-choice questions on the various topics surrounding vaccination, sourced from a government information website. Additionally, the LLMs were asked two open-ended questions regarding the vaccine information sources, as well as advantages and disadvantages of universal vaccination. Answers were analysed quantitatively and qualitatively. Data was collected in March 2026. Results: Biggest proportion of correct answers was achieved by Gemini (0.98), smallest one – by ChatGPT (0.86). Models’ results were found to be non-homogeneous ( p = 0.0049). While answering open-ended questions, all LLMs were able to cite various, appropriate vaccine information sources. For advantages of vaccination all models pointed to herd immunity. Answers regarding disadvantages to vaccination all included the possibility of adverse reactions. The answers were judged to be factually correct, yet lacking in detail. Conclusion: Tested LLMs were able to identify correct option when presented with multiple choices and to prepare correct, yet vague information when asked an open-ended question. This points to a limited potential of LLMs serving as a vaccine information source, as the generated content, while factually correct, was not enough to be usable on its own.
Introduction: In 2023 Poland launched a free, nationwide HPV vaccination program for children aged 12 and 13 years. This study aims to identify psychological drivers behind HPV vaccination-related behavior among Polish-speaking Facebook users and links between different categories of drivers. Methods: Entries published between 15 October 2020 and 15 October 2023 were categorized based on five antecedents of vaccination, using a scale created in accordance with the 5C short scale (consisting of categories: confidence, complacency, constraints, calculation, and collective responsibility) tailored to the Polish language. Results: The discourse on HPV vaccination was present throughout the study period, with a significant rise in popularity in 2023, accounting for 51% of all entries. Timing of surges in entries was corresponding to HPV vaccine-related decisions of various authorities and events of special interest happening in the country and abroad. 42% of all entries were assigned to 5C model categories. Factors influencing HPV vaccine hesitancy coexisted, with the Calculation and Confidence categories being the most prevalent (60% and 47% of coded entries, respectively). 30% of coded entries included words only from the Calculation category. The most common combination was Confidence and Calculation, with 13% of entries belonging to both categories. Conclusion: Psychological factors affecting HPV vaccination include most commonly individuals' extensive information-seeking efforts, susceptibility to conspiracy theories, and their trust levels in vaccines and government authorities. HPV vaccination educational campaigns should target Facebook users with consideration of their diverse needs and psychological profiles and prioritize the dissemination of reliable information in an easily comprehensible manner to ensure optimal reception of the content.