Chatbots using the Large Language Model (LLM) generate human responses to questions from all categories. Due to staff shortages in healthcare systems, patients waiting for an appointment increasingly use chatbots to get information about their condition. Given the number of chatbots currently available, assessing the responses they generate is essential. Five chatbots with free access were selected (Gemini, Microsoft Copilot, PiAI, ChatGPT, ChatSpot) and blinded using letters (A, B, C, D, E). Each chatbot was asked questions about cardiology, oncology, and psoriasis. Responses were compared to guidelines from the European Society of Cardiology, American Academy of Dermatology and American Society of Clinical Oncology. All answers were assessed using readability scales (Flesch Reading Scale, Gunning Fog Scale Level, Flesch-Kincaid Grade Level and Dale-Chall Score). Using a 3-point Likert scale, two independent medical professionals assessed the compliance of the responses with the guidelines. A total of 45 questions were asked of all chatbots. Chatbot C gave the shortest answers, 7.0 (6.0 - 8.0), and Chatbot A the longest 17.5 (13.0 - 24.5). The Flesch Reading Ease Scale ranged from 16.3 (12.2 - 21.9) (Chatbot D) to 39.8 (29.0 - 50.4) (Chatbot A). Flesch-Kincaid Grade Level ranged from 12.5 (10.6 - 14.6) (Chatbot A) to 15.9 (15.1 - 17.1) (Chatbot D). Gunning Fog Scale Level ranged from 15.77 (Chatbot A) to 19.73 (Chatbot D). Dale-Chall Score ranged from 10.3 (9.3 - 11.3) (Chatbot A) to 11.9 (11.5 - 12.4) (Chatbot D). This study indicates that chatbots vary in length, quality, and readability. They answer each question in their own way, based on the data they have pulled from the web. Reliability of the responses generated by chatbots is high. This suggests that people who want information from a chatbot need to be careful and verify the answers they receive, particularly when they ask about medical and health aspects.
The rapid development of large language models (LLMs), including ChatGPT, Gemini, and Copilot, has led to their increasing use in health communication and patient education. However, their growing popularity raises important concerns about whether the language they generate aligns with recommended readability standards and patient health literacy levels. This review synthesizes evidence on the readability of medical information generated by chatbots using established linguistic readability indices. A comprehensive search of PubMed, Scopus, Web of Science, and Cochrane Library identified 4209 records, from which 140 studies met the eligibility criteria. Across the included publications, 21 chatbots and 14 readability scales were examined, with the Flesch–Kincaid Grade Level and Flesch Reading Ease being the most frequently applied metrics. The results demonstrated substantial variability in readability across chatbot models; however, most texts corresponded to a secondary or early tertiary reading level, exceeding the commonly recommended 8th-grade level for patient-facing materials. ChatGPT-4, Gemini, and Copilot exhibited more consistent readability patterns, whereas ChatGPT-3.5 and Perplexity produced more linguistically complex content. Notably, DeepSeek-V3 and DeepSeek-R1 generated the most accessible responses. The findings suggest that, despite technological advances, AI-generated medical content remains insufficiently readable for general audiences, posing a potential barrier to equitable health communication. These results underscore the need for readability-aware AI design, standardized evaluation frameworks, and future research integrating quantitative readability metrics with patient-level comprehension outcomes.
11 December 2023 in Archives of Gerontology and Geriatrics
Brachial aortic Pulse Wave Velocity (baPWV) and bone mineral density (BMD) are important indicators of cardiovascular health and bone strength, respectively. However, the gender-specific association between baPWV and BMD remains unclear. The aim of our study is to evaluate the relationship between baPWV and BMD in men and women populations METHODS: A comprehensive search was conducted in electronic databases for relevant studies published between the 1th and 30rd of April 2023. Studies reporting the correlation between baPWV and BMD in both males and females were considered. A random-effects model was used to calculate pooled correlation coefficients (r). Relevant data for both genders were found in six articles. In all publications included in the meta-analysis, the total number of studied individuals was 3800, with 2054 women and 1746 men. Pooled correlation coefficient was -0,24 (95 % CI: -0.34; -0.15) in women population, and -0.12 (95 %CI: -0.16, -0.06) in men. Based on the published data, we found that baPWV is negatively correlated with bone density in women. However, in men we do not find such a relationship. These findings suggest the importance of considering gender-specific factors when assessing the cardiovascular and bone health relationship.
16 December 2024 in Advances in Clinical and Experimental Medicine
The coronavirus disease 2019 (COVID-19) pandemic has significantly accelerated the development and use of new healthcare technologies. While younger individuals may have been able to quickly embrace virtual advancements, older adults may still have different needs in terms of health communication. To identify areas of interest and preferred sources of information related to the COVID-19 pandemic among older adults and to verify their eHealth competencies. The study was conducted between February 2022 and July 2022. It included listeners from the University of the Third Age (U3A) and younger students. Both groups received information about the HealthBuddy+ chatbot, a questionnaire that addressed respondents' interests about COVID-19, and the PL-eHEALS (eHealth Literacy Scale) questionnaire to measure their eHealth competencies. There were 573 participants in the study (U3A listeners - 303 participants, median age: 73 years (interquartile range (IQR): 69-77); young adult students - 270, median age: 24 years (IQR: 23-24). The primary source of information about COVID-19 for older adults was television (84.5%), and for younger adults, internet (84.4%). Among the older adults, only 17% ever interacted with a chatbot (younger adults - 78% respectively), and 19% considered it a trustworthy source of information on COVID-19 compared to 79% of younger respondents. Older adults and younger adults in our study were most interested in COVID-19 treatment methods (45.5% and 69.3%, respectively), symptoms of the disease (36.6% and 35.2%, respectively) and chronic diseases coexisting with COVID-19 (35.0% and 51.5%, respectively). However, their eHealth competencies were generally low (median (Me): 34; IQR: 30-39) compared to younger adults (Me: 42; IQR: 40-47). Health education for older adults should be appropriately tailored to their current needs and differentiated. The level of eHealth competencies of older adults suggests that much work remains to narrow the gap between the eHealth competencies of the younger and older generations.
Aim: This study aimed to evaluate the readability and citation practices of artificial intelligence (AI)-generated responses to questions about human metapneumovirus, a respiratory virus of growing public health concern. Subject and methods: Five widely used AI chatbots—ChatGPT-4, Copilot, Gemini, Claude.ai, and Grok—were prompted with 14 standardized questions based on official guidelines from the World Health Organization, the Centers for Disease Control and Prevention, and the Australian National Health and Medical Research Council. Responses were anonymized and assessed using six established readability metrics: Flesch–Kincaid Reading Ease and Flesch–Kincaid Grade Level, Gunning Fog Index, SMOG (Simple Measure of Gobbledygook) Index, Coleman–Liau Index, and Automated Readability Index. Scores were compared to standards recommended by the American Medical Association and the National Institutes of Health. Citation frequency and credibility were also analyzed. Results: Among 70 chatbot responses, only one met the recommended readability level. Median readability scores ranged from grade 10.4 to 16.0, indicating high complexity. One chatbot generated the most readable content, while another scored lowest. Only two chatbots included source citations. One cited 68 reliable sources, primarily from health organizations and academic institutions, while the other referenced 31 sources of varying quality. Conclusion: AI-generated health content often exceeds recommended readability thresholds and lacks consistent citation practices. These issues may hinder understanding and trust. Improving default readability settings and integrating real-time citation features could enhance the accessibility and credibility of chatbot-based health communication.
Objectives: Bone mineral density (BMD) assessment, the gold standard for diagnosing osteoporosis, does not account for bone quality and fracture susceptibility. Trabecular bone score (TBS) adds value to traditional densitometry. No studies have been conducted in the Polish population to date to confirm the association between TBS and fracture occurrence. This study aimed to evaluate the TBS derived from lumbar spine (L1–L4) dual-energy X-ray absorptiometry (DXA) scans in Polish women aged 40–76 years, both with and without osteoporotic fractures. The relationship between TBS, fracture risk (assessed by FRAX and TBS-adjusted FRAX), and BMD at the lumbar spine, femoral neck, and total hip was investigated. Methods: A total of 933 Caucasian women (760 without fracture and 173 with fracture) who underwent DXA examinations (Hologic Discovery A) between 2022 and 2024 were included. Lumbar TBS, BMD, and clinical fracture risk factors were analyzed, excluding subjects with scan artefacts or extreme BMI. Group differences were assessed using t-tests and chi-square tests. Pearson correlation was used to evaluate associations between TBS, age, and BMI. Logistic regression models assessed TBS and BMD as fracture discrimination, and model performance was compared using the Akaike Information Criterion (AIC) and the area under the receiver operating characteristic (ROC) curve (AUC). Results: TBS values were significantly lower in the fracture group (p < 0.001). TBS demonstrated negative correlations with age (r ≈ −0.36) and BMI (r ≈ −0.14). Low TBS values (≤1.23) were associated with the highest fracture prevalence (28.8%) and a threefold increased risk compared to high TBS (odds ratio = 3.0). Each one standard-deviation decrease in BMD or TBS T-score increased fracture risk by 56–67% (both p < 0.001). Models combining TBS and BMD improved discrimination, as indicated by higher AUC and lower AIC, with TBS remaining an independent predictor. In subgroups with osteopenia or osteoporosis, TBS retained statistical significance. Conclusions: TBS combined with BMD effectively discriminates fracture risk in Polish women and offers superior diagnostic accuracy compared to BMD alone. Integrating TBS with BMD enhances fracture accuracy. Routine assessment of TBS may improve clinical management of osteoporosis. Prospective studies are needed to confirm its long-term predictive value.