Ankylosing spondylitis (AS), which usually occurs in the second and third decades of life, is associated with chronic pain, limitation of mobility, and severe decreases in quality of life. This study aimed to make a comparative evaluation in terms of the readability, information accuracy and quality of the answers given by artificial intelligence (AI)-based chatbots such as ChatGPT, Perplexity and Gemini, which have become popular with the widespread access to medical information, to user questions about AS, a chronic inflammatory joint disease. In this study, the 25 most frequently queried keywords related to AS determined through Google Trends were directed to each 3 AI-based chatbots. The readability of the resulting responses was evaluated using readability indices such as Simple Gunning Fog (GFOG), Flesch Reading Ease Score (FRES) and Measure of Gobbledygook (SMOG). The quality of the responses was measured by Ensuring Quality Information for Patients (EQIP) and Global Quality Score (GQS) scores, and the reliability was measured using the modified DISCERN and Journal of American Medical Association (JAMA) scales. According to Google Trends data, the most frequently searched keywords related to AS are “Ankylosing spondylitis pain”, “Ankylosing spondylitis symptoms” and “Ankylosing spondylitis disease”, respectively. It was found that the readability levels of the answers produced by AI-based chatbots were above the 6th grade level and showed a statistically significant difference (p < 0.001). In EQIP, JAMA, mDISCERN and GQS evaluations, Perplexity stood out in terms of information quality and reliability, receiving higher scores compared to other chat robots (p < 0.05). It has been found that the answers given by AI chatbots to AS-related questions exceed the recommended readability level and the reliability and quality assessment raises concerns due to some low scores. It is possible for future AI chatbots to have sufficient quality, reliability and appropriate readability levels with an audit mechanism in place.

Background: Patients who are informed about the causes, pathophysiology, treatment and prevention of a disease are better able to participate in treatment procedures in the event of illness. Artificial intelligence (AI), which has gained popularity in recent years, is defined as the study of algorithms that provide machines with the ability to reason and perform cognitive functions, including object and word recognition, problem solving and decision making. This study aimed to examine the readability, reliability and quality of responses to frequently asked keywords about low back pain (LBP) given by three different AI-based chatbots (ChatGPT, Perplexity and Gemini), which are popular applications in online information presentation today. Methods: All three AI chatbots were asked the 25 most frequently used keywords related to LBP determined with the help of Google Trend. In order to prevent possible bias that could be created by the sequential processing of keywords in the answers given by the chatbots, the study was designed by providing input from different users (EO, VH) for each keyword. The readability of the responses given was determined with the Simple Measure of Gobbledygook (SMOG), Flesch Reading Ease Score (FRES) and Gunning Fog (GFG) readability scores. Quality was assessed using the Global Quality Score (GQS) and the Ensuring Quality Information for Patients (EQIP) score. Reliability was assessed by determining with DISCERN and Journal of American Medical Association (JAMA) scales. Results: The first three keywords detected as a result of Google Trend search were “Lower Back Pain”, “ICD 10 Low Back Pain”, and “Low Back Pain Symptoms”. It was determined that the readability of the responses given by all AI chatbots was higher than the recommended 6th grade readability level (p < 0.001). In the EQIP, JAMA, modified DISCERN and GQS score evaluation, Perplexity was found to have significantly higher scores than other chatbots (p < 0.001). Conclusion: It has been determined that the answers given by AI chatbots to keywords about LBP are difficult to read and have low reliability and quality assessment. It is clear that when new chatbots are introduced, they can provide better guidance to patients with increased clarity and text quality. This study can provide inspiration for future studies on improving the algorithms and responses of AI chatbots.

It is clear that artificial intelligence-based chatbots will be popular applications in the field of healthcare in the near future. It is known that more than 30% of the world’s population suffers from chronic pain and individuals try to access the health information they need through online platforms before applying to the hospital. This study aimed to examine the readability, reliability and quality of the responses given by 3 different artificial intelligence chatbots (ChatGPT, Gemini and Perplexity) to frequently asked questions about pain. In this study, the 25 most frequently used keywords related to pain were determined using Google Trend and asked to every 3 artificial intelligence chatbots. The readability of the response texts was determined by Flesch Reading Ease Score (FRES), Simple Measure of Gobbledygook, Gunning Fog and Flesch-Kincaid Grade Level readability scoring. Reliability assessment was determined by the Journal of American Medical Association (JAMA), DISCERN scales. Global Quality Score and Ensuring Quality Information for Patients (EQIP) score were used in quality assessment. As a result of Google Trend search, the first 3 keywords were determined as “back pain,” “stomach pain,” and “chest pain.” The readability of the answers given by all 3 artificial intelligence applications was determined to be higher than the recommended 6th grade readability level ( P < .001). In the readability evaluation, the order from easy to difficult was determined as Google Gemini, ChatGPT and Perplexity. Higher GQS scores ( P = .008) were detected in Gemini compared to other chatbots. Perplexity had higher JAMA, DISCERN and EQIP scores compared to other chatbots, respectively ( P < .001, P < .001, P < .05). It has been determined that the answers given by ChatGPT, Gemini, and Perplexity to pain-related questions are difficult to read and their reliability and quality are low. It can be stated that these artificial intelligence chatbots cannot replace a comprehensive medical consultation. In artificial intelligence applications, it may be recommended to facilitate the readability of text content, create texts containing reliable references, and control them by a supervisory expert team.

Background: There are concerns over the reliability and comprehensibility of health-related information on the internet. We analyzed the readability, reliability and quality of online patient education materials obtained from websites associated with chronic low back pain (cLBP). Methods: On 26 April 2022, the term ‘cLBP’ was used to perform a search on Google, and 95 eligible websites were identified. The Flesch Reading Ease Score (FRES) and Gunning Fog (GFOG) index were used to evaluate the readability. The Journal of the American Medical Association (JAMA) score was used to assess the reliability and the Health on the Net Foundation code of conduct (HONcode) was used to assess quality. Results: The mean (SD) FRES was 55.74 (13.57) (very difficult) and the mean (SD) GFOG was 12.76 (2.8) (very difficult) of the websites reviwed. According to the JAMA scores, 28.4% of the websites had a high reliability rating and 33.7% adhered to the HONcode. Websites of different typologies were found to significantly differ in their reliability and the quality scores (p<0.05). Conclusion: The reading ability required for cLBP-related information on the internet was found to be considerably higher than that recommended by the National Health Institute and had low reliability and poor quality. We believe that online information should have readability appropriate for most readers and must have reliable content that is appropriate to educate the public, particularly for websites that provide patient education material.

Background: The use of the Internet to access healthcare-related information is increasing day by day. However, there are concerns regarding the reliability and comprehensibility of this information. This study aimed to investigate the readability, reliability, and quality of Internet-based patient educational materials (PEM) related to “post-COVID-19 pain.” Methods: One-hundred websites that fit the purposes of the study were identified by searching for the terms “post-COVID-19 pain” and “pain after COVID-19” using the Google search engine on February 24, 2022. The website readability was assessed using the Flesch Reading Ease Score (FRES), Flesch–Kincaid Grade Level (FKGL), Simple Measure of Gobbledygook (SMOG), and Gunning FOG (GFOG). The reliability, quality, and popularity of the websites were assessed using the JAMA score, DISCERN score/Health on the Net Foundation code of conduct, and Alexa, respectively. Results: Upon investigation of the textual contents, the mean FRES was 51.40 ± 10.65 (difficult), the mean FKGL and SMOG were 10.93 ± 2.17 and 9.83 ± 1.66 years, respectively, and the mean GFOG was 13.14 ± 2.16 (very difficult). Furthermore, 24.5% of the websites were highly reliable according to JAMA scores, 8% were of high quality according to GQS values, and 10% were HONcode-compliant. There was a statistically significant difference between the website types and reliability (p = 0.003) and quality scores (p = 0.002). Conclusion: The readability level of PEM on post-COVID-19 pain was considerably higher than grade 6 educational level, as recommended by the National Institutes of Health, and had low reliability and poor quality. We suggest that Internet-based PEM should have a certain degree of readability that is in accordance with the educational level of the general public and feature reliable content.

Background: The use of the internet as a source of information has increased during the pandemic, and YouTube has become an increasingly important source of information on Coronavirus disease 2019 (COVID-19). In the long COVID picture, which occurs when symptoms related to COVID-19 last longer than 1 month, pain involving the musculoskeletal system affects the quality of life quite negatively. The aim of this study was to investigate the informational value and quality of YouTube videos related to post-COVID pain. Methods: In this study, 180 videos were listed using the search terms “pain after COVID,” “post-COVID pain,” and “long COVID and pain”(15 April 2022). Videos were classified according to video parameters and content analysis. Quality, reliability and accuracy of the videos were determined with the Global Quality Score (GQS), the Journal of American Medical Association (JAMA) Benchmark Criteria and the Modified DISCERN Questionnaire, respectively. Results: One hundred videos that met the inclusion criteria were included in the assessment. Of these videos, 74 were found to be of low quality, 14 of moderate quality, and 12 of high quality; 21% contained insufficient data, 73% contained partially sufficient data, and 6% contained completely sufficient data. Videos uploaded by academic sources (66.7%) and physicians (12.5%) made up the majority of the high-quality group. A statistically significant correlation was found between the source of upload and number of views (p = 0.014), likes (p = 0.030), comments (p = 0.007), and video duration (p = 0.004). Video duration was found to have a poor positive correlation with GQS (r = 0.500), JAMA (r = 0.528), and modified DISCERN (r = 0.470) scores (p < 0.001). Conclusion: The findings of this study revealed that the majority of YouTube videos on post-COVID pain had low quality and partially sufficient data. High-quality videos were found to have longer durations and were uploaded by academic sources and physicians. The fact that only videos with English content at a certain time can be counted among the limitations. For patients suffering from post-COVID pain whose access to healthcare services was interrupted during the COVID pandemic, YouTube can be considered as an alternative source as well as a means of telerehabilitation. It can be argued that higher quality videos created by healthcare professionals could aid in patient education in the future.