Rotator cuff tears are common upper-extremity injuries that significantly impair shoulder function, leading to pain, reduced range of motion, and a decrease in quality of life. With the increasing reliance on artificial intelligence large language models (AI LLMs) for health information, it is crucial to evaluate the quality and readability of the information provided by these models. A pool of 50 questions was generated related to rotator cuff tear by querying popular AI LLMs (ChatGPT 3.5, ChatGPT 4, Gemini, and Microsoft CoPilot) and using Google search. After that, responses from the AI LLMs were saved and evaluated. For information quality the DISCERN tool and a Likert Scale was used, for readability the Patient Education Materials Assessment Tool for Printable Materials (PEMAT) Understandability Score and the Flesch-Kincaid Reading Ease Score was used. Two orthopedic surgeons assessed the responses, and discrepancies were resolved by a senior author. Out of 198 answers, the median DISCERN score was 40, with 56.6% considered sufficient. The Likert Scale showed 96% sufficiency. The median PEMAT Understandability score was 83.33, with 77.3% sufficiency, while the Flesch-Kincaid Reading Ease score had a median of 42.05 with 88.9% sufficiency. Overall, 39.8% of the answers were sufficient in both information quality and readability. Differences were found among AI models in DISCERN, Likert, PEMAT Understandability, and Flesch-Kincaid scores. AI LLMs generally cannot offer sufficient information quality and readability. While they are not ready for use in medical field, they show a promising future. There is a necessity for continuous re-evaluation of these models due to their rapid evolution. Developing new, comprehensive tools for evaluating medical information quality and readability is crucial for ensuring these models can effectively support patient education. Future research should focus on enhancing readability and consistent information quality to better serve patients.
18 December 2024 in Joint Diseases and Related Surgery
Joint Diseases and Related Surgery (JDRS), the official journal of the Turkish Joint Diseases Foundation, is an international, peer-reviewed, immediate open access journal. All published articles in JDRS has been free for everyone, without embargo, since 1990.
17 March 2025 in Acta Orthopaedica et Traumatologica Turcica
Objective: The aim of this study was to evaluate and compare the performance of the artificial intelligence (AI) models ChatGPT-3.5, ChatGPT-4, and Gemini on the Turkish Specialization Training and Development Examination (UEGS) to determine their utility in medical education and their potential to improve patient care. Methods: This retrospective study analyzed responses of ChatGPT-3.5, ChatGPT-4, and Gemini to 1000 true or false questions from UEGS administered over 5 years (2018-2023). Questions, encompassing 9 orthopedic subspecialties, were categorized by 2 independent residents, with discrepancies resolved by a senior author. Artificial intelligence models were restarted for each query to prevent data retention. Performance was evaluated by calculating net scores and comparing them to orthopedic resident scores obtained from the Turkish Orthopedics and Traumatology Education Council (TOTEK) database. Statistical analyses included chi-squared tests, Bonferroni-adjustedZ tests, Cochran’s Q test, and receiver operating characteristic (ROC) analysis to determine the optimal question length for AI accuracy. All AI responses were generated independently without retaining prior information. Results: Significant di!erences in AI tool accuracy were observed across di!erent years and subspecialties (P < .001). ChatGPT-4 consistently outperformed other models, achieving the highest overall accuracy (95% in specific subspecialties). Notably, ChatGPT-4 demonstrated superior performance in Basic and General Orthopedics and Foot and Ankle Surgery, while Gemini and ChatGPT-3.5 showed variability in accuracy across topics and years. Receiver operating characteristic analysis revealed a significant relationship between shorter letter counts and higher accuracy for ChatGPT-4 (P=.002). ChatGPT-4 showed significant negative correlations between letter count and accuracy across all years (r="0.099, P=.002), outperformed residents in basic and general orthopedics (P=.015) and trauma (P=.012), unlike other AI models. Conclusion: The findings underscore the advancing role of AI in the medical field, with ChatGPT-4 demonstrating significant potentialas a tool for medical education and clinical decision-making. Continuous evaluation and refinement of AI technologies are essential toenhance their educational and clinical impact. Cite this article as: Ayik G, Can Kolac U, Aksoy T, et al. Exploring the role of artificial intelligence in Turkish orthopedic progression exams. Acta OrthopTraumatol Turc., 2025;59(1):18-26.
Purpose: Midshaft clavicle fractures are common and often associated with scapular dyskinesis (ScD), particularly in cases of shortening. While fractures with less than 2 cm shortening are often treated conservatively, emerging evidence suggests that even minor shortening can increase the risk of ScD and impair functional outcomes. This study investigates the impact of surgical versus conservative treatment on ScD and functional recovery. Methods: A retrospective analysis of 60 patients with isolated midshaft clavicle fractures was conducted. Patients were categorized into surgical and conservative groups. Fracture shortening was assessed using radiographs, outcomes were assessed using the SICK Scapula Rating Scale, Simple Shoulder Test (SST), American Shoulder and Elbow Surgeons Scale and Visual Analog Scale (VAS). Logistic regression and ROC analysis was applied to identify ScD predictors, and critical shortening threshold. Results: ScD was observed in 43.3% of all patients, with 53.6% of the conservative group, and 34.4% of the surgical group; however, the difference was not statistically significant ( p = 0.216). Surgical treatment was associated with significantly better SST and VAS scores at the final follow-up ( p < 0.05). Logistic regression identified clavicular shortening ( p < 0.001) and lower BMI ( p = 0.033 - univariate) as significant predictors of ScD. ROC analysis revealed that a shortening threshold of 0.4 cm had a sensitivity of 73.08% and a specificity of 91.18% for predicting ScD (AUC = 0.874, p < 0.001). Conclusion: Surgical treatment may reduce residual shortening and lower the prevalence of ScD, indicating a possible benefit in limiting dyskinesis even in minimally displaced fractures. Level of evidence: Level III, retrospective comperative study.