Background: The readability of most online patient educational materials (OPEMs) in orthopaedic surgery is above the American Medical Association/National Institutes of Health recommended reading level of sixth grade for both English- and Spanish-language content. The current project evaluates ChatGPT’s performance across English- and Spanish-language orthopaedic OPEMs when prompted to rewrite the material at a sixth-grade reading level. Methods: We performed a cross-sectional study evaluating the readability of 57 English- and 56 Spanish-language publicly available OPEMs found by querying online in both English and Spanish for 6 common orthopaedic procedures. Five distinct, validated readability tests were used to score the OPEMs before and after ChatGPT 4.0 was prompted to rewrite the OPEMs at a sixth-grade reading level. We compared the averages of each readability test, the cumulative average reading grade level, average total word count, average number of complex words (defined as ≥3 syllables), and average number of long sentences (defined as >22 words) between original content and ChatGPT-rewritten content for both languages using paired t tests. Results: The cumulative average reading grade level of original English- and Spanish-language OPEMs was 9.6 ± 2.6 and 9.5 ± 1.5, respectively. ChatGPT significantly lowered the reading grade level (improved comprehension) to 7.7 ± 1.9 (95% CI of difference, 1.68 to 2.15; p < 0.05) for English-language content and 8.3 ± 1.3 (95% CI, 1.17 to 1.45; p < 0.05) for Spanish-language content. English-language OPEMs saw a reduction of 2.0 ± 1.8 grade levels, whereas Spanish-language OPEMs saw a reduction of 1.5 ± 1.2 grade levels. Word count, use of complex words, and long sentences were also reduced significantly in both languages while still maintaining high accuracy and similarity compared with original content. Conclusions: Our study supports the potential of artificial intelligence as a low-cost, accessible tool to assist health professionals in improving the readability of orthopaedic OPEMs in both English and Spanish.

It is becoming increasingly important to evaluate the effectiveness of large language models (LLMs) and query-assisted platforms like Google and ChatGPT in providing clinically relevant and accurate information to patient-initiated inquiries. Limited studies have characterized the performance of these platforms on semi-elective orthopedic trauma surgeries. This study evaluates the function of both interactive online models on frequently queried patient searches on ankle and clavicle fracture operations. An inquiry set consisting of ten prevalent patient questions for each fracture type was extracted using Google and ChatGPT. Responses, alongside their source attributions, were curated for clinical relevance and accuracy. This assessment was conducted in a double-blind fashion. Grading was completed by two researchers with academic and clinical experience in orthopedic trauma (D.E.P, H.F.B), with oversight and validation provided by a board-certified orthopedic trauma surgeon (A.N.M). Descriptive and comparative statistics were used to understand the dataset. ChatGPT performed better than Google in terms of clinical relevance (p = 0.001) and accuracy (p = 0.004) in ankle fracture patient queries. Clavicle fracture queries showed a significant difference favoring ChatGPT for accuracy (p = 0.04), though relevance did not reach statistical significance (p = 0.06). When answer ground-truth was analyzed, GPT outperformed Google by providing more academic sources (p < 0.05). Large Language Models outperformed traditionally used online platforms in delivering clinically precise and contextually relevant information on semi-elective, common orthopedic trauma surgeries. The ability of LLMs to synthesize responses from credible medical sources significantly diminishes the variability and potential inaccuracies inherent in conventional web searches. These insights strongly suggest that LLMs could play a pivotal role in enhancing patient engagement and comprehension in trauma care, meriting further exploration of their integration within clinical frameworks. Therapeutic Level III.

Background: Current Procedural Terminology (CPT) codes are critical for billing and documentation in orthopaedic surgery, yet coding can be complex and costly. Large language models (LLMs) have shown promise in automating CPT coding. Methods: Ten orthopaedic procedures were selected from each of the subspecialties of sports, foot and ankle, and hand. Procedure names were generated by the authors to reflect practice at our institution, and CPT codes for each procedure were secondarily validated for ground truth analysis using the American Association of Professional Coders “Codify” tool. An institutional Health Insurance Portability and Accountability Act (HIPAA)-compliant, ChatGPT-4 Omni (ChatGPT4o)-based LLM was prompted to generate CPT codes based on a procedure name. Three independent trials were performed per procedure. Responses were categorized as “correct”, “partially correct”, or “incorrect” and assigned a score (+1, +0.5, or +0, respectively). Model performance was assessed as a percentage of the maximum possible score. For incorrect responses, the model was prompted to try again. Results: The LLM generated the correct response in 86.7% of trials, with an overall performance of 88.9% of the maximum possible score. Performance varied by subspecialty ( P =0.018), with lower scores in hand compared to sports ( P =0.019). After re-prompting, LLM performance improved to 93.9% of the maximum possible score. The LLM had the most difficulty distinguishing between closely related codes for hand procedures. Conclusions: This study demonstrates promising potential for rapid, accurate, and cost-effective LLM-driven orthopedic coding, suggesting practical applications in reducing administrative workload, improving coding consistency, and potentially lowering costs in orthopaedic clinical practices.