With the increasing development of artificial intelligence, large language models (LLMs) have been utilized to solve problems in natural language processing tasks. More recently, LLMs have shown unique potential in numerous applications within medicine but have been particularly investigated for their ability in clinical reasoning. Although the diagnostic accuracy of LLMs in forming differential diagnoses has been reviewed in general internal medicine applications, much is unknown in autoinflammatory disorders. From the nature of autoinflammatory diseases, forming a differential diagnosis is challenging due to the overlapping symptoms between disorders and even more difficult without genetic screening. In this work, the diagnostic accuracy of the Generative Pre-Trained Transformer Model-4 (GPT-4), GPT-3.5, and Large Language Model Meta AI (LLaMa) were evaluated in clinical vignettes of Deficiency of Interleukin-1 Receptor Antagonist (DIRA) and Familial Mediterranean Fever (FMF). We then compared these models to a control group including one internal medicine physician. It was found that GPT-4 did not significantly differ in correctly identifying DIRA and FMF patients compared to the internist. However, the physician maintained a significantly higher accuracy than GPT-3.5 and LLaMa 2 for either disease. Overall, we explore and discuss the unique potential of LLMs in diagnostics for autoimmune diseases.

Given the rapid development of large language models (LLMs), such as ChatGPT, in its ability to understand and generate human-like texts, these technologies inspired efforts to explore their capabilities in natural language processing tasks, especially those in healthcare contexts. The performance of these tools have been evaluated thoroughly across medicine in diverse tasks, including standardized medical examinations, medical-decision making, and many others. In this journal, Anaya et al. published a study comparing the readability metrics of medical education resources formulated by ChatGPT with those of major U.S. institutions (AHA, ACC, HFSA) about heart failure. In this work, we provide a critical review of this article and further describe approaches to help mitigate challenges in reproducibility of studies evaluating LLMs in cardiology. Additionally, we provide suggestions to optimize sampling of responses provided by LLMs for future studies. Overall, while the study by Anaya et al. provides a meaningful contribution to literature of LLMs in cardiology, further comprehensive studies are necessary to address current limitations and further strengthen our understanding of these novel tools.

Background. Deficiency of interleukin-1 receptor antagonist (DIRA) is a rare life-threatening autosomal recessive autoinflammatory disease with symptoms including but not limited to osteomyelitis, periostitis, and systemic inflammation. DIRA is developed from the loss-of-function biallelic mutations of the IL1RN gene that encodes IL-1 receptor antagonist (IL-1RA), leading to the unchecked pro-inflammatory signaling and subsequent systemic inflammation. Thus, anakinra as the recombinant IL-1RA has become the primary drug to treat DIRA. Although anakinra has been effective for the complete remission of DIRA, it has also shown various side effects. To confirm the efficacy and safety issues associated with DIRA treatment, we conducted a literature review and secondary data analysis to enhance our understanding on this important topic. Methods. Through comprehensive literature search, we have identified 15 papers with 25 patients studied. The demographic, clinical, and genetic data were extracted, followed by statistical analysis to support the physiological mechanisms of anakinra treatment. Results. Through the literature review and data analysis, it was found that 88% of patients had complete clinical remission of DIRA upon continual treatment with anakinra; patients had a mean improvement of Hemoglobin (+3.18 g/dL), Erythrocyte Sedimentation Rate (−53.4 mm/h), and C-reactive Protein (−135.45 mg/L) levels, suggesting that the improvement of hematopoietic function and inflammation is a mechanism for anakinra treatment. Various genetic variants were also identified from the patient data that cause the loss of function of IL-1RA, providing real patient genomic data to support the anakinra treatment. Conclusions. Considering the inconsistency and certain variations from clinical research influenced by specific conditions, this review along with the data analysis confirms the efficacy and safety of anakinra treatment for DIRA.

Pathogenic missense mutations are commonly found in protein-coding regions of DNA, and often alter protein function. In the past decade, enormous experimental efforts have been undertaken through the development of functional assays, mutagenesis screenings, and computational prediction algorithms to characterize these variants. Indeed, most efforts have been focused towards identifying degrees of loss-of-function (LoF) effects on the three-dimensional structures of proteins, but gain-of-function (GoF) mutations remain poorly understood. Herein, we performed a case study of the PIEZO1 mechanosensitive ion channel protein whose GoF variants (n = 56) are implicated in hereditary xerocytosis (HX) and LoF variants (n = 6) in lymphatic dysplasia (LD), respectively. This study evaluated the abilities of AlphaMissense (AM) to decipher both mutation types, and benchmarked its performance against other algorithmic approaches, including Combined Annotation Dependent Depletion (CADD) v1.7, evolutionary model of variant effect (EVE), and Evolutionary Scale Modeling-1b (ESM-1B). We found that all approaches excelled in identifying LoF variants but were often ambiguous in their predictions for GoF PIEZO1 variants. ESM-1b was a notable exception that demonstrated balance sensitivity to both GoF and LoF variants as it likely identified certain sequential features not utilized in other approaches. Secondly, our findings suggest that GoF variants of HX do not significantly destabilize the PIEZO1 structure and are not generally identified by conventional signatures of damage from these algorithms. GoF variants are not synonymous with direct changes in changes in free energy upon mutations. Furthermore, we validated computational structure predictions of PIEZO1 against resolved cryo-EM structures, and provided biophysical data for variants located on unresolved residues of this ion channel. We formulated a weighted ensemble model that performed similarly to AM and outperformed all other traditional approaches evaluated in this study. Overall, this is the first study to directly evaluate the capabilities of pathogenicity prediction algorithms for deciphering GoF and LoF variants, and underscores the limitations present in current approaches.