Background and Objectives: Interpreting acid–base disorders is challenging, particularly in complex or mixed cases. Given the growing potential of large language models (LLMs) to assist in cognitively demanding tasks, this study evaluated their performance in interpreting arterial blood gas (ABG) results. Materials and Methods: In this single-center retrospective study, 200 ABG datasets were curated to include 40 cases in each of five diagnostic categories: metabolic acidosis, respiratory acidosis, metabolic alkalosis, respiratory alkalosis, and no acid–base disorder. Three medical students, each assigned to one LLM (ChatGPT GPT-4o, Copilot GPT-4, or Gemini 1.5-flash/2.5-flash), perform ABG interpretation using two evaluation methods: interpretation (LLM-I) and interpretation with supervision model (LLM-S). Two clinical pathologists independently performed the conventional evaluation to serve as the reference standard. Results: Agreement for identifying the primary acid–base (APD) disorder was strong across all approaches (Cohen’s κ ≥ 0.88). For identifying both primary and secondary disorders regardless of order (APSD), LLM-I showed moderate agreement (ChatGPT κ = 0.65, Copilot κ = 0.61, Gemini κ = 0.62), whereas LLM-S achieved strong agreement (ChatGPT κ = 0.91, Copilot κ = 0.81, Gemini κ = 0.81). Conclusions: LLM-assisted ABG interpretation demonstrates strong concordance with expert interpretation in detecting primary acid–base disorders. These tools may enhance the understanding of acid–base disorders while reducing calculation-related errors among medical students.
Background: The diagnostic performance of sepsis biomarkers may vary according to the clinical comparator population. We evaluated 11 routinely available inflammatory and hematologic biomarkers in two matched diagnostic settings. Methods: We conducted a retrospective observational study using linked MIMIC-IV, MIMIC-IV-ED, and MIMIC-IV-Note data. Adult emergency department episodes were classified as sepsis, suspected infection without sepsis, or controls without suspected infection. Comparison-specific profile matching retained the same 525 sepsis episodes in both analyses, together with 214 controls without suspected infection and 315 episodes with suspected infection without sepsis. Biomarker discrimination was assessed using area under the receiver operating characteristic curve (AUROC) with DeLong inference. Results: Against controls without suspected infection, C-reactive protein (CRP) had the highest AUROC (0.786; 95% CI, 0.748–0.823), followed by absolute lymphocyte count (0.654; 0.613–0.696) and the neutrophil-to-lymphocyte ratio (NLR; 0.636; 0.594–0.677). Against suspected infection without sepsis, absolute lymphocyte count had the highest AUROC (0.677; 0.641–0.714), followed by NLR (0.626; 0.588–0.664), whereas CRP discrimination decreased to 0.555 (0.514–0.595). The cross-setting difference for CRP was 0.231 (95% CI, 0.183–0.278; Holm-adjusted p <0.001). Sensitivity analyses produced similar qualitative findings. Conclusions: Biomarker discrimination for sepsis varied according to the clinical comparator population, with the magnitude of cross-setting change differing across biomarkers. CRP showed the largest change, whereas absolute lymphocyte count retained modest discrimination in both settings. These findings emphasize the importance of interpreting biomarker performance within clinically relevant diagnostic contrasts.
15 April 2025 in Revista Hospital Medicine and Clinical Management