Artificial intelligence (AI) and machine learning (ML) are rapidly transforming clinical decision support systems (CDSSs) in intensive care units (ICUs), where vast amounts of real-time data present both an opportunity and a challenge for timely clinical decision-making. Here, we trace the evolution of machine intelligence in critical care. This technology has been applied across key ICU domains such as early warning systems, sepsis management, mechanical ventilation, and diagnostic support. We highlight a transition from rule-based systems to more sophisticated machine learning approaches, including emerging frontier models. While these tools demonstrate strong potential to improve predictive performance and workflow efficiency, their implementation remains constrained by concerns around transparency, workflow integration, bias, and regulatory challenges. Ensuring the safe, effective, and ethical use of AI in intensive care will depend on validated, human-centered systems supported by transdisciplinary collaboration, technological literacy, prospective evaluation, and continuous monitoring.
Background/Objectives: Artificial intelligence (AI), particularly large language models (LLMs), has demonstrated versatility in various applications but faces challenges in specialized domains like neurology. This study evaluates a specialized LLM’s capability and trustworthiness in complex neurological diagnosis, comparing its performance to neurologists in simulated clinical settings. Methods: We deployed GPT-4 Turbo (OpenAI, San Francisco, CA, US) through Neura (Sciense, New York, NY, US), an AI infrastructure with a dual-database architecture integrating “long-term memory” and “short-term memory” components on a curated neurological corpus. Five representative clinical scenarios were presented to 13 neurologists and the AI system. Participants formulated differential diagnoses based on initial presentations, followed by definitive diagnoses after receiving conclusive clinical information. Two senior academic neurologists blindly evaluated all responses, while an independent investigator assessed the verifiability of AI-generated information. Results: AI achieved a significantly higher normalized score (86.17%) compared to neurologists (55.11%, p < 0.001). For differential diagnosis questions, AI scored 85% versus 46.15% for neurologists, and for final diagnosis, 88.24% versus 70.93%. AI obtained 15 maximum scores in its 20 evaluations and responded in under 30 s compared to neurologists’ average of 9 min. All AI-provided references were classified as relevant with no hallucinatory content detected. Conclusions: A specialized LLM demonstrated superior diagnostic performance compared to practicing neurologists across complex clinical challenges. This indicates that appropriately harnessed LLMs with curated knowledge bases can achieve domain-specific relevance in complex clinical disciplines, suggesting potential for AI as a time-efficient asset in clinical practice.
Objective: To evaluate predictive diagnostic performance of open- and closed-source large language models (LLMs) in emergency medicine, addressing the urgent need for innovative clinical decision support tools amid rising patient volumes and staffing shortages. Materials and Methods: We generated 2370 AI-driven diagnostic predictions (Top-5 diagnoses from each of 6 model pipelines per patient), using data from 79 real-world emergency department cases collected consecutively during a 24-hour peak influx period at a tertiary care center. Pipelines combined open- and closed-source embedding models (text-embedding-ada-002, MXBAI) with foundational models (GPT-4, Llama3, and Qwen2) grounded via retrieval-augmented generation using emergency medicine textbooks. Models’ predictions were assessed against reference diagnoses established by expert consensus. Results: All pipelines achieved comparable diagnostic match rates (62.03%-72.15%). Diagnostic performance was significantly influenced by case characteristics: match rates were notably higher for specific versus unspecific diagnoses (85.53% vs 31.41%, P < .001) and surgical versus medical cases (79.49% vs 56.25%, P < .001). Open-source models demonstrated markedly superior sourcing capabilities compared to GPT-4-based combinations (P < 1.4e-12), with MBXAI/Qwen2 pipeline achieving perfect citation verification. Discussion: Diagnostic accuracy primarily depended on case characteristics rather than the choice of model pipeline, highlighting fundamental AI alignment challenges in clinical reasoning. Low performance in unspecific diagnoses underscores inherent complexities in clinical definitions rather than technological shortcomings alone. Conclusion: Open-source LLM pipelines provide enhanced sourcing capabilities, crucial for transparent clinical decision-making and interpretability. Further research should expand knowledge bases to include hospital guidelines and regional epidemiology, while exploring on-premises solutions to better align with privacy regulations and clinical integration.
2 December 2023 in Internal and Emergency Medicine
Purpose: Foreign body ingestion (FBI) is a frequent concern in emergency departments. Clinical guidelines recommend performing plain x-rays as the primary modality of diagnosis. Although point-of-care ultrasound (PoCUS) has increasingly been integrated into the daily practice of emergency medicine, it has been poorly investigated in the diagnostic approach for FBI. This review aims to highlight the current state of PoCUS use for pediatric FBI. The following research question was considered in this narrative review: Is PoCUS useful for FBI management? Methods: A literature search was conducted to identify articles reporting PoCUS use for the management of FBI. All articles were assessed for quality by 2 reviewers. Main Results: The 14 selected articles reported 52 FBI cases in which PoCUS successfully identified and located the ingested FB. Point-of-care ultrasound was used either as the primary imaging technique or after positive or negative x-ray findings. In 5 cases (9.6%), PoCUS was the only modality used for the diagnosis. Of these cases, 3 (60%) underwent a successful procedure to remove the FB and 2 (40%) received conservative treatment without complications. Conclusions: This review suggests that PoCUS might be a reliable modality for the initial management of FBI. PoCUS can locate, identify, and evaluate the size of the FB in a wide range of materials and gastrointestinal locations. Point-of-care ultrasound could eventually become the go-to modality in the case of radiolucent FB, thus avoiding the use of radiation. Further studies are nevertheless required to validate PoCUS use for FBI management.