Background: Generative artificial intelligence (GenAI) has the potential to revolutionise healthcare delivery. The nuances of real-life clinical practice and complex clinical environments demand a rigorous, evidence-based approach to ensure safe and effective deployment of AI. Methods: We present a protocol for the systematic evaluation of large language models (LLMs) as GenAI chatbots within the context of clinical microbiology and infectious diseases clinical consultations. We aim to critically assess recommendations produced by four leading GenAI models, including Claude 2, Gemini Pro, GPT-4.0, and a GPT-4.0-based custom AI chatbot. Discussion: A standardised, healthcare-specific, universal prompt template is developed to elicit clinically impactful AI responses. Generated responses will be graded by two panels of practicing clinicians, encompassing a wide spectrum of domain expertise in clinical microbiology and virology, as well as infectious diseases. Evaluations will be performed using a 5-point Likert scale across four clinical domains: factual consistency, comprehensiveness, coherence, and medical harmfulness. Our study will offer insights into the feasibility, limitations, and boundaries of GenAI in clinical consultations, providing guidance for future research and clinical implementation. Ethical guidelines and safety guardrails should be developed to uphold patient safety and clinical standards.

Background/Objectives: The potential of generative artificial intelligence (GenAI) to augment clinical consultation services in clinical microbiology and infectious diseases (ID) is being evaluated. Methods: This cross-sectional study evaluated the performance of four GenAI chatbots (GPT-4.0, a Custom Chatbot based on GPT-4.0, Gemini Pro, and Claude 2) by analysing 40 unique clinical scenarios. Six specialists and resident trainees from clinical microbiology or ID units conducted randomised, blinded evaluations across factual consistency, comprehensiveness, coherence, and medical harmfulness. Results: Analysis showed that GPT-4.0 achieved significantly higher composite scores compared to Gemini Pro (p = 0.001) and Claude 2 (p = 0.006). GPT-4.0 outperformed Gemini Pro and Claude 2 in factual consistency (Gemini Pro, p = 0.02; Claude 2, p = 0.02), comprehensiveness (Gemini Pro, p = 0.04; Claude 2, p = 0.03), and the absence of medical harm (Gemini Pro, p = 0.02; Claude 2, p = 0.04). Within-group comparisons showed that specialists consistently awarded higher ratings than resident trainees across all assessed domains (p < 0.001) and overall composite scores (p < 0.001). Specialists were five times more likely to consider responses as “harmless”. Overall, fewer than two-fifths of AI-generated responses were deemed “harmless”. Post hoc analysis revealed that specialists may inadvertently disregard conflicting or inaccurate information in their assessments. Conclusions: Clinical experience and domain expertise of individual clinicians significantly shaped the interpretation of AI-generated responses. In our analysis, we have demonstrated disconcerting human vulnerabilities in safeguarding against potentially harmful outputs, which seemed to be most apparent among experienced specialists. At the current stage, none of the tested AI models should be considered safe for direct clinical deployment in the absence of human supervision.

Background Generative artificial intelligence (AI) technology has the revolutionary potentials to augment clinical practice and telemedicine. The nuances of real-life patient scenarios and complex clinical environments demand a rigorous, evidence-based approach to ensure safe and effective application. Methods We present a protocol for the systematic evaluation of generative AI large language models (LLMs) as chatbots within the context of clinical microbiology and infectious disease consultations. We aim to critically assess the clinical accuracy, comprehensiveness, coherence, and safety of recommendations produced by leading generative AI models, including Claude 2, Gemini Pro, GPT-4.0, and a GPT-4.0-based custom AI chatbot. Discussion A standardised healthcare-specific prompt template is employed to elicit clinically impactful AI responses. Generated responses will be graded by a panel of human evaluators, encompassing a wide spectrum of domain expertise in clinical microbiology and virology and clinical infectious diseases. Evaluations are performed using a 5-point Likert scale across four clinical domains: factual consistency, comprehensiveness, coherence, and medical harmfulness. Our study will offer insights into the feasibility, limitations, and boundaries of generative AI in healthcare, providing guidance for future research and clinical implementation. Ethical guidelines and safety guardrails should be developed to uphold patient safety and clinical standards.

Background: Infection prevention and control (IPC) in hospitals relies heavily on infection control nurses (ICNs) who manage complex consultations to prevent and control infections. This study evaluated large language models (LLMs) as artificial intelligence (AI) tools to support ICNs in IPC decision-making processes. Our goal is to enhance the efficiency of IPC practices while maintaining the highest standards of safety and accuracy. Methods: A cross-sectional benchmarking study at Queen Mary Hospital, Hong Kong assessed three LLMs—GPT-4.1, DeepSeek V3, and Gemini 2.5 Pro Exp—using 30 clinical infection control scenarios. Each model generated clarifying questions to understand the scenarios before providing IPC recommendations through two prompting methods: an open-ended inquiry and a structured template. Sixteen experts, including senior and junior ICNs and physicians, rated these responses on coherence, conciseness, usefulness and relevance, evidence quality, and actionability (1–10 scale). Quantitative and qualitative analyses assessed AI performance, reliability, and clinical applicability. Results: GPT-4.1 and DeepSeek V3 scored significantly higher on the composite quality scale, with adjusted means (95% CI) of 36.77 (33.98–39.57) and 36.25 (33.45–39.04), respectively, compared with Gemini 2.5 Pro Exp at 33.19 (30.39–35.99) (p < 0.001). GPT-4.1 led in evidence quality, usefulness, and relevance. Gemini 2.5 Pro Exp failed to generate responses in 50% of scenarios under structured prompt conditions. Structured prompting yielded significant improvements, primarily by enhancing evidence quality (p < 0.001). Evaluator background influenced scoring, with doctors rating outputs higher than nurses (38.83 vs. 32.06, p < 0.001). However, a qualitative review revealed critical deficiencies across all models, for example, tuberculosis treatment solely based on a positive acid-fast bacilli (AFB) smear without considering nontuberculous mycobacteria in DeepSeek V3 and providing an impractical and noncommittal response regarding the de-escalation of precautions for Candida auris in Gemini 2.5 Pro Exp. These errors highlight potential safety risks and limited real-world applicability, despite generally positive scores. Conclusions: While GPT-4.1 and DeepSeek V3 deliver useful IPC advice, they are not yet reliable for autonomous use. Critical errors in clinical judgment and practical applicability highlight that LLMs cannot replace the expertise of ICNs. These technologies should serve as adjunct tools to support, rather than automate, clinical decision-making.

In 2022, an outbreak of melioidosis occurred at a highly urbanised district (the SSP district) in Hong Kong. There was a 5-fold increase in cases compared to the average number of cases in the preceding years, with the SSP district accounting for 71.1% of the cases. A total of 568 environmental specimens were collected, including 290 environmental samples and 38 air samples from the freshwater service reservoirs (FWSRs), 16 water samples, and 224 environmental samples from the households of infected cases. Burkholderia pseudomallei was cultured from 24 (10.7%) soil samples obtained from the lawn overlying the FWSRs supplying the SSP district or from the surroundings of the air vents used for pressure equalisation in the FWSRs. These culture isolates from soil have the same genotype ST-1996 as the patients' isolates. As B. pseudomallei DNA was also detected from the swabs collected from the internal roofs of the FWSRs and inside the water tap at the home of an infected case, the chlorine concentration at the upstream treatment plant outlet was increased from 1 ppm to 1.2 ppm as a precautionary measure. The number and proportion of newly diagnosed melioidosis cases from the SSP district significantly reduced from 32 (71.1%) in 2022 to 6 (35.3%) in 2023 (p = 0.018).

Hand hygiene is vital for infection prevention, yet compliance among medical staff remains low. This study assessed whether training medical students using