Background: Large language model (LLM)–based artificial intelligence chatbots direct the power of large training data sets toward successive, related tasks as opposed to single-ask tasks, for which artificial intelligence already achieves impressive performance. The capacity of LLMs to assist in the full scope of iterative clinical reasoning via successive prompting, in effect acting as artificial physicians, has not yet been evaluated. Objective: This study aimed to evaluate ChatGPT’s capacity for ongoing clinical decision support via its performance on standardized clinical vignettes. Methods: We inputted all 36 published clinical vignettes from the Merck Sharpe & Dohme (MSD) Clinical Manual into ChatGPT and compared its accuracy on differential diagnoses, diagnostic testing, final diagnosis, and management based on patient age, gender, and case acuity. Accuracy was measured by the proportion of correct responses to the questions posed within the clinical vignettes tested, as calculated by human scorers. We further conducted linear regression to assess the contributing factors toward ChatGPT’s performance on clinical tasks. Results: ChatGPT achieved an overall accuracy of 71.7% (95% CI 69.3%-74.1%) across all 36 clinical vignettes. The LLM demonstrated the highest performance in making a final diagnosis with an accuracy of 76.9% (95% CI 67.8%-86.1%) and the lowest performance in generating an initial differential diagnosis with an accuracy of 60.3% (95% CI 54.2%-66.6%). Compared to answering questions about general medical knowledge, ChatGPT demonstrated inferior performance on differential diagnosis (β=–15.8%; P<.001) and clinical management (β=–7.4%; P=.02) question types. Conclusions: ChatGPT achieves impressive accuracy in clinical decision-making, with increasing strength as it gains more clinical information at its disposal. In particular, ChatGPT demonstrates the greatest accuracy in tasks of final diagnosis as compared to initial diagnosis. Limitations include possible model hallucinations and the unclear composition of ChatGPT’s training data set.
BACKGROUND: Extreme temperatures contribute significantly to global mortality. While previous studies on temperature and stroke-specific outcomes presented conflicting results, these studies were predominantly limited to single-city or single-country analyses. Their findings are difficult to synthesize due to variations in methodologies and exposure definitions. METHODS: Within the Multi-Country Multi-City Network, we built a new mortality database for ischemic and hemorrhagic stroke. Applying a unified analysis protocol, we conducted a multinational case-crossover study on the relationship between extreme temperatures and stroke. In the first stage, we fitted a conditional quasi-Poisson regression for daily mortality counts with distributed lag nonlinear models for temperature exposure separately for each city. In the second stage, the cumulative risk from each city was pooled using mixed-effect meta-analyses, accounting for clustering of cities with similar features. We compared temperature-stroke associations across country-level gross domestic product per capita. We computed excess deaths in each city that are attributable to the 2.5% hottest and coldest of days based on each city’s temperature distribution. RESULTS: We collected data for a total of 3 443 969 ischemic strokes and 2 454 267 hemorrhagic stroke deaths from 522 cities in 25 countries. For every 1000 ischemic stroke deaths, we found that extreme cold and hot days contributed 9.1 (95% empirical CI, 8.6–9.4) and 2.2 (95% empirical CI, 1.9–2.4) excess deaths, respectively. For every 1000 hemorrhagic stroke deaths, extreme cold and hot days contributed 11.2 (95% empirical CI, 10.9–11.4) and 0.7 (95% empirical CI, 0.5–0.8) excess deaths, respectively. We found that countries with low gross domestic product per capita were at higher risk of heat-related hemorrhagic stroke mortality than countries with high gross domestic product per capita ( P =0.02). CONCLUSIONS: Both extreme cold and hot temperatures are associated with an increased risk of dying from ischemic and hemorrhagic strokes. As climate change continues to exacerbate these extreme temperatures, interventional strategies are needed to mitigate impacts on stroke mortality, particularly in low-income countries.
The COVID-19 pandemic is challenging nations with devastating health and economic consequences. The spread of the disease has revealed major geographical heterogeneity because of regionally varying individual behaviour and mobility patterns, unequal meteorological conditions, diverse viral variants, and locally implemented non-pharmaceutical interventions and vaccination roll-out. To support national and regional authorities in surveilling and controlling the pandemic in real-time as it unfolds, we here develop a newregionalmathematical and statistical model. The model, which has been in use in Norway during the first two years of the pandemic, is informed by real-time mobility estimates from mobile phone data and laboratory-confirmed case and hospitalisation incidence. To estimate regional and time-varying transmissibility, case detection probabilities, and missed imported cases, we developed a novel sequential Approximate Bayesian Computation method allowing inference in useful time, despite the high parametric dimension. We test our approach on Norway and find that three-week-ahead predictions are precise and well-calibrated, enabling policy-relevant situational awareness at a local scale. By comparing the reproduction numbers before and after lockdowns, we identify spatially heterogeneous patterns in their effect on the transmissibility, with a stronger effect in the most populated regions compared to the national reduction estimated to be 85% (95% CI 78%-89%). Our approach is the first regional changepoint stochastic metapopulation model capable of real time spatially refined surveillance and forecasting during emergencies.
Background: Given the societal, economic and health costs of COVID-19 non-pharmaceutical interventions (NPI), it is important to assess their effects. Human mobility serves as a surrogate measure for human contacts and compliance with NPI. In Nordic countries, NPI have mostly been advised and sometimes made mandatory. It is unclear if making NPI mandatory further reduced mobility. Aim: We investigated the effect of non-compulsory and follow-up mandatory measures in major cities and rural regions on human mobility in Norway. We identified NPI categories that most affected mobility. Methods: We used mobile phone mobility data from the largest Norwegian operator. We analysed non-compulsory and mandatory measures with before–after and synthetic difference-in-differences approaches. By regression, we investigated the impact of different NPI on mobility. Results: Nationally and in less populated regions, time travelled, but not distance, decreased after follow-up mandatory measures. In urban areas, however, distance decreased after follow-up mandates, and the reduction exceeded the decrease after initial non-compulsory measures. Stricter metre rules, gyms reopening, and restaurants and shops reopening were significantly associated with changes in mobility. Conclusion: Overall, distance travelled from home decreased after non-compulsory measures, and in urban areas, distance further decreased after follow-up mandates. Time travelled reduced more after mandates than after non-compulsory measures for all regions and interventions. Stricter distancing and reopening of gyms, restaurants and shops were associated with changes in mobility.
19 September 2022 in Infection Control & Hospital Epidemiology
Many data-driven patient risk stratification models have not been evaluated prospectively. We performed and compared the prospective and retrospective evaluations of 2 Clostridioides difficile infection (CDI) risk-prediction models at 2 large academic health centers, and we discuss the models’ robustness to data-set shifts.
BACKGROUND Large language model (LLM)–based artificial intelligence chatbots direct the power of large training data sets toward successive, related tasks as opposed to single-ask tasks, for which artificial intelligence already achieves impressive performance. The capacity of LLMs to assist in the full scope of iterative clinical reasoning via successive prompting, in effect acting as artificial physicians, has not yet been evaluated. OBJECTIVE This study aimed to evaluate ChatGPT’s capacity for ongoing clinical decision support via its performance on standardized clinical vignettes. METHODS We inputted all 36 published clinical vignettes from the Merck Sharpe & Dohme (MSD) Clinical Manual into ChatGPT and compared its accuracy on differential diagnoses, diagnostic testing, final diagnosis, and management based on patient age, gender, and case acuity. Accuracy was measured by the proportion of correct responses to the questions posed within the clinical vignettes tested, as calculated by human scorers. We further conducted linear regression to assess the contributing factors toward ChatGPT’s performance on clinical tasks. RESULTS ChatGPT achieved an overall accuracy of 71.7% (95% CI 69.3%-74.1%) across all 36 clinical vignettes. The LLM demonstrated the highest performance in making a final diagnosis with an accuracy of 76.9% (95% CI 67.8%-86.1%) and the lowest performance in generating an initial differential diagnosis with an accuracy of 60.3% (95% CI 54.2%-66.6%). Compared to answering questions about general medical knowledge, ChatGPT demonstrated inferior performance on differential diagnosis (β=–15.8%; P<.001) and clinical management (β=–7.4%; P=.02) question types. CONCLUSIONS ChatGPT achieves impressive accuracy in clinical decision-making, with increasing strength as it gains more clinical information at its disposal. In particular, ChatGPT demonstrates the greatest accuracy in tasks of final diagnosis as compared to initial diagnosis. Limitations include possible model hallucinations and the unclear composition of ChatGPT’s training data set.