Objectives We propose and test a framework to detect disease diagnosis using a recent large language model (LLM), Meta’s Llama-3-8B, on French-language electronic health record (EHR) documents. Specifically, it focuses on detecting gout (‘goutte’ in French), a ubiquitous French term that has multiple meanings beyond the disease. The study compares the performance of the LLM-based framework with traditional natural language processing techniques and tests its dependence on the parameter used. Methods The framework was developed using a training and testing set of 700 paragraphs assessing ‘gout’ from a random selection of EHR documents from a tertiary university hospital in Geneva, Switzerland. All paragraphs were manually reviewed and classified by two healthcare professionals into disease (true gout) and non-disease (gold standard). The LLM’s accuracy was tested using few-shot and chain-of-thought prompting and compared with a regular expression (regex)-based method, focusing on the effects of model parameters and prompt structure. The framework was further validated on 600 paragraphs assessing ‘Calcium Pyrophosphate Deposition Disease (CPPD)’. Results The LLM-based algorithm outperformed the regex method, achieving a 92.7% (88.7%–95.4%) positive predictive value, a 96.6% (94.6%–97.8%) negative predictive value and an accuracy of 95.4% (93.6%–96.7%) for gout. In the validation set on CPPD, accuracy was 94.1% (90.2%–97.6%). The LLM framework performed well over a wide range of parameter values. Conclusion LLMs accurately detected disease diagnoses from EHRs, even in non-English languages. They could facilitate creating large disease registers in any language, improving disease care assessment and patient recruitment for clinical trials.
Objectives: No French validated concise scales are available for measuring the experience of inpatients in pediatrics. This study aims to adapt the adult PPE-15 to a pediatric population, and translating it in French, as well as to establish reference values for adults, teenagers, and parents of young children.Methods: Cultural adaptation involved forward and backward translations, along with pretests in all three populations. Dimensional structure and internal consistency were assessed using principal component analysis, exploratory factor analysis, and Cronbach's alpha. Construct validity was assessed by examining established associations between patient satisfaction and inpatient variables, including length of stay, and preventable readmission.Results: A total of 25,626 adults, 293 teenagers and 1,640 parents of young children completed the French questionnaires. Factor analysis supported a single dimension (Cronbach's alpha: adults: 0.85, teenagers: 0.82, parents: 0.80). Construct validity showed the expected pattern of association, with dissatisfaction correlating with patient- and stay-related factors, notably length of stay, and readmission.Conclusion: The French versions of the PPE-15 for adults, teenagers and parents of pediatric patients stand as valid and reliable instruments for gauging patient satisfaction regarding their hospital stay after discharge.
Objective To develop an automatic gout register from electronic health records (EHRs) data. Methods We analysed the EHR of all patients >18 years old from a tertiary academic hospital (2013–2022) based on six criteria: International Classification of Diseases 10 gout diagnosis, urate-lowering therapy prescription, monosodium urate crystals in joint aspiration and gout-related terms in problem lists, clinical or imaging reports. We assessed the positive and negative predictive value (PPV and NPV) of the query by chart reviews. Results Of 2 110 902 outpatients and inpatients, 10 289 had at least one criterion for gout. The combination of joint aspiration OR diagnostic in the problem list OR≥2 other criteria created a register of 5138 patients, with a PPV of 92.4% (95% CI 88.5% to 95.0%) and an NPV of 94.3% (95% CI 91.9% to 96.0%). PPV and NPV were similar among outpatients and inpatients. Incidence was 2.9 per 1000 person-year and dropped by 30% from the COVID-19 pandemic onward. Patients with gout were on average 71.2 years old (SD 14.9), mainly male (76.5%), overweight (69.5%) and polymorbid (mean number of comorbidities of 3, IQR 1–5). More than half (57.4%) had received a urate-lowering treatment, 6.7% had a gout that led to a hospitalisation or ≥2 flares within a year and 32.9% received a rheumatology consultation. Conclusion An automatic EHR-based gout register is feasible, valid and could be used to evaluate and improve gout management. Interestingly, the register uncovered a marked underdiagnosis or under-reporting of gout since the COVID-19 pandemic.
(1) Background: France implemented a COVID-19 certificate in July 2021 to incentivize the population to uptake COVID-19 vaccines. However, little is known about the variation in its impact across age groups and its dependence on socio-demographic, economic, logistic, or political factors. (2) Methods: Using France’s weekly first dose vaccination rate, a counterfactual trend approach allowed for the estimation of the vaccination rate across age groups at a small geographical level before and after the implementation of the health pass. The effect of the health pass was operationalized as the vaccination rate among those who would not be vaccinated without it. (3) Results: Vaccination before the health pass varied greatly among age groups and was mainly influenced by territory (lower in rural and overseas territories when compared to urban and metropolitan ones), political beliefs, and socio-economic disparities. Vaccine logistics played a minor but significant role, while the impact of COVID-19 did not affect the vaccination rate. The health pass increased the vaccination overall but with varying efficiency across groups. It convinced mainly young people politically close to the governmental vaccination strategy and living in urban metropolitan areas with low socio-economical discrepancies. The selected variables explained most of the variability of the vaccination rate before the health pass; they explained, at most, a third of the variation in the health pass effect on vaccination. (4) Conclusions: From a public health perspective, the French health pass increased the overall vaccination, but failed to promote preventive behaviours in all segments of society, particularly in vulnerable communities.
Importance The use of large language models (LLMs) in medicine is increasing, with potential applications in electronic health records (EHR) to create patient cohorts or identify patients who meet clinical trial recruitment criteria. However, significant barriers remain, including the extensive computer resources required, lack of performance evaluation, and challenges in implementation. Objective This study aims to propose and test a framework to detect disease diagnosis using a recent light LLM on French-language EHR documents. Specifically, it focuses on detecting gout (“goutte” in French), a ubiquitous French term that have multiple meanings beyond the disease. The study will compare the performance of the LLM-based framework with traditional natural language processing techniques and test its dependence on the parameter used. Design The framework was developed using a training and testing set of 700 paragraphs assessing “gout”, issued from a random selection of retrospective EHR documents. All paragraphs were manually reviewed and classified by two health-care professionals (HCP) into disease (true gout) and non-disease (gold standard). The LLM’s accuracy was tested using few-shot and chain-of-thought prompting and compared to a regular expression (regex)-based method, focusing on the effects of model parameters and prompt structure. The framework was further validated on 600 paragraphs assessing “Calcium Pyrophosphate Deposition Disease (CPPD)”. Setting The documents were sampled from the electronic health-records of a tertiary university hospital in Geneva, Switzerland. Participants Adults over 18 years of age. Exposure Meta’s Llama 3 8B LLM or traditional method, against a gold standard. Main Outcomes and Measures Positive and negative predictive value, as well as accuracy of tested models. Results The LLM-based algorithm outperformed the regex method, achieving a 92.7% [88.7-95.4%] positive predictive value, a 96.6% [94.6-97.8%] negative predictive value, and an accuracy of 95.4% [93.6-96.7%] for gout. In the validation set on CPPD, accuracy was 94.1% [90.2-97.6%]. The LLM framework performed well over a wide range of parameter values. Conclusions and Relevance LLMs were able to accurately detect disease diagnoses from EHRs, even in non-English languages. They could facilitate creating large disease registries in any language, improving disease care assessment and patient recruitment for clinical trials. Question How accurate and efficient are large language models (LLMs) in detecting diseases from unstructured electronic health records (EHR) text compared to traditional natural language processing techniques? Findings This study proposes a framework based on Meta’s Llama 3 8B, a recent public LLM, outperforming traditional natural language processing techniques in detecting gout and calcium pyrophosphate deposition disease in unstructured text. It achieves high positive and negative predictive values and accuracy. Performance was robust over a wide range of parameters. Meaning The proposed framework can ease the use of LLMs in effectively detecting disease in EHR data for various clinical applications.
5 August 2024 in European Journal Of Cancer Prevention
Objective: The objective of this study is to examine how the effect of organized mammography screening programs on breast cancer screening participation differ between socioeconomic strata and how this relationship may be modified by the context of linguistic differences. Switzerland, marked by its diverse linguistic landscape, reflects cultural variations alongside differences in public health strategies. The goal of this study was to assess potential socioeconomic differences in regional mammography screening programs effectiveness to improve breast cancer screening participation. Methods: Data on 14 173 women in the regionally adapted breast cancer screening age range was drawn from five cross-sectional waves of the nationally representative Swiss Health Interview Survey (1997–2017). Socioeconomic indicators included education, household income, and employment status. Poisson regression was used to estimate the adjusted prevalence ratios of up-to-date (last 2 years) mammography uptake. Inequality was assessed using relative index of inequality and the slope index of inequality. Results: Organized screening programs were generally effective and increased up-to-date mammography uptake by close to 20 percentage points in both regions. While in the Latin cantons, screening programs had no impact on socioeconomic inequalities in screening, it reduced inequalities for women with lower education in the German cantons. This modification effect of screening programs was not seen for income and employment-related inequalities and did not differ across linguistic regions. Conclusions: Public health agencies should consider the different cultural reception of programs as addressing these differences could help ensure that breast cancer screening initiatives are not only effective, but also culturally equitable across different socioeconomic groups.