Florian D. van Leeuwen

Florian D. van Leeuwen

Auto-generated profile, based on publicly available information

Florian D. van Leeuwen

Department of Biomedical Data Sciences Leiden University Medical Center Leiden the Netherlands

Research Focus & Metrics

Academic Career Length

5Years (2022-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
8 February 2025 in Statistics in Medicine
Verified
Journal Article
Open access
FLFlorian D. van Leeuwen
Florian D. van Leeuwen
ESEwout W. Steyerberg
Ewout W. Steyerberg
DKDavid van Klaveren
David van Klaveren
BWBen Wessler
Ben Wessler
DKDavid M. Kent
David M. Kent
EZErik W. van Zwet
Erik W. van Zwet
Background: External validations are essential to assess the performance of a clinical prediction model (CPM) before deployment. Apart from model misspecification, also differences in patient population, the standard of care, predictor definitions, and other factors influence a model's discriminative ability, as commonly quantified by the AUC (or c‐statistic). We aimed to quantify the variation in AUCs across sets of external validation studies and propose ways to adjust expectations of a model's performance in a new setting. Methods: The Tufts‐PACE CPM Registry holds a collection of CPMs for prognosis in cardiovascular disease. We analyzed the AUC estimates of 469 CPMs with at least one external validation. Combined, these CPMs had a total of 1603 external validations reported in the literature. For each CPM and its associated set of validation studies, we performed a random‐effects meta‐analysis to estimate the between‐study standard deviation among the AUCs. Since the majority of these meta‐analyses have only a handful of validations, this leads to very poor estimates of . So, instead of focusing on a single CPM, we estimated a log‐normal distribution of across all 469 CPMs. We then used this distribution as an empirical prior. We used cross‐validation to compare this empirical Bayesian approach with frequentist fixed and random‐effects meta‐analyses. Results: The 469 CPMs included in our study had a median of 2 external validations with an IQR of [1–3]. The estimated distribution of had a mean of 0.055 and a standard deviation of 0.015. If = 0.05, then the 95% prediction interval for the AUC in a new setting has a width of at least 0.1, no matter how many validations have been done. When there are fewer than 5 validations, which is typically the case, the usual frequentist methods grossly underestimate the uncertainty about the AUC in a new setting. Accounting for in a Bayesian approach achieved near nominal coverage. Conclusion: Due to large heterogeneity among the validated AUC values of a CPM, there is great irreducible uncertainty in predicting the AUC in a new setting. This uncertainty is underestimated by existing methods. The proposed empirical Bayes approach addresses this problem which merits wide application in judging the validity of prediction models.

7 August 2024 in PLOS ONE
Verified
Journal Article
Open access
PDF file
FLFlorian D. van Leeuwen
Florian D. van Leeuwen
PLPeter Lugtig
Peter Lugtig
RFRemco Feskens
Remco Feskens
Interrupted time series (ITS) designs are increasingly used for estimating the effect of shocks in natural experiments. Currently, ITS designs are often used in scenarios with many time points and simple data structures. This research investigates the performance of ITS designs when the number of time points is limited and with complex data structures. Using a Monte Carlo simulation study, we empirically derive the performance—in terms of power, bias and precision- of the ITS design. Scenarios are considered with multiple interventions, a low number of time points and different effect sizes based on a motivating example of the learning loss due to COVID school closures. The results of the simulation study show the power of the step change depends mostly on the sample size, while the power of the slope change depends on the number of time points. In the basic scenario, with both a step and a slope change and an effect size of 30% of the pre-intervention slope, the required sample size for detecting a step change is 1,100 with a minimum of twelve time points. For detecting a slope change the required sample size decreases to 500 with eight time points. To decide if there is enough power researchers should inspect their data, hypothesize about effect sizes and consider an appropriate model before applying an ITS design to their research. This paper contributes to the field of methodology in two ways. Firstly, the motivation example showcases the difficulty of employing ITS designs in cases which do not adhere to a single intervention. Secondly, models are proposed for more difficult ITS designs and their performance is tested.


6 June 2024 in medRxiv
Verified
Preprint
Open access
PDF file
RVRick J.G. Vreeburg
Rick J.G. Vreeburg
FLFlorian D. van Leeuwen
Florian D. van Leeuwen
GMGeoffrey A. Manley
Geoffrey A. Manley
John K YueJohn K Yue
John K Yue
PBPaul M. Brennan
Paul M. Brennan
PAParticipants
Participants
Objective: To compare the incremental prognostic value of pupillary reactivity as captured in the GCS-Pupils score (GCS-P) or added as separate variable to the Glasgow Coma Scale (GCS) in traumatic brain injury (TBI). Methods: We analyzed patients enrolled between 2014 and 2018 in the Collaborative European NeuroTrauma Effectiveness Research in Traumatic Brain Injury (CENTER-TBI, n=3521) and the Transforming Research and Clinical Knowledge in Traumatic Brain Injury (TRACK-TBI, n=1439) cohorts. We used logistic regression to quantify the prognostic performances of GCS-P versus GCS according to Nagelkerke’s R 2 . Endpoints were mortality and unfavorable outcome (Glasgow Outcome Scale-Extended score 1-4) at 6 months after injury. We estimated 95% confidence intervals with bootstrap resampling to summarize the improvement in prognostic capability. Results: GCS as a linear score had a R 2 of 24% (95% confidence interval [CI] 17-30) and 30% (95%CI 17-43) for mortality and 29% (95%CI 25-34) and 38% (95%CI 29-47) for unfavorable outcome in CENTER-TBI and TRACK-TBI respectively. In the meta-analysis, pupillary reactivity as a separate variable improved the R 2 by an absolute value of 6% and 2% for mortality and unfavorable outcome (95%CI 4.0-7.7 and 1.2-3.0, respectively), with half the improvement captured in the GCS-P score (3%, 95%CI 2.1-3.3 and 1%, 95%CI 1-1.7, respectively). Conclusions: GCS-P has a stronger association with outcome after TBI than the GCS alone. However, for prognostic models, inclusion of GCS and pupillary reactivity as separate scores is preferable.

15 May 2024 in medRxiv
Verified
Preprint
Open access
PDF file
SBShubhayu Bhattacharyay
Shubhayu Bhattacharyay
FLFlorian D van Leeuwen
Florian D van Leeuwen
EBErta Beqiri
Erta Beqiri
CÅCecilia Åkerlund
Cecilia Åkerlund
LWLindsay Wilson
Lindsay Wilson
PAParticipants
Participants
Practices for controlling intracranial pressure (ICP) in traumatic brain injury (TBI) patients admitted to the intensive care unit (ICU) vary considerably between centres. To help understand the rational basis for such variance in care, this study aims to identify the patient-level predictors of changes in ICP management. We extracted all heterogeneous data (2,008 pre-ICU and ICU variables) collected from a prospective cohort ( n =844, 51 ICUs) of ICP-monitored TBI patients in the Collaborative European NeuroTrauma Effectiveness Research in TBI (CENTER-TBI) study. We developed the TILTomorrow modelling strategy, which leverages recurrent neural networks to map a token-embedded time series representation of all variables (including missing values) to an ordinal, dynamic prediction of the following day’s five-category therapy intensity level (TIL (Basic) ) score. With 20 repeats of 5-fold cross-validation, we trained TILTomorrow on different variable sets and applied the TimeSHAP (temporal extension of SHapley Additive exPlanations) algorithm to estimate variable contributions towards predictions of next-day changes in TIL (Basic) . Based on Somers’ D xy , the full range of variables explained 68% (95% CI: 65–72%) of the ordinal variation in next-day changes in TIL (Basic) on day one and up to 51% (95% CI: 45–56%) thereafter, when changes in TIL (Basic) became less frequent. Up to 81% (95% CI: 78–85%) of this explanation could be derived from non-treatment variables (i.e., markers of pathophysiology and injury severity), but the prior trajectory of ICU management significantly improved prediction of future de-escalations in ICP-targeted treatment. Whilst there was no significant difference in the predictive discriminability (i.e., area under receiver operating characteristic curve [AUC]) between next-day escalations (0.80 [95% CI: 0.77–0.84]) and de-escalations (0.79 [95% CI: 0.76– 0.82]) in TIL (Basic) after day two, we found specific predictor effects to be more robust with de-escalations. The most important predictors of day-to-day changes in ICP management included preceding treatments, age, space-occupying lesions, ICP, metabolic derangements, and neurological function. Serial protein biomarkers were also important and may serve a useful role in the clinical armamentarium for assessing therapeutic needs. Approximately half of the ordinal variation in day-to-day changes in TIL (Basic) after day two remained unexplained, underscoring the significant contribution of unmeasured factors or clinicians’ personal preferences in ICP treatment. At the same time, specific dynamic markers of pathophysiology associated strongly with changes in treatment intensity and, upon mechanistic investigation, may improve the timing and personalised targeting of future care.