Lotty Hooft

Lotty Hooft

Auto-generated profile, based on publicly available information

Lotty Hooft

Julius Center for Health Sciences and Primary care, University Medical Center Utrecht, Utrecht University, Utrecht, the Netherlands

Research Focus & Metrics

Academic Career Length

13Years (2014-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
24 March 2025 in BMJ
Verified
Journal Article
Open access
PDF file
KMKarel G M Moons
Karel G M Moons
JDJohanna A A Damen
Johanna A A Damen
TKTabea Kaul
Tabea Kaul
LHLotty Hooft
Lotty Hooft
CNConstanza Andaur Navarro
Constanza Andaur Navarro
MSMaarten van Smeden
Maarten van Smeden
The Prediction model Risk Of Bias ASsessment Tool (PROBAST) is used to assess the quality, risk of bias, and applicability of prediction models or algorithms and of prediction model/algorithm studies. Since PROBAST’s introduction in 2019, much progress has been made in the methodology for prediction modelling and in the use of artificial intelligence, including machine learning, techniques. An update to PROBAST-2019 is thus needed. This article describes the development of PROBAST+AI. PROBAST+AI consists of two distinctive parts: model development and model evaluation. For model development, PROBAST+AI users assess quality and applicability using 16 targeted signalling questions. For model evaluation, PROBAST+AI users assess the risk of bias and applicability using 18 targeted signalling questions. Both parts contain four domains: participants and data sources, predictors, outcome, and analysis. Applicability of the prediction model is rated for the participants and data sources, predictors, and outcome domains. PROBAST+AI may replace the original PROBAST tool and allows all key stakeholders (eg, model developers, AI companies, researchers, editors, reviewers, healthcare professionals, guideline developers, and policy organisations) to examine the quality, risk of bias, and applicability of any type of prediction model in the healthcare sector, irrespective of whether regression modelling or AI techniques are used.

20 October 2021 in BMJ
Verified
Journal Article
Open access
PDF file
CNConstanza L Andaur Navarro
Constanza L Andaur Navarro
JDJohanna A A Damen
Johanna A A Damen
TTToshihiko Takada
Toshihiko Takada
SNSteven W J Nijman
Steven W J Nijman
PDPaula Dhiman
Paula Dhiman
LHLotty Hooft
Lotty Hooft
Objective To assess the methodological quality of studies on prediction models developed using machine learning techniques across all medical specialties. Design Systematic review. Data sources PubMed from 1 January 2018 to 31 December 2019. Eligibility criteria Articles reporting on the development, with or without external validation, of a multivariable prediction model (diagnostic or prognostic) developed using supervised machine learning for individualised predictions. No restrictions applied for study design, data source, or predicted patient related health outcomes. Review methods Methodological quality of the studies was determined and risk of bias evaluated using the prediction risk of bias assessment tool (PROBAST). This tool contains 21 signalling questions tailored to identify potential biases in four domains. Risk of bias was measured for each domain (participants, predictors, outcome, and analysis) and each study (overall). Results 152 studies were included: 58 (38%) included a diagnostic prediction model and 94 (62%) a prognostic prediction model. PROBAST was applied to 152 developed models and 19 external validations. Of these 171 analyses, 148 (87%, 95% confidence interval 81% to 91%) were rated at high risk of bias. The analysis domain was most frequently rated at high risk of bias. Of the 152 models, 85 (56%, 48% to 64%) were developed with an inadequate number of events per candidate predictor, 62 handled missing data inadequately (41%, 33% to 49%), and 59 assessed overfitting improperly (39%, 31% to 47%). Most models used appropriate data sources to develop (73%, 66% to 79%) and externally validate the machine learning based prediction models (74%, 51% to 88%). Information about blinding of outcome and blinding of predictors was, however, absent in 60 (40%, 32% to 47%) and 79 (52%, 44% to 60%) of the developed models, respectively. Conclusion Most studies on machine learning based prediction models show poor methodological quality and are at high risk of bias. Factors contributing to risk of bias include small study size, poor handling of missing data, and failure to deal with overfitting. Efforts to improve the design, conduct, reporting, and validation of such studies are necessary to boost the application of machine learning based prediction models in clinical practice. Systematic review registration PROSPERO CRD42019161764.

23 May 2014 in BMC Medical Research Methodology
Verified
Journal Article
Open access
PDF file
WEW Annefloor van Enst
W Annefloor van Enst
EOEleanor Ochodo
Eleanor Ochodo
RSRob Jpm Scholten
Rob Jpm Scholten
LHLotty Hooft
Lotty Hooft
MLMariska M Leeflang
Mariska M Leeflang
The validity of a meta-analysis can be understood better in light of the possible impact of publication bias. The majority of the methods to investigate publication bias in terms of small study-effects are developed for meta-analyses of intervention studies, leaving authors of diagnostic test accuracy (DTA) systematic reviews with limited guidance. The aim of this study was to evaluate if and how publication bias was assessed in meta-analyses of DTA, and to compare the results of various statistical methods used to assess publication bias.

13 June 2019 in BMC Medicine
Verified
Journal Article
Open access
PDF file
JDJohanna A. Damen
Johanna A. Damen
RPRomin Pajouheshnia
Romin Pajouheshnia
PHPauline Heus
Pauline Heus
KMKarel G. M. Moons
Karel G. M. Moons
JRJohannes B. Reitsma
Johannes B. Reitsma
TDThomas P. Debray
Thomas P. Debray
The Framingham risk models and pooled cohort equations (PCE) are widely used and advocated in guidelines for predicting 10-year risk of developing coronary heart disease (CHD) and cardiovascular disease (CVD) in the general population. Over the past few decades, these models have been extensively validated within different populations, which provided mounting evidence that local tailoring is often necessary to obtain accurate predictions. The objective is to systematically review and summarize the predictive performance of three widely advocated cardiovascular risk prediction models (Framingham Wilson 1998, Framingham ATP III 2002 and PCE 2013) in men and women separately, to assess the generalizability of performance across different subgroups and geographical regions, and to determine sources of heterogeneity in the findings across studies. A search was performed in October 2017 to identify studies investigating the predictive performance of the aforementioned models. Studies were included if they externally validated one or more of the original models in the general population for the same outcome as the original model. We assessed risk of bias for each validation and extracted data on population characteristics and model performance. Performance estimates (observed versus expected (OE) ratio and c-statistic) were summarized using a random effects models and sources of heterogeneity were explored with meta-regression. The search identified 1585 studies, of which 38 were included, describing a total of 112 external validations. Results indicate that, on average, all models overestimate the 10-year risk of CHD and CVD (pooled OE ratio ranged from 0.58 (95% CI 0.43–0.73; Wilson men) to 0.79 (95% CI 0.60–0.97; ATP III women)). Overestimation was most pronounced for high-risk individuals and European populations. Further, discriminative performance was better in women for all models. There was considerable heterogeneity in the c-statistic between studies, likely due to differences in population characteristics. The Framingham Wilson, ATP III and PCE discriminate comparably well but all overestimate the risk of developing CVD, especially in higher risk populations. Because the extent of miscalibration substantially varied across settings, we highly recommend that researchers further explore reasons for overprediction and that the models be updated for specific populations.

21 April 2016 in BMC Medical Informatics and Decision Making
Verified
Journal Article
Open access
PDF file
JWJulia C. M. van Weert
Julia C. M. van Weert
BMBarbara C. van Munster
Barbara C. van Munster
RSRemco Sanders
Remco Sanders
RSRené Spijker
René Spijker
LHLotty Hooft
Lotty Hooft
JJJesse Jansen
Jesse Jansen
Decision aids have been overall successful in improving the quality of health decision making. However, it is unclear whether the impact of the results of using decision aids also apply to older people (aged 65+). We sought to systematically review randomized controlled trials (RCTs) and clinical controlled trials (CCTs) evaluating the efficacy of decision aids as compared to usual care or alternative intervention(s) for older adults facing treatment, screening or care decisions. A systematic search of (1) a Cochrane review of decision aids and (2) MEDLINE, Embase, PsycINFO, Cochrane library central registry of studies and Cinahl. We included published RCTs/CCTs of interventions designed to improve shared decision making (SDM) by older adults (aged 65+) and RCTs/CCTs that analysed the effect of the intervention in a subgroup with a mean age of 65+. Based on the International Patient Decision aid Standards (IPDAS), the primary outcomes were attributes of the decision and the decision process. Other behavioral, health, and health system effects were considered as secondary outcomes. If data could be pooled, a meta-analysis was conducted. Data for which meta-analysis was not possible were synthesized qualitatively. The search strategy yielded 11,034 references. After abstract and full text screening, 22 papers were included. Decision aids performed better than control resp. usual care interventions by increasing knowledge and accurate risk perception in older people (decision attributes). With regard to decision process attributes, decision aids resulted in lower decisional conflict and more patient participation. This review shows promising results on the effectiveness of decision aids for older adults. Decision aids improve older adults’ knowledge, increase their risk perception, decrease decisional conflict and seem to enhance participation in SDM. It must however be noted that the body of literature on the effectiveness of decision aids for older adults is still in its infancy. Only one decision aid was specifically developed for older adults, and the mean age in most studies was between 65 and 70, indicating that the oldest-old were not included. Future research should expand on the design, application and evaluation of decision aids for older, more vulnerable adults.

19 July 2018 in BMC Medicine
Verified
Journal Article
Open access
PDF file
PHPauline Heus
Pauline Heus
JDJohanna A. A. G. Damen
Johanna A. A. G. Damen
RPRomin Pajouheshnia
Romin Pajouheshnia
RSRob J. P. M. Scholten
Rob J. P. M. Scholten
JRJohannes B. Reitsma
Johannes B. Reitsma
LHLotty Hooft
Lotty Hooft
As complete reporting is essential to judge the validity and applicability of multivariable prediction models, a guideline for the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) was introduced. We assessed the completeness of reporting of prediction model studies published just before the introduction of the TRIPOD statement, to refine and tailor its implementation strategy. Within each of 37 clinical domains, 10 journals with the highest journal impact factor were selected. A PubMed search was performed to identify prediction model studies published before the launch of TRIPOD in these journals (May 2014). Eligible publications reported on the development or external validation of a multivariable prediction model (either diagnostic or prognostic) or on the incremental value of adding a predictor to an existing model. We included 146 publications (84% prognostic), from which we assessed 170 models: 73 (43%) on model development, 43 (25%) on external validation, 33 (19%) on incremental value, and 21 (12%) on combined development and external validation of the same model. Overall, publications adhered to a median of 44% (25th–75th percentile 35–52%) of TRIPOD items, with 44% (35–53%) for prognostic and 41% (34–48%) for diagnostic models. TRIPOD items that were completely reported for less than 25% of the models concerned abstract (2%), title (5%), blinding of predictor assessment (6%), comparison of development and validation data (11%), model updating (14%), model performance (14%), model specification (17%), characteristics of participants (21%), model performance measures (methods) (21%), and model-building procedures (24%). Most often reported were TRIPOD items regarding overall interpretation (96%), source of data (95%), and risk groups (90%). More than half of the items considered essential for transparent reporting were not fully addressed in publications of multivariable prediction model studies. Essential information for using a model in individual risk prediction, i.e. model specifications and model performance, was incomplete for more than 80% of the models. Items that require improved reporting are title, abstract, and model-building procedures, as they are crucial for identification and external validation of prediction models.