The increasing availability and use of predictive models to facilitate informed decision making highlights the need for careful assessment of the validity of these models. In particular, models involving biomarkers require careful validation for two reasons: issues with overfitting when complex models involve a large number of biomarkers, and interlaboratory variation in assays used to measure biomarkers. In this article, we distinguish between internal and external statistical validation. Internal validation, involving training-testing splits of the available data or cross-validation, is a necessary component of the model building process and can provide valid assessments of model performance. External validation consists of assessing model performance on one or more data sets collected by different investigators from different institutions. External validation is a more rigorous procedure necessary for evaluating whether the predictive model will generalize to populations other than the one on which it was developed. We stress the need for an external data set to be truly external, that is, to play no role in model development and ideally be completely unavailable to the researchers building the model. In addition to reviewing different types of validation, we describe different types and features of predictive models and strategies for model building, as well as measures appropriate for assessing their performance in the context of validation. No single measure can characterize the different components of the prediction, and the use of multiple summary measures is recommended.

Adverse metabolic changes associated with loss of ovarian function increase the risk of developing metabolic syndrome and non-alcoholic fatty liver disease (NAFLD) in postmenopausal women. Naringenin improves metabolic disturbances in vitro and in vivo. In the present study, we tested the effects of naringenin on metabolic disturbances resulting from estrogen deficiency in ovariectomized mice. Ovariectomized C57BL/6 J female mice were fed a control diet (10% calories from fat) for 11 weeks. Mice either continued on the control diet (n = 9) or were switched to the control diet supplemented with 3% naringenin (n = 10) for the next 11 weeks. Energy expenditure was measured by indirect calorimetry and activity was monitored by infrared beam breaks. Intra-abdominal and subcutaneous adiposity was evaluated by magnetic resonance imaging (MRI). Blood biochemical measures of metabolic response included glucose, insulin, adipokines, and lipids. Lipid content in liver and muscle and expression of relevant genes in adipose tissue, liver, and muscle were quantified. Ovariectomized mice fed naringenin exhibited lower fasting glucose and insulin levels compared to controls, with over 50% reduction of intra-abdominal and subcutaneous adiposity. Plasma leptin and leptin mRNA in adipose depots were also decreased in mice fed a naringenin diet. Monocyte chemoattractant protein-1 (MCP1/Ccl2) and interleukin 6 (IL-6/Il6) mRNA expression levels were significantly lower in perigonadal adipose tissue of naringenin-supplemented mice. We also observed that mice fed a naringenin diet had less hepatic lipid accumulation with corresponding alterations of hepatic gene expression associated with de novo lipogenesis, fatty acid oxidation, and gluconeogenesis. Dietary naringenin attenuates many of the metabolic disturbances associated with ovariectomy in female mice.