29 August 2022 in Statistical Methods in Medical Research
The Brier score has been a popular measure of prediction accuracy for binary outcomes. However, it is not straightforward to interpret the Brier score for a prediction model since its value depends on the outcome prevalence. We decompose the Brier score into two components, the mean squares between the estimated and true underlying binary probabilities, and the variance of the binary outcome that is not reflective of the model performance. We then propose to modify the Brier score by removing the variance of the binary outcome, estimated via a general sliding window approach. We show that the new proposed measure is more sensitive for comparing different models through simulation. A standardized performance improvement measure is also proposed based on the new criterion to quantify the improvement of prediction performance. We apply the new measures to the data from the Breast Cancer Surveillance Consortium and compare the performance of predicting breast cancer risk using the models with and without its most important predictor.
15 July 2019 in Journal of Cerebral Blood Flow & Metabolism
Obstructive sleep apnea (OSA) is characterized by intermittent obstruction of the airways during sleep. Cerebrovascular reactivity (CVR) is an index of cerebral vessels' ability to respond to a vasoactive stimulus, such as increased CO2. We hypothesized that OSA alters CVR, expressed as a breath-hold index (BHI) defined as the rate of change in CBF or BOLD signal during a controlled breath-hold stimulus mimicking spontaneous apneas by being both hypercapnic and hypoxic. In 37 OSA and 23 matched non sleep apnea (NSA) subjects, we obtained high temporal resolution CBF and BOLD MRI data before, during, and between five consecutive BH stimuli of 24 s, each averaged to yield a single BHI value. Greater BHI was observed in OSA relative to NSA as derived from whole-brain CBF (78.6 ± 29.6 vs. 60.0 ± 20.0 mL/min2/100 g, P = 0.010) as well as from flow velocity in the superior sagittal sinus (0.48 ± 0.18 vs. 0.36 ± 0.10 cm/s2, P = 0.014). Similarly, BOLD-based BHI was greater in OSA in whole brain (0.19 ± 0.08 vs. 0.15 ± 0.03%/s, P = 0.009), gray matter (0.22 ± 0.09 vs. 0.17 ± 0.03%/s, P = 0.011), and white matter (0.14 ± 0.06 vs. 0.10 ± 0.02%/s, P = 0.010). The greater CVR is not currently understood but may represent a compensatory mechanism of the brain to maintain oxygen supply during intermittent apneas.
13 May 2022 in Journal of the American Statistical Association
By clustering patients with the urologic chronic pelvic pain syndromes (UCPPS) into homogeneous subgroups and associating these subgroups with baseline covariates and other clinical outcomes, we provide opportunities to investigate different potential elements of pathogenesis, which may also guide us in selection of appropriate therapeutic targets. Motivated by the longitudinal urologic symptom data with extensive subject heterogeneity and differential variability of trajectories, we propose a functional clustering procedure where each subgroup is modeled by a functional mixed effects model, and the posterior probability is used to iteratively classify each subject into different subgroups. The classification takes into account both group-average trajectories and between-subject variabilities. We develop an equivalent state-space model for efficient computation. We also propose a cross-validation based Kullback–Leibler information criterion to choose the optimal number of subgroups. The performance of the proposed method is assessed through a simulation study. We apply our methods to longitudinal bi-weekly measures of a primary urological urinary symptoms score from a UCPPS longitudinal cohort study, and identify four subgroups ranging from moderate decline, mild decline, stable and mild increasing. The resulting clusters are also associated with the one-year changes in several clinically important outcomes, and are also related to several clinically relevant baseline predictors, such as sleep disturbance score, physical quality of life and painful urgency. Supplementary materials for this article are available online.
26 October 2021 in Biometrics
Prediction modeling for clinical decision making is of great importance and needed to be updated frequently with the changes of patient population and clinical practice. Existing methods are either done in an ad hoc fashion, such as model recalibration or focus on studying the relationship between predictors and outcome and less so for the purpose of prediction. In this article, we propose a dynamic logistic state space model to continuously update the parameters whenever new information becomes available. The proposed model allows for both time‐varying and time‐invariant coefficients. The varying coefficients are modeled using smoothing splines to account for their smooth trends over time. The smoothing parameters are objectively chosen by maximum likelihood. The model is updated using batch data accumulated at prespecified time intervals, which allows for better approximation of the underlying binomial density function. In the simulation, we show that the new model has significantly higher prediction accuracy compared to existing methods. We apply the method to predict 1 year survival after lung transplantation using the United Network for Organ Sharing data.
1 September 2024 in The Annals of Applied Statistics
Some patients with COVID-19 show changes in signs and symptoms, such as temperature and oxygen saturation days before being positively tested for SARS-CoV-2, while others remain asymptomatic. It is important to identify these subgroups and to understand what biological and clinical predictors are related to these subgroups. This information will provide insights into how the immune system may respond differently to infection and can further be used to identify infected individuals. We propose a flexible nonparametric mixed-effects mixture model that identifies risk factors and classifies patients with biological changes. We model the latent probability of biological changes using a logistic regression model and trajectories in the latent groups using smoothing splines. We developed an EM algorithm to maximize the penalized likelihood for estimating all parameters and mean functions. We evaluate our methods by simulations and apply the proposed model to investigate changes in temperature in a cohort of COVID-19-infected hemodialysis patients.