The Brier score is a widely used metric in epidemiological and clinical research for evaluating the accuracy of probabilistic predictions for binary outcomes, such as disease occurrence, treatment response, and screening performance. Despite its popularity, the Brier score is frequently misunderstood, leading to flawed interpretation of prediction models and potentially misguided public health and clinical decisions. This study aims to didactically clarify common misconceptions about realised Brier scores and to provide practical, statistically rigorous guidance for its correct interpretation in epidemiologic and public health prediction models. We analytically examined its statistical properties and conducted simulation studies across diverse scenarios, varying the distribution of true outcome probabilities, prediction accuracy, sample size, and event prevalence. Five prevalent misconceptions were identified, including the mistaken belief that a Brier score of zero indicates a perfect model. Analytic arguments and simulations demonstrated that even perfectly specified models yield non-zero Brier scores under realistic conditions. The Brier score was shown to reflect not only prediction accuracy but also the underlying distribution of true risks and random variation in outcomes. Comparisons across different populations or disease settings can therefore be misleading, and the Brier score does not directly measure calibration. We recommend restricting comparisons to the same population and complementing the Brier score with calibration metrics and measures of clinical or public health utility. Adopting these practices will improve the validity and interpretability of risk prediction in epidemiologic research and enhance decision-making in population health.
1 January 2019 in SIAM Journal on Applied Mathematics
We investigate a broad family of stochastically modeled reaction networks by looking at their stationary distributions. Most known results on stationary distributions assume weak reversibility and ...
30 November 2023 in American Journal of Transplantation
Surgical site infections (SSI) are common healthcare-associated infections. SSIs after kidney transplantation (K-Tx) can endanger patient and allograft survival. Multicenter studies on this early post-transplant complication are scarce. We analyzed consecutive adult K-Tx recipients enrolled in the Swiss Transplant Cohort Study (STCS) that received a K-tx between May 2008 and September 2020. All data were prospectively collected with the exception of the categorization of SSI that was performed retrospectively according to the Centers for Disease Control and Prevention criteria. A total of 58 out of 3059 (1.9%) K-Tx recipients were affected by SSIs. Deep incisional (15, 25.9%) and organ/space infections (34, 58.6%) predominated. In the majority of SSIs (52, 89.6%) bacteria were detected, most frequently Escherichia coli (15, 28.9%), Enterococcus spp. (14, 26.9%), and coagulase-negative staphylococci (13, 25.0%). A BMI ≥25kg/m2 (multivariable OR 2.16, 95% CI 1.07-4.34, P=0.023) and delayed graft function (multivariable OR 2.88, 95% CI 1.56-5.34, P=0.001) were independent risk factors for SSI. In Cox proportional hazard models, SSI was independently associated with graft loss (multivariable HR 3.75, 95% CI 1.35-10.38, P=0.011). In conclusion, SSI was a rare complication after K-Tx. BMI ≥25kg/m2 and delayed graft function were independent risk factors. SSI were independently associated with graft loss.
We examine reaction networks (CRNs) through their associated continuous-time Markov processes. Studying the dynamics of such networks is in general hard, both analytically and by simulation. In particular, stationary distributions of stochastic reaction networks are only known in some cases. We analyze class properties of the underlying continuous-time Markov chain of CRNs under the operation of join and examine conditions such that the form of the stationary distributions of a CRN is derived from the parts of the decomposed CRNs. The conditions can be easily checked in examples and allow recursive application. The theory developed enables sequential decomposition of the Markov processes and calculations of stationary distributions. Since the class of processes expressible through such networks is big and only few assumptions are made, the principle also applies to other stochastic models. We give examples of interest from CRN theory to highlight the decomposition.
Probabilistic clinical prediction models play a critical role by informing healthcare professionals both in diagnosis and prognosis. To assess the qualities of a probabilistic prediction model, performance evaluation measures are used. We explain the kinds of measures that are usually considered and focus on the interpretation for some typical measures in the case of a binary outcome.
24 May 2022 in European Journal of Applied Mathematics
It is standard in chemistry to represent a sequence of reactions by a single overall reaction, often called a complex reaction in contrast to an elementary reaction. Photosynthesis $6 \text{CO}_2+6 \text{H}_2\text{O} \longrightarrow \text{C}_6\text{H}_{12}\text{O}_6 + 6 \text{O}_2$ is an example of such complex reaction. We introduce a mathematical operation that corresponds to summing two chemical reactions. Specifically, we define an associative and non-communicative operation on the product space ${\mathbb{N}}_0^n\times {\mathbb{N}}_0^n$ (representing the reactant and the product of a chemical reaction, respectively). The operation models the overall effect of two reactions happening in succession, one after the other. We study the algebraic properties of the operation and apply the results to stochastic reaction networks (RNs), in particular to reachability of states, and to reduction of RNs.