The use of large datasets for targeted therapeutic interventions requires new ways to characterize the heterogeneity observed across subgroups of a specific population. In particular, models for partially exchangeable data are needed for inference on nested datasets, where the observations are assumed to be organized in different units and some sharing of information is required to learn distinctive features of the units. In this manuscript, we propose a nested common atoms model (CAM) that is particularly suited for the analysis of nested datasets where the distributions of the units are expected to differ only over a small fraction of the observations sampled from each unit. The proposed CAM allows a two-layered clustering at the distributional and observational level and is amenable to scalable posterior inference through the use of a computationally efficient nested slice sampler algorithm. We further discuss how to extend the proposed modeling framework to handle discrete measurements, and we conduct posterior inference on a real microbiome dataset from a diet swap study to investigate how the alterations in intestinal microbiota composition are associated with different eating habits. We further investigate the performance of our model in capturing true distributional structures in the population by means of a simulation study.
Summary: While the cost of sequencing genomes has decreased dramatically in recent years, this expense often remains nontrivial. Under a fixed budget, scientists face a natural trade-off between quantity and quality: spending resources to sequence a greater number of genomes or spending resources to sequence genomes with increased accuracy. Our goal is to find the optimal allocation of resources between quantity and quality. Optimizing resource allocation promises to reveal as many new variations in the genome as possible. We introduce a Bayesian nonparametric methodology to predict the number of new variants in a follow-up study based on a pilot study. When experimental conditions are kept constant between the pilot and follow-up, we find that our prediction is competitive with the best existing methods. Unlike current methods, though, our new method allows practitioners to change experimental conditions between the pilot and the follow-up. We demonstrate how this distinction allows our method to be used for more realistic predictions and for optimal allocation of a fixed budget between quality and quantity. We validate our method on cancer and human genomics data.
20 May 2025 in Journal of the Royal Statistical Society Series B: Statistical Methodology
The study of almost surely discrete random probability measures is an active line of research in Bayesian non-parametrics. The idea of assuming interaction across the atoms of the random probability measure has recently spurred significant interest in the context of Bayesian mixture models. This allows the definition of priors that encourage well-separated and interpretable clusters. In this work, we provide a unified framework for the construction and the Bayesian analysis of random probability measures with interacting atoms, encompassing both repulsive and attractive behaviours. Specifically, we derive closed-form expressions for the posterior distribution, the marginal and predictive distributions, previously unavailable except for the case of measures with i.i.d. atoms. We show how these quantities are fundamental for both prior elicitation and developing new posterior simulation algorithms for hierarchical mixture models. Our results are obtained without any assumption on the finite point process governing the atoms of the random measure. Their proofs rely on analytical tools borrowed from Palm calculus theory, which might be of independent interest. We specialize our treatment to the classes of Poisson, Gibbs, and determinantal point processes, as well as in the case of shot-noise Cox processes. Finally, we illustrate different modelling strategies on simulated and real datasets.
1 October 2025 in Journal of the Royal Statistical Society Series B: Statistical Methodology
Feature allocation models are an extension of Bayesian nonparametric clustering models, where individuals can share multiple features. We study a broad class of models whose probability distribution has a product form, which includes the popular Indian buffet process. This class plays a prominent role among existing priors, and it shares structural characteristics with Gibbs-type priors in the species sampling framework. We develop a general theory for the entire class, obtaining closed form expressions for the predictive structure and the posterior law of the underlying stochastic process. Additionally, we describe the distribution for the number of features and the number of hitherto unseen features in a future sample, leading to the α-diversity for feature models. We also examine notable novel examples, such as mixtures of Indian buffet processes and beta Bernoulli models, where the latter entails a finite random number of features. This methodology finds significant applications in ecology, allowing the estimation of species richness for incidence data, as we demonstrate by analyzing plant diversity in Danish forests and trees in Barro Colorado Island.
Many real phenomena may be modelled as random closed sets in ℝd, of different Hausdorff dimensions. The problem of the estimation of pointwise mean densities of absolutely continuous, and spatially inhomogeneous, random sets with Hausdorff dimension n < d, has been the subject of extended mathematical analysis by the authors. In particular, two different kinds of estimators have been recently proposed, the first one is based on the notion of Minkowski content, the second one is a kernel-type estimator generalizing the well-known kernel density estimator for random variables. The specific aim of the present paper is to validate the theoretical results on statistical properties of those estimators by numerical experiments. We provide a set of simulations which illustrates their valuable properties via typical examples of lower dimensional random sets.
Post randomization methods are among the most popular disclosure limitation techniques for both categorical and continuous data. In the categorical case, given a stochastic matrixMand a specified variable, an individual belonging to categoryiis changed to categoryjwith probability $$M_{i,j}$$ . Every approach to choose the randomization matrixMhas to balance between two desiderata: (1) preserving as much statistical information from the raw data as possible; (2) guaranteeing the privacy of individuals in the dataset. This trade-off has generally been shown to be very challenging to solve. In this work, we use recent tools from the computer science literature and propose to chooseMas the solution of a constrained maximization problems. Specifically,Mis chosen as the solution of a constrained maximization problem, where we maximize the mutual information between raw and transformed data, given the constraint that the transformation satisfies the notion of differential privacy. For the general categorical model, it is shown how this maximization problem reduces to a convex linear programming and can be therefore solved with known optimization algorithms.