25 April 2022 in IEEE Transactions on Information Theory
The problem of completing a large matrix with lots of missing entries has received widespread attention in the last couple of decades. Two popular approaches to the matrix completion problem are based on singular value thresholding and nuclear norm minimization. Most of the past works on this subject assume that there is a single number $p$ such that each entry of the matrix is available independently with probability $p$ and missing otherwise. This assumption may not be realistic for many applications. In this work, we replace it with the assumption that the probability that an entry is available is an unknown function $f$ of the entry itself. For example, if the entry is the rating given to a movie by a viewer, then it seems plausible that high value entries have greater probability of being available than low value entries. We propose two new estimators, based on singular value thresholding and nuclear norm minimization, to recover the matrix under this assumption. The estimators involve no tuning parameters, and are shown to be consistent under a low rank assumption. We also provide a consistent estimator of the unknown function $f$ .
6 April 2021 in Random Structures & Algorithms
3 March 2025 in IEEE Transactions on Information Theory
In this paper, we study a structured signal detection problem in Ferromagnetic Ising models with examples encompassing Ising Models on lattices, and Mean-Field type Ising Models such as dense Erdős-Rényi, and dense random regular graphs. We provide sharp constants of detection in each of these cases and thereby pinpoint an asymptotically precise relationship between the detection problem with the underlying dependence. To obtain this sharp characterization of the detection boundary at the level of sharp multiplicative constants, we derive necessary moderate deviation bounds for partial summands of magnetizations which might be of independent interest. Finally, we demonstrate how our tests can be designed to be adaptive over the strength of dependence present in the respective models.
18 May 2026 in Journal of the American Statistical Association
1 IntroductionIn various fields of study, such as citation networks, protein interactions, health, finance, trade, and social networks, we often come across large amounts of data that describe the ...
4 June 2025 in Biometrika
Summary: identified the issue of ′nonsense correlations’ in time series data, where dependence within each of two random vectors causes overdispersion, i.e., variance inflation, for measures of dependence between the two. Since then much has been written about nonsense correlations, but nearly all of it confined to the time series literature. In this paper we provide the first, to our knowledge, rigorous study of this phenomenon for other forms of (positive) dependence, specifically for Markov random fields on lattices and graphs. We consider binary and continuous random vectors and three different measures of association: correlation, covariance and the ordinary least-squares coefficient from projecting one random vector onto the other. In some settings we find variance inflation consistent with Yule’s nonsense correlation. Surprisingly, we also find variance deflation in some settings, and in others the variance is unchanged under dependence. Perhaps most notably, we find general conditions under which ordinary least-squares inference that ignores dependence is valid despite positive dependence in the regression errors, contradicting the presentation of ordinary least squares in countless textbooks and courses.