Po-Ling Loh

Dr. Po-Ling Loh

orcid

Research Focus & Metrics

Academic Career Length

12Years (2015-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
8 October 2024 in Journal of the American Statistical Association
Verified
Journal Article
Closed access
APAnkit Pensia
Ankit Pensia
VJVarun Jog
Varun Jog
PLPo-Ling Loh
Po-Ling Loh

7 March 2025 in Annual Review of Statistics and Its Application
Verified
Journal Article
Open access
PLPo-Ling Loh
Po-Ling Loh
Robust statistics is a fairly mature field that dates back to the early 1960s, with many foundational concepts having been developed in the ensuing decades. However, the field has drawn a new surge of attention in the past decade, largely due to a desire to recast robust statistical principles in the context of high-dimensional statistics. In this article, we begin by reviewing some of the central ideas in classical robust statistics. We then discuss the need for new theory in high dimensions, using recent work in high-dimensional M -estimation as an illustrative example. Next, we highlight a variety of interesting recent topics that have drawn a flurry of research activity from both statisticians and theoretical computer scientists, demonstrating the need for further research in robust estimation that embraces new estimation and contamination settings, as well as a greater emphasis on computational tractability in high dimensions.

6 May 2020 in IEEE Journal on Selected Areas in Information Theory
Verified
Journal Article
Open access
APAnkit Pensia
Ankit Pensia
VJVarun Jog
Varun Jog
PLPo-Ling Loh
Po-Ling Loh
We propose a novel strategy for extracting features in supervised learning that can be used to construct a classifier which is more robust to small perturbations in the input space. Our method builds upon the idea of the information bottleneck, by introducing an additional penalty term that encourages the Fisher information of the extracted features to be small when parametrized by the inputs. We present two formulations where the relevance of the features to output labels is measured using either mutual information or MMSE. By tuning the regularization parameter, we can explicitly trade off the opposing desiderata of robustness and accuracy when constructing a classifier. We derive optimal solutions to both robust information bottleneck formulations when the inputs and outputs are jointly Gaussian, proving that the optimally robust features are also jointly Gaussian in this setting. We also propose methods for optimizing variational bounds on the robust information bottleneck objectives in general settings using stochastic gradient descent, which may be implemented efficiently in neural networks. Our experimental results for synthetic and real data sets show that the proposed feature extraction methods indeed produce classifiers with increased robustness to perturbations.

16 November 2023 in IEEE Transactions on Information Theory
Verified
Journal Article
Closed access
APAnkit Pensia
Ankit Pensia
VJVarun Jog
Varun Jog
PLPo-Ling Loh
Po-Ling Loh
We study hypothesis testing under communication constraints, where each sample is quantized before being revealed to a statistician. Without communication constraints, it is well known that the sample complexity of simple binary hypothesis testing is characterized by the Hellinger distance between the distributions. We show that the sample complexity of simple binary hypothesis testing under communication constraints is at most a logarithmic factor larger than in the unconstrained setting and this bound is tight. We develop a polynomial-time algorithm that achieves the aforementioned sample complexity. Our framework extends to robust hypothesis testing, where the distributions are corrupted in total variation distance. Our proofs rely on a new reverse data processing inequality and a reverse Markov inequality, which may be of independent interest. For simple M -ary hypothesis testing, the sample complexity in the absence of communication constraints has a logarithmic dependence on M . We show that communication constraints can cause an exponential blow-up, leading to Ω( M ) sample complexity even for adaptive algorithms.

15 November 2017 in Entropy
Verified
Journal Article
Open access
PDF file
PLPo-Ling Loh
Po-Ling Loh
In recent years, tools from information theory have played an increasingly prevalent role in statistical machine learning. In addition to developing efficient, computationally feasible algorithms for analyzing complex datasets, it is of theoretical importance to determine whether such algorithms are “optimal” in the sense that no other algorithm can lead to smaller statistical error. This paper provides a survey of various techniques used to derive information-theoretic lower bounds for estimation and learning. We focus on the settings of parameter and function estimation, community recovery, and online learning for multi-armed bandits. A common theme is that lower bounds are established by relating the statistical learning problem to a channel decoding problem, for which lower bounds may be derived involving information-theoretic quantities such as the mutual information, total variation distance, and Kullback–Leibler divergence. We close by discussing the use of information-theoretic quantities to measure independence in machine learning applications ranging from causality to medical imaging, and mention techniques for estimating these quantities efficiently in a data-driven manner.

13 October 2020 in IEEE Transactions on Information Theory
Verified
Journal Article
Closed access
VJVarun Jog
Varun Jog
PLPo-Ling Loh
Po-Ling Loh
We investigate a simple model for social learning with two agents: a teacher and a student. The teacher’s goal is to teach the student the state of the world; however, the teacher himself is not certain about the state of the world and needs to simultaneously learn this parameter and teach it to the student. We model the teacher’s and student’s uncertainties via noisy transmission channels, and employ two simple decoding strategies for the student. We focus on two teaching strategies: a “low-effort” strategy of simply forwarding information, and a “high-effort” strategy of communicating the teacher’s current best estimate of the world at each time instant, based on his own cumulative learning. Using tools from large deviation theory, we calculate the exact learning rates for these strategies and demonstrate regimes where the low-effort strategy outperforms the high-effort strategy. Finally, we present a conjecture concerning the optimal learning rate for the student over all joint strategies between the student and the teacher.