This paper investigates the influence of directed networks on decentralized stochastic nonconvex optimization associated with column-stochastic mixing matrices. Surprisingly, we find that the canonical spectral gap, a widely used metric in undirected networks, is insufficient to characterize the impact of directed topology on decentralized algorithms. To overcome this limitation, we introduce a novel metric termed equilibrium skewness. This metric, together with the spectral gap, accurately and comprehensively captures the influence of column-stochastic mixing matrices on decentralized stochastic algorithms. With these two metrics, we clarify, for the first time, how the directed network topology influences the performance of prevalent algorithms such as Push-Sum and Push-DIGing. Furthermore, we establish the first lower bound of the convergence rate for decentralized stochastic nonconvex algorithms over directed networks. Since existing algorithms cannot match our lower bound, we further propose the MG-Push-DIGing algorithm, which integrates Push-DIGing with a multiround gossip technique. MG-Push-DIGing attains our lower bound up to logarithmic factors, demonstrating its near-optimal performance and the tightness of the lower bound. Numerical experiments verify our theoretical results.
15 December 2025 in IEEE Transactions on Signal Processing
Most decentralized optimization algorithms are handcrafted. While endowed with strong theoretical guarantees, these algorithms generally target a broad class of problems, thereby not being adaptive or customized to specific problem features. This paper studies data-driven decentralized algorithms trained to exploit problem features to boost convergence. Existing learning-to-optimize methods typically suffer from poor generalization or prohibitively vast search spaces. In addition, they face more challenges in decentralized settings where nodes must reach consensus through neighborhood communications without global information. To resolve these challenges, this paper first derives necessary conditions for decentralized algorithmic rules to achieve both optimality and consensus. Then, we propose a novel Convergence-Motivated Learning-to-optimize framework for Decentralized optimization (MiLoDo) based on these conditions. Empirical results demonstrate that MiLoDo-trained algorithms outperform handcrafted algorithms and exhibit strong generalizations. Algorithms learned via MiLoDo in 100 iterations perform robustly when running 100,000 iterations during inferences. Moreover, MiLoDo-trained algorithms on synthetic datasets perform well on problems involving real data, higher dimensions, and different loss functions. Codes are available at: https://github.com/pkumelon/MiLoDo.
The prediction accuracy of machine learning methods is steadily increasing, but the calibration of their uncertainty predictions poses a significant challenge. Numerous works focus on obtaining well-calibrated predictive models, but less is known about reliably assessing model calibration. This limits our ability to know when algorithms for improving calibration have a real effect, and when their improvements are merely artifacts due to random noise in finite datasets. In this work, we consider detecting mis-calibration of predictive models using a finite validation dataset as a hypothesis testing problem. The null hypothesis is that the predictive model is calibrated, while the alternative hypothesis is that the deviation from calibration is sufficiently large. We find that detecting mis-calibration is only possible when the conditional probabilities of the classes are sufficiently smooth functions of the predictions. When the conditional class probabilities are H\"older continuous, we propose T-Cal, a minimax optimal test for calibration based on a debiased plug-in estimator of the $\ell_2$-Expected Calibration Error (ECE). We further propose Adaptive T-Cal, a version that is adaptive to unknown smoothness. We verify our theoretical findings with a broad range of experiments, including with several popular deep neural net architectures and several standard post-hoc calibration methods. T-Cal is a practical general-purpose tool, which -- combined with classical tests for discrete-valued predictors -- can be used to test the calibration of virtually any probabilistic classification method.