27 June 2019 in IEEE Transactions on Signal Processing
In this paper, we study the graph Laplacian estimation problem under a given connectivity topology. We aim at enriching the unified graph learning framework proposed by Egilmez et al. and improve the optimality performance of the Combinatorial Graph Laplacian (CGL) case. We apply the well-known Alternating Direction Method of Multipliers (ADMM) and Majorization-Minimization (MM) algorithmic frameworks and propose two algorithms, namely, GLE-ADMM and GLE-MM, for graph Laplacian estimation. Both algorithms can achieve an optimality gap as low as $10^{-4}$ , around three orders of magnitude more accurate than the benchmark. In addition, we find that GLE-ADMM is more computationally efficient in a dense topology (e.g., an almost complete graph) while GLE-MM is more suitable for sparse graphs (e.g., trees). Furthermore, we consider exploiting the leading eigenvectors of the sample covariance matrix as a nominal eigensubspace and propose a third algorithm named GLENE, which is also based on ADMM. Numerical experiments show that the inclusion of a nominal eigensubspace significantly improves the estimation of the graph Laplacian when the sample size is smaller than or comparable to the problem dimension.
21 February 2019 in IEEE Transactions on Signal Processing
12 February 2021 in IEEE Transactions on Signal Processing
This paper proposes a framework for optimizing cost functions of orthonormal basis learning problems, such as principal component analysis (PCA), subspace recovery, orthogonal dictionary learning, etc. The optimization algorithm is derived using the majorization-minimization framework in conjunction with orthogonal projection reformulations to deal with the orthonormality constraint in a systematic manner. In this scope, we derive surrogate functions for various standard objectives that can then be used as building blocks, with examples for robust learning costs and sparsity enforcing penalties. To illustrate this point, we propose a new set of algorithms for sparse PCA driven by this methodology, whose objective function is composed of an $M$ -estimation type subspace fitting term plus a regularizer that promotes sparsity. Simulations and experiments on real data illustrate the interest of the proposed approach, both in terms of performance and computational complexity.
23 October 2020 in IEEE Transactions on Signal Processing
The vector autoregressive (VAR) models provide a significant tool for multivariate time series analysis. Owing to the mathematical simplicity, existing works on VAR modeling are rigidly inclined towards the multivariate Gaussian distribution. However, heavy-tailed distributions are suggested more reasonable for capturing the real-world phenomena, like the presence of outliers and a stronger possibility of extreme values. Furthermore, missing values in observed data is a real problem, which typically happens during the data observation or recording process. Although there exist numerous works on VAR modeling with heavy-tailed distributions, they assume the availability of complete data and are not applicable in the presence of missing data. In this paper, we propose an algorithmic framework to estimate the parameters of a VAR model with heavy-tailed Student’ s $t$ distributed innovations from incomplete data based on the stochastic approximation expectation maximization (SAEM) algorithm coupled with a Markov Chain Monte Carlo (MCMC) procedure. We propose two fast and computationally cheap Gibbs sampling schemes, both based on MCMC procedure. The algorithms developed are effective in capturing the heavy-tailed phenomenon and being robust against outliers and missing data. In addition, owing to their low computational complexity, the algorithms are amenable for high-dimensional and big data applications. Extensive experiments with both synthetic data and real financial data corroborate our claims.
1 May 2023 in Journal of the Franklin Institute
24 February 2025 in Computers in Biology and Medicine