15 July 2022 in IEEE Transactions on Neural Networks and Learning Systems
In the unsupervised feature selection method based on spectral analysis, constructing a similarity matrix is a very important part. In existing methods, the linear low-dimensional projection used in the process of constructing the similarity matrix is too hard, it is very challenging to construct a reliable similarity matrix. To this end, we propose a method to construct a flexible optimal graph. Based on this, we propose an unsupervised feature selection method named unsupervised feature selection with flexible optimal graph and $\ell_{2,1}$ -norm regularization (FOG-R). Unlike other methods that use linear projection to approximate the low-dimensional manifold of the original data when constructing a similarity matrix, FOG-R can learn a flexible optimal graph, and by combining flexible optimal graph learning and feature selection into a unified framework to get an adaptive similarity matrix. In addition, an iterative algorithm with a strict convergence proof is proposed to solve FOG-R. $\ell_{2,1}$ -norm regularization will introduce an additional regularization parameter, which will cause parameter-tuning trouble. Therefore, we propose another unsupervised feature selection method, that is, unsupervised feature selection with a flexible optimal graph and $\ell_{2,0}$ -norm constraint (FOG-C), which can avoid tuning additional parameters and obtain a more sparse projection matrix. Most critically, we propose an effective iterative algorithm that can solve FOG-C globally with strict convergence proof. Comparative experiments conducted on 12 public datasets show that FOG-R and FOG-C perform better than the other nine state-of-the-art unsupervised feature selection algorithms.
27 January 2022 in IEEE Transactions on Knowledge and Data Engineering
Since obtaining data labels is a time-consuming and laborious task, unsupervised feature selection has become a popular feature selection technique. However, the current unsupervised feature selection methods are facing three challenges: (1) they rely on a fixed similarity matrix derived from the original data, which will affect their performance; (2) due to the limitation of sparsity, they can only obtain sub-optimal solutions; (3) they have high computational complexity and cannot handle large-scale data. To solve this dilemma, we propose a fast unsupervised feature selection algorithm with bipartite graph and 2;0-norm constraint (BGCFS). We use the original data and the selected anchors to construct an adaptive bipartite graph in the subspace, and apply the l2,0-norm constraint to the projection matrix for feature selection. In this way, we can update the adaptive bipartite graph and the projection matrix simultaneously, and we can get the feature subset directly, without sorting the features. In addition, we propose an iterative algorithm that can solve the proposed problem globally to obtain a closed-form solution, and we provide a strict proof of convergence for it. Experiments on eight real data sets with different scales show that our method can select more valuable feature subsets more quickly
1 April 2019 in International Journal of Heat and Mass Transfer
8 October 2021 in IEEE Signal Processing Letters
Graph-based dimensionality reduction methods are widely used in classification and clustering tasks due to their superior performance. The key to the performance of these methods is how to construct the graph. Although some existing methods can obtain an adaptive graph and a projection matrix simultaneously by combining manifold learning and graph construction in a unified framework, the linear constraint used in manifold learning is too hard. To solve this problem, we propose a novel method named adaptive flexible optimal graph (AFOG) for unsupervised dimensionality reduction. AFOG can obtain an adaptive flexible optimal graph by relaxing the linear constraint in the process of low-dimensional manifold mapping. At the same time, by using the principle of maximum separability, it can also obtain an effective projection matrix, which can solve out-of-sample problems. Experiments on six public benchmark data sets indicate that AFOG outperforms several other state-of-the-art methods.
28 October 2022 in IEEE Transactions on Neural Networks and Learning Systems
Studying the relationship between linear discriminant analysis (LDA) and least squares regression (LSR) is of great theoretical and practical significance. It is well-known that the two-class LDA is equivalent to an LSR problem, and directly casting multiclass LDA as an LSR problem, however, becomes more challenging. Recent study reveals that the equivalence between multiclass LDA and LSR can be established based on a special class indicator matrix, but under a mild condition which may not hold under the scenarios with low-dimensional or oversampled data. In this article, we show that the equivalence between multiclass LDA and LSR can be established based on arbitrary linearly independent class indicator vectors and without any condition. In addition, we show that LDA is also equivalent to a constrained LSR based on the data-dependent indicator vectors. It can be concluded that under exactly the same mild condition, such two regressions are both equivalent to the null space LDA method. Illuminated by the equivalence of LDA and LSR, we propose a direct LDA classifier to replace the conventional framework of LDA plus extra classifier. Extensive experiments well validate the above theoretic analysis.
13 May 2025 in Neural Networks