The use of complementary information, namely depth or thermal information, has shown its benefits to salient object detection (SOD) during recent years. However, the RGB-D or RGB-T SOD problems are currently only solved independently, and most of them directly extract and fuse raw features from backbones. Such methods can be easily restricted by low-quality modality data and redundant cross-modal features. In this work, a unified end-to-end framework is designed to simultaneously analyze RGB-D and RGB-T SOD tasks. Specifically, to effectively tackle multi-modal features, we propose a novel multi-stage and multi-scale fusion network (MMNet), which consists of a cross-modal multi-stage fusion module (CMFM) and a bi-directional multi-scale decoder (BMD). Similar to the visual color stage doctrine in the human visual system (HVS), the proposed CMFM aims to explore important feature representations in feature response stage, and integrate them into cross-modal features in adversarial combination stage. Moreover, the proposed BMD learns the combination of multi-level cross-modal fused features to capture both local and global information of salient objects, and can further boost the multi-modal SOD performance. The proposed unified cross-modality feature analysis framework based on two-stage and multi-scale information fusion can be used for diverse multi-modal SOD tasks. Comprehensive experiments ( $\sim 92\text{K}$ image-pairs) demonstrate that the proposed method consistently outperforms the other 21 state-of-the-art methods on nine benchmark datasets. This validates that our proposed method can work well on diverse multi-modal SOD tasks with good generalization and robustness, and provides a good multi-modal SOD benchmark.
29 January 2024 in IEEE Transactions on Image Processing
To manipulate large-scale data, anchor-based multi-view clustering methods have grown in popularity owing to their linear complexity in terms of the number of samples. However, these existing approaches pay less attention to two aspects. 1) They target at learning a shared affinity matrix by using the local information from every single view, yet ignoring the global information from all views, which may weaken the ability to capture complementary information. 2) They do not consider the removal of feature redundancy, which may affect the ability to depict the real sample relationships. To this end, we propose a novel fast multi-view clustering method via pick-and-place transform learning named PPTL, which could capture insightful global features to characterize the sample relationships quickly. Specifically, PPTL first concatenates all the views along the feature direction to produce a global matrix. Considering the redundancy of the global matrix, we design a pick-and-place transform with ℓ 2,p -norm regularization to abandon the poor features and consequently construct a compact global representation matrix. Thus, by conducting anchor-based subspace clustering on the compact global representation matrix, PPTL can learn a consensus skinny affinity matrix with a discriminative clustering structure. Numerous experiments performed on small-scale to large-scale datasets demonstrate that our method is not only faster but also achieves superior clustering performance over state-of-the-art methods across a majority of the datasets.
Incomplete multi-view clustering is gaining increased attention owing to its great success in mining underlying information from the missing views. However, the existing approaches still encounter two issues: 1) They generally do not give sufficient consideration to the robustness of incomplete multi-view data with noise; 2) They only exploit the low-rank structures in the intra-view graphs, while the low-rank priors embedded in inter-view graphs are ignored. To this end, we propose a Robust Tensor Recovery for Incomplete Multi-view Clustering (RIMC) method, which transforms the view-missing problem into the tensor graph recovery problem by manipulating the comprehensive low-rank priors. Specifically, RIMC first employs a marginalized denoising operation to construct robust graphs and further builds a tensor graph by stacking these robust graphs. Then, we develop a novel tensor completion to recover the tensor graph by performing comprehensive low-rank priors: low-rank structures in the inter-view graphs (i.e., horizontal and lateral slices); low-rank structures in the intra-view graphs (i.e., frontal slices). Meanwhile, we integrate the tensor completion and spectral clustering to learn a unified indicator matrix. Extensive experiments show the promising performance of our method.
17 January 2025 in IEEE Transactions on Circuits and Systems for Video Technology
Under large-scale data with missing views, fast incomplete multi-view clustering (IMVC) with anchor learning is of critical importance due to its linear complexity O(n). However, existing anchor-based methods only explore the column orthogonality of anchor points, where their arbitrary column orthogonal basis vectors have weak constraint relationships with real samples and significant deviations from more representative anchors, thereby impeding the precise representation of sample similarities. To solve this issue, we propose a Reliable Entropy-induced anchor learning for incomplete Multi-view subspace Clustering (REMC), which performs an entropy approximation term to learn more representative anchors, and we prove that the information entropy minimization can be relaxed into the ℓ2, 1-norm paradigm. Specifically, the proposed REMC first integrates anchor learning and subspace clustering to produce multiple view-specific bipartite graphs and capture the high-order correlations by imposing these bipartite graphs with the tensor nuclear norm. Then, we fuse all the view-specific bipartite graphs to build a consensus bipartite graph with entropy approximation regularization, and hence the proposed REMC can produce a more discriminative similarity graph, preserving each non-zero element in its column close to 1, while the other elements are approaching 0. Besides, an efficient algorithm is designed to solve the proposed REMC. Numerous results show the superior performance of our method on both the complete and incomplete data.
7 August 2023 in IEEE Transactions on Circuits and Systems for Video Technology
Incomplete multi-view clustering (IMVC), excavating diversity and consistency from multiple incomplete views, has aroused widespread research enthusiasm. Nevertheless, most existing methods still encounter the following issues: 1) they generally concentrate on pair-wise instance correlation, which consumes at least a quadratic complexity and precludes them from applying at large scales; 2) they only concentrate on pair-wise instance relevance, whereas ignoring the discriminative correlation hidden across views. To overcome these drawbacks, we propose the Self-Completed Bipartite Graph Learning (SCBGL) method for fast IMVC, which adaptively learns a self-completed consensus bipartite graph with the guidance of global information. Specifically, SCBGL learns the consensus anchor matrix shared among diverse views and further constructs a consensus intra-view bipartite graph with missing instances to explore the diversity and complementarity underlying different views. Meanwhile, we concatenate all the multiple features with projection learning to learn global anchors that would be employed to construct an inter-view bipartite graph. Furthermore, SCBGL dexterously utilizes the abundant inter-view information to tutor the self-completion of the consensus intra-view bipartite graph. By devising an alternatively iterative strategy, we present an efficient algorithm, which enjoys a linear time complexity, to solve the proposed SCBGL model. Numerous experiments conducted on large-scale datasets substantiate the superior performance of the SCBGL beyond the state-of-the-arts.
7 May 2021 in IEEE Transactions on Circuits and Systems for Video Technology
As well known, low rank representation method (LRR) has obtained promising performance for subspace clustering, and many LRR variants have been developed, which mainly solve the three problems existing in LRR: 1) Problem of mean calculation; 2) Problem of deviating from the real low rank solution; 3) Problem of high computation cost on the large-scale data. In this paper, we first propose a universal LRR method referring to the first two problems. More specifically, we introduce the ability of removing the optimal mean automatically into LRR and extend it to a Schatten $p$ -norm minimization problem. Then, referring to the third problem, we reformulate the universal LRR version as an equivalent fast optimization version by removing the null space of data. More importantly, the effective theory proof is proposed to guarantee that the fast optimization method can dramatically improve the algorithmic computation efficiency on the large-scale data but without any loss of information. Finally, experimental results demonstrate the effectiveness and efficiency of the proposed method.