With the proliferation of multi-modal data generated by various sensors, unsupervised multi-modal hashing retrieval has been extensively studied due to its advantages in storage, retrieval efficiency, and label independence. However, there are still two obstacles to existing unsupervised methods: (1) As existing methods cannot fully capture the complementary and co-occurrence information of multi-modal data, existing methods suffer from inaccurate similarity measures. (2) Existing methods suffer from unbalanced multi-modal learning and data semantic structure being corrupted in the process of hash codes binarization. To address these obstacles, we devise an effective CLIP-based Adaptive Graph Attention Network (CAGAN) for large-scale unsupervised multi-modal hashing retrieval. Firstly, we use the multi-modal model CLIP to extract fine-grained semantic features, mine similar information from different perspectives of multi-modal data and perform similarity fusion and enhancement. In addition, this paper proposes an adaptive graph attention network to assist the learning of hash codes, which uses an attention mechanism to learn adaptive graph similarity across modalities. It further aggregates the intrinsic neighborhood information of neighboring data nodes through a graph convolutional network to generate more discriminative hash codes. Finally, this paper employs an iterative approximate optimization strategy to mitigate the information loss in the binarization process. Extensive experiments on three benchmark datasets demonstrate that the proposed method significantly outperforms several representative hashing methods in unsupervised multi-modal retrieval tasks.

Multi-view clustering (MVC) exploits the information captured from diverse views to partition data into different groups and attracts much attention recently. Despite significant progress, most MVC methods fuse multi-view information via one-stage fusion while neglecting the merits of multi-stage fusion which causes insufficient in utilizing rich information within data and therefore degrades the clustering performance. To this end, designing a functional framework that can fully exploit multi-view information becomes a key challenge in multi-view clustering research. In this paper, we propose a novel multi-stage fusion method, which elegantly unifies the late and early fusion into one unified framework, to capture sufficient information underlying the multi-view data and to effectively reduce the effect of low-quality views. Specifically, we construct a low dimensional latent representation from multi-view data by learning proper correlation among multi-view data in the early fusion stage. The late fusion establishes a new optimal combinational data partition from base partitions constructed by spectral clustering, which suppresses the influence of low-quality basic partitions. Then we couple the low dimensional latent representation with the learned combinational data partition to share the same cluster structure by $k$-means and maximization alignment. As a result, we collaboratively learn an accurate and robust partition representation for the following clustering task. Besides, the late fusion and early fusion are jointly learned to achieve mutual collaboration for better performance. Finally, an alternating optimization algorithm is designed to solve the resultant optimization problem. Extensive experiments conducted on eight datasets show the superiority of our method in terms of effectiveness and efficiency.

Change detection (CD) is essential for various real-world applications, such as urban management and disaster assessment. Numerous CD methods have been proposed, and considerable results have been achieved recently. However, detecting changes in hard regions, for example, the boundaries of changes, small changes, and irrelevant pseudo-changes caused by background clutter, remains difficult for these methods, since they pay equal attention to all regions in bi-temporal images. This article proposes a novel CD network, termed HRMNet, which provides accurate change maps via hard region mining. Specifically, the HRMNet performs hard region mining from two different perspectives: 1) a cross-layer knowledge review module (CKRM) is introduced to identify hard regions embedded in high-level features and distill temporal change information from low-level features to high-level features, thereby enhancing feature representation capabilities and 2) an online hard region estimation (OHRE) branch is constructed to model pixel-wise hard samples, supervised by the error between predicted change maps and the corresponding ground truth during training. Finally, the hard region-aware features extracted from the OHRE branch and multilevel temporal difference features are aggregated into a unified feature representation to improve the accuracy of CD. Experimental results on three benchmark datasets demonstrate the superior performance of the HRMNet in the CD task. The demo code for our work is publicly available at https://github.com/guanyuezhen/HRMNet

The research on identifying drug-disease associations (DDAs) is widely used in scenarios such as drug development, clinical decision-making, and drug repurposing, holding significant biological and medical significance. Existing methods for drug-disease association prediction have achieved decent performance, they primarily rely on simplistic drug-disease association graphs or similarity graphs. These methods often struggle to capture the high-order correlations of complex multimodal data, limiting their ability to handle the complexity of data associations effectively. In addition, real drug-disease associations are highly sparse, posing a significant challenge to prediction accuracy. To tackle these issues, we propose a general hypergraph neural network framework for drug-disease association prediction based on hierarchical contrastive learning and cross-attention learning. It leverages hypergraph neural networks to learn representations of drugs and diseases carrying high-order correlations and strengthens representation quality using interactive attention learning and hierarchical contrastive learning. Meanwhile, the $\lambda$-weighted loss function is utilized to adapt to the high sparsity property of real drug-disease associations during model training and improve prediction performance. Extensive experiments demonstrate that DD-HGNN$^+$ surpasses other state-of-the-art methods in predicting drug-disease associations and further validation through case studies on Leukemia and Colorectal Neoplasms underscores its reliability.

Light field (LF) image depth estimation is a critical technique for LF-related applications such as 3D reconstruction, target detection, and tracking. The refocusing property of LF images provide rich information for depth estimations; however, it is still challenging in cases of occlusion regions, edge regions, noise interference, etc. The epipolar plane image (EPI) of LF can effectively deal with the depth estimation because of its characteristics of multidirectionality and pixel consistency—in which the LF depth estimations are converted to calculate the EPI slope. This paper proposed an EPI LF depth estimation algorithm based on a directional relationship model and attention mechanism. Unlike the subaperture LF depth estimation method, the proposed method takes EPIs as input images. Specifically, a directional relationship model was used to extract direction features of the horizontal and vertical EPIs, respectively. Then, a multiviewpoint attention mechanism combining channel attention and spatial attention is used to give more weight to the EPI slope information. Subsequently, multiple residual modules are used to eliminate the redundant features that interfere with the EPI slope information—in which a small stride convolution operation is used to avoid losing key EPI slope information. The experimental results revealed that the proposed algorithm outperformed the compared algorithms in terms of accuracy.