Brain functional network analysis has shown great potential in understanding brain functions and also in identifying biomarkers for brain diseases, such as Alzheimer's disease (AD) and its early stage, mild cognitive impairment (MCI). In these applications, accurate construction of biologically meaningful brain network is critical. Sparse learning has been widely used for brain network construction; however, its l₁‐norm penalty simply penalizes each edge of a brain network equally, without considering the original connectivity strength which is one of the most important inherent linkwise characters. Besides, based on the similarity of the linkwise connectivity, brain network shows prominent group structure (i.e., a set of edges sharing similar attributes). In this article, we propose a novel brain functional network modeling framework with a “connectivity strength‐weighted sparse group constraint.” In particular, the network modeling can be optimized by considering both raw connectivity strength and its group structure, without losing the merit of sparsity. Our proposed method is applied to MCI classification, a challenging task for early AD diagnosis. Experimental results based on the resting‐state functional MRI, from 50 MCI patients and 49 healthy controls, show that our proposed method is more effective (i.e., achieving a significantly higher classification accuracy, 84.8%) than other competing methods (e.g., sparse representation, accuracy = 65.6%). Post hoc inspection of the informative features further shows more biologically meaningful brain functional connectivities obtained by our proposed method. Hum Brain Mapp 38:2370–2383, 2017. © 2017 Wiley Periodicals, Inc.

The explorations of brain functional connectivity (FC) network using resting-state functional magnetic resonance imaging (rs-fMRI) can provide crucial insights into discriminative analysis of neuropsychiatric disorders such as schizophrenia (SZ). Graph attention network (GAT), which could capture the local stationary on the network topology and aggregate the features of neighboring nodes, has advantages in learning the feature representation of brain regions. However, GAT only can obtain the node-level features that reflect local information, ignoring the spatial information within the connectivity-based features that proved to be important for SZ diagnosis. In addition, existing graph learning techniques usually rely on a single graph topology to represent neighborhood information, and only consider a single correlation measure for connectivity features. Comprehensive analysis of multiple graph topologies and multiple measures of FC can leverage their complementary information that may contribute to identifying patients. In this paper, we propose a multi-graph attention network (MGAT) with bilinear convolution (BC) neural network framework for SZ diagnosis and functional connectivity analysis. Besides multiple correlation measures to construct connectivity networks from different perspectives, we further propose two different graph construction methods to capture both the low- and high-level graph topologies, respectively. Especially, the MGAT module is developed to learn multiple node interaction features on each graph topology, and the BC module is utilized to learn the spatial connectivity features of the brain network for disease prediction. Importantly, the rationality and advantages of our proposed method can be validated by the experiments on SZ identification. Therefore, we speculate that this framework may also be potentially used as a diagnostic tool for other neuropsychiatric disorders.

Object detection in remote sensing images has received significant attention for a wide range of applications. However, traditional unimodal remote sensing images, whether based on visible light or infrared, have limitations that cannot be ignored. Visible light images are susceptible to ambient lighting conditions, and their detection accuracy can be greatly reduced. Infrared images often lack rich texture information, resulting in a high false-detection rate during target identification and classification. To address these challenges, we propose a novel multimodal fusion network detection model, named ACDF-YOLO, basedon the lightweight and efficient YOLOv5 structure, which aims to amalgamate synergistic data from both visible and infrared imagery, thereby enhancing the efficiency of target identification in remote sensing imagery. Firstly, a novel efficient shuffle attention module is designed to assist in extracting the features of various modalities. Secondly, deeper multimodal information fusion is achieved by introducing a new cross-modal difference module to fuse the features that have been acquired. Finally, we combine the two modules mentioned above in an effective manner to achieve ACDF. The ACDF not only enhances the characterization ability for the fused features but also further refines the capture and reinforcement of important channel features. Experimental validation was performed using several publicly available multimodal real-world and remote sensing datasets. Compared with other advanced unimodal and multimodal methods, ACDF-YOLO separately achieved a 95.87% and 78.10% mAP0.5 on the LLVIP and VEDAI datasets, demonstrating that the deep fusion of different modal information can effectively improve the accuracy of object detection.

With the development of resting-state functional magnetic resonance imaging (rs-fMRI) technology, the functional connectivity network (FCN) which reflects the statistical similarity of temporal activity between brain regions has shown promising results for the identification of neuropsychiatric disorders. Alteration in FCN is believed to have the potential to locate biomarkers for classifying or predicting schizophrenia (SZ) from healthy control. However, the traditional FCN analysis with stationary assumption, i.e., static functional connectivity network (SFCN) at the time only measures the simple functional connectivity among brain regions, ignoring the dynamic changes of functional connectivity and the high-order dynamic interactions. In this article, the dynamic functional connectivity network (DFCN) is constructed to delineate the characteristic of connectivity variation across time. A high-order functional connectivity network (HFCN) designed based on DFCN, could characterize more complex spatial interactions across multiple brain regions with the potential to reflect complex functional segregation and integration. Specifically, the temporal variability and the high-order network topology features, which characterize the brain FCNs from region and connectivity aspects, are extracted from DFCN and HFCN, respectively. Experiment results on SZ identification prove that our method is more effective (i.e., obtaining a significantly higher classification accuracy, 81.82%) than other competing methods.Post hocinspection of the informative features in the individualized classification task further could serve as the potential biomarkers for identifying associated aberrant connectivity in SZ.

Multiple imaging modalities and specific proteins in the cerebrospinal fluid, providing a comprehensive understanding of neurodegenerative disorders, have been widely used for computer-aided diagnosis of Alzheimer’s disease (AD). Given the proven effectiveness of contrastive learning in aligning multimodal representation, in this paper, we investigate effective contrastive learning strategies to learn better cross-modal representations for the integration of multi-modal complementary information. To enhance the overall performance in AD diagnosis, we construct a unified hybrid network that integrates feature learning and classifier learning into an end-to-end framework. Specifically, we propose a weighted multi-modal contrastive learning based on hybrid network (WMCL-HN) method. Firstly, an adaptive weighted strategy is implemented on the multi-modal contrastive learning to dynamically regulate the degree of information exchange across modalities. It assigns higher weights to more important modality pair, thus the most important underlying relationships across modalities can be captured. Secondly, we construct a hybrid network, which employs a curriculum learning strategy that gradually transitions the training from feature learning to classifier learning, ensuring that the learned features are tailored to the diagnostic task. Experimental results on ADNI dataset demonstrate the effectiveness of the proposed WMCL-HN in AD-related diagnosis tasks. The source code is available at https://github.com/pcehnago/WMCL-HN.