Convolutional neural networks (CNNs) have demo-nstrated outstanding performance on image classification. To classify the hyperspectral images (HSIs), existing CNN-based approaches commonly adopt the architecture using single or several fixed spatial windows as inputs. This kind of architecture may lose contextual information or incorporate heterogeneous information due to the neglect of various land-cover distributions in HSIs. To deal with this problem, a novel attention multibranch CNN method based on adaptive region search (RS-AMCNN) is proposed for HSI classification. In RS-AMCNN, sizes and locations of spatial windows are searched in the nonlocal candidate region adaptively according to sample-specific distribution. These flexible spatial windows are input into several branches of RS-AMCNN. In each branch, convolutional long short-term memories (ConvLSTMs) are merged into CNN from shallow to deep layers, which not only extracts joint spatial-spectral features, but also exploits complementary information among different layers. Then, a branch attention mechanism is devised to emphasize more discriminative branches and suppress less useful ones. It forces RS-AMCNN to extract multiscale and multicontextual attention features for classification. Finally, RS-AMCNN is optimized end-to-end by combining the losses from the ramose classifiers of different branches and the main classifier. Experiments carried on several benchmark HSI data sets demonstrate that RS-AMCNN provides promising classification performance, especially in edge preservation and region uniformity.

Methods based on generative adversarial network (GAN) have been widely used in infrared and visible images fusion. However, these methods cannot perceive the discriminative parts of an image. Therefore, we introduce a multi-grained attention module into encoder-decoder network to fuse infrared and visible images (MgAN-Fuse). The infrared and visible images are encoded by two independent encoder networks due to their diverse modalities. Then, the results of the two encoders are concatenated to calculate the fused result by the decoder. To exploit the features of multi-scale layers fully and force the model focus on the discriminative regions, we integrate attention modules into multi-scale layers of the encoder to obtain multi-grained attention maps, and then the multi-grained attention maps are concatenated with the corresponding multi-scale features of decoder network. Thus, the proposed method can preserve the foreground target information of the infrared image, and capture the context information of the visible image. Furthermore, we design an additional feature loss in the training process to preserve the important features of the visible image, and a dual adversarial architecture is employed to help the model capture enough infrared intensity information and visible details simultaneously. The ablation studies illustrate the validity of multi-grained attention network and feature loss function. Extensive experiments on two infrared and visible image datasets demonstrate that the proposed MgAN-Fuse has better performance than state-of-the-art methods.

Hyperspectral band selection is of great value to alleviate the curse of dimensionality. For many band selection methods, however, the neglect of bandwise usefulness tends to result in the loss of valuable bands, but the retention of useless ones; consequently, this causes deterioration of the classification performance. In this sense, bandwise significance should be emphasized. To address this issue, this article proposes a manifold-preserving and weakly redundant (MPWR) unsupervised band selection method. In the method, a manifold-preserving band-importance metric is put forward to measure the bandwise essentiality. This ensures the retention of bands involving abundant intrinsic structures conductive to classification. Specifically, aimed at obtaining the presented band-importance metric, an attainment algorithm is presented, which mainly relies on the embedding learning and linear regression, followed by the introduction of multi-normalization combination. In addition, concerning the massive redundancy caused by the highly correlated bands, MPWR further establishes a constrained band-weight optimization model. Then, both bandwise manifold-preserving capability and intraband correlation are fully integrated into the band selection process. To solve the problem, a corresponding algorithm within the framework of the alternating direction method of multipliers (ADMM) is also developed. Regarding evaluating the effectiveness of the proposed method, comparative experiments with the state-of-the-art methods are conducted on three public hyperspectral data sets. Experimental results demonstrate the superiority and robustness of MPWR.

Multimodal image fusion has recently garnered increasing interest in the field of remote sensing. By leveraging the complementary information in different modalities, the fused results may be more favorable in characterizing objects of interest, thereby increasing the chance of a more comprehensive and accurate perception of the scene. Unfortunately, most existing fusion methods tend to extract modality-specific features independently without considering inter-modal alignment and complementarity, leading to a suboptimal fusion process. To address this issue, we propose a novel interactive guided generative adversarial network, named IG-GAN, for the task of multi-modal image fusion. IG-GAN comprises guided dual streams tailored for enhanced learning of details and content, as well as cross-modal consistency. Specifically, a details-guided interactive running-in module and a content-guided interactive running-in module are developed, with the stronger modality serving as guidance for detail richness or content integrity, and the weaker one assisting. To fully integrate multi-granularity features from dual-modality, a hierarchical fusion and reconstruction branch is established. Specifically, a shallow interactive fusion module followed by a multi-level interactive fusion module is designed to aggregate multi-level local and long-range features. Concerning feature decoding and fused image generation, a high-level interactive fusion and reconstruction module is further developed. Additionally, to empower the fusion network to generate fused images with complete content, sharp edges, and high fidelity without supervision, a loss function facilitating the mutual game between the generator and two discriminators is also formulated. Comparative experiments with fourteen state-of-the-art methods are conducted on three datasets. Qualitative and quantitative results indicate that IG-GAN exhibits obvious superiority in terms of both visual effect and quantitative metrics. Moreover, experiments on two RGB-IR object detection datasets are also conducted, which demonstrate that IG-GAN can enhance the accuracy of object detection by integrating complementary information from different modalities.The code will be available at https://github.com/flower6top.

Band selection is an effective means to alleviate the curse of dimensionality in hyperspectral data. Many methods select a compact and low redundant band subset, which is inadequate as it may degrade the classification performance. Instead, more emphasis shall be put on selecting representative bands. In this article, we propose a robust unsupervised band selection method to address this issue. Our method reveals bandwise representativeness based on the comprehensive interband neighborhood structure. It incorporates an interband neighborhood graph into a sparse self-contained regression model in order to provide a reasonable measure for bandwise representativeness. The derived coefficient matrix not only uncovers bandwise importance values but also is coherent to the generalized interband local neighborhood structure. For constructing the interband neighboring structural graph, an integrated multigraph model is employed to achieve better generalization performance. It combines the benefit of multiple graphs but is insusceptible to the defects of a single one. To enhance the reliability of this model, a joint trace minimum and nonnegative constraint is imposed on the coefficient matrix. Accordingly, a multigraph integrated embedding and robust self-contained regression model (MGRSR) is formulated. In addition, an iterative update algorithm is developed to solve the problem. Comparative experiments on three hyperspectral data sets illustrate that MGRSR is robust to various data and has superior performance compared with several state-of-the-art methods.

The technology of non-contact heart rate detection has been proven of great use. However, it is still limited by several challenges, particularly the influence of human motion and random disturbances, which significantly degrade the accuracy of the measurement. To address these challenges, this paper proposes a novel scheme for heart rate detection. Firstly, the proposed scheme utilizes the Multi-Channel Averaging (MCA) technique to improve the signal-to-noise ratio (SNR) of the extracted phase signal from the echoes in the multiple receivers, and a high-pass filter to roughly extract a heartbeat signal. Secondly, to suppress motion artifacts (MA) in the heartbeat signal caused by human motion, the Adaptive Parameter Selection for Expectation-Maximization (APSEM) method is proposed. Further, a novel processing sub-framework which combines the Kalman filtering, the Variational Mode Extraction (VME) algorithm, and the Rife spectral analysis method (called KFRV method) is proposed to mitigate the effect of random disturbances and achieve accurate frequency estimation. Experimental results using the Polar H10 heart rate sensor as a reference show that the proposed scheme achieves accurate heart rate detection in the presence of human motion, with the Mean Absolute Error (MAE) of less than 2.5 beats per minute (bpm), which is much better than traditional schemes. Compared to traditional methods, the proposed scheme exhibits negligible loss in heart rate detection under static state, with an average MAE of 0.99 bpm. Overall, the experimental results demonstrate the applicability of the proposed scheme for accurate heart rate detection in both human motion and static states.