Heterogeneous defect prediction (HDP) aims to predict the defect tendency of modules in one project using heterogeneous data collected from other projects. It sufficiently incorporates the two characteristics of the defect prediction data: (1) datasets could have different metrics and distribution, and (2) data could be highly imbalanced. In this paper, we propose a few-shot learning based balanced distribution adaptation (FSLBDA) approach for heterogeneous defect prediction, which takes into consideration the two characteristics of the defect prediction data. Class imbalance of the defect datasets can be solved with undersampling, but the scale of the training datasets will be smaller. Specifically, we first remove redundant metrics of datasets with extreme gradient boosting. Then, we reduce the data difference between the source domain and the target domain with the balanced distribution adaptation. It considers the marginal distribution and the probability of conditional distribution differences and adaptively assigns different weights to them. Finally, we use adaptive boosting to relieve the influence caused by the size of the training dataset is smaller, which can improve the accuracy of the defect prediction model. We conduct experiments on 17 projects from 4 datasets using 3 indicators (i.e., AUC, G-mean, F-measure). Compared to three classic approaches, the experimental results show that FSLBDA can effectively improve the prediction performance.

HSI has abundant spectral–spatial information. Using this information to improve the accuracy of HSI classification is a hot issue in the industry. This letter proposes an end-to-end multilevel hybrid attention network (DMCN). It is composed of a dense 3-D convolutional neural network (3D-CNN), grouped residual 2D-CNN, and coordinate attention that can perceive categories. In the case of a small number of training samples, DMCN can still extract spectral–spatial fusion information and learn spatial features more deeply for classification. Experiments are conducted on three well-known hyperspectral datasets, i.e., Indian Pines (IP), University of Pavia (UP), and Salinas (SA). The results show that DMCN achieved 92.39%, 97.28%, and 98.40% classification accuracy in IP, UP, and SA.

Utilizing multi-modal data, as opposed to only hyperspectral image (HSI), enhances target identification accuracy in remote sensing. Transformers are applied to multi-modal data classification for their long-range dependency but often overlook intrinsic image structure by directly flattening image blocks into vectors. Moreover, as the encoder deepens, unprofitable information negatively impacts classification performance. Therefore, this paper proposes a learnable transformer with an adaptive gating mechanism (AGMLT). Firstly, a spectral–spatial adaptive gating mechanism (SSAGM) is designed to comprehensively extract the local information from images. It mainly contains point depthwise attention (PDWA) and asymmetric depthwise attention (ADWA). The former is for extracting spectral information of HSI, and the latter is for extracting spatial information of HSI and elevation information of LiDAR-derived rasterized digital surface models (LiDAR-DSM). By omitting linear layers, local continuity is maintained. Then, the layer Scale and learnable transition matrix are introduced to the original transformer encoder and self-attention to form the learnable transformer (L-Former). It improves data dynamics and prevents performance degradation as the encoder deepens. Subsequently, learnable cross-attention (LC-Attention) with the learnable transfer matrix is designed to augment the fusion of multi-modal data by enriching feature information. Finally, poly loss, known for its adaptability with multi-modal data, is employed in training the model. Experiments in the paper are conducted on four famous multi-modal datasets: Trento (TR), MUUFL (MU), Augsburg (AU), and Houston2013 (HU). The results show that AGMLT achieves optimal performance over some existing models.

Transformers are increasingly popular in computer vision, which treat an image as a sequence of image patches and learn robust global features from the sequence. However, pure transformers are not entirely suitable for hyperspectral and LiDAR image classification because image classification requires both robust global features and discriminative local features. Therefore, this paper introduces a novel convolution interaction transformer network (CITNet) for jointly classifying hyperspectral and LiDAR images. The process begins with a carefully designed multi-scale asymmetric depthwise convolution (MADC) module that exploits the local-global correlations of shallow features. On this basis, a novel local-global transformer (LGTM) is equipped with a local-global feed-forward network (LGF) to extract in-depth local-global joint features from the multimodal data. Then, an optimization convolution cross-attention (OCA) module, incorporating a convolution layer, is developed to simulate the spatial relationships of semantic tokens. Finally, extensive experiments are conducted on the well-known Trento (TR), Augsburg (AU), MUUFL (MU), and Houston2013 (HU) datasets. The overall accuracy (OA) reaches 99.76%, 97.40%, 91.06%, and 99.90%, respectively, which are 0.2%-1.66%, 0.32%-7.37%, 1.52%-12.71%, and 0.14%-93.79% higher than the state-of-the-art (SOTA) methods, demonstrating the effectiveness of CITNet in improving the joint classification accuracy of hyperspectral and LiDAR images.

In hyperspectral image classification (HSIC), every pixel of the HSI is assigned to a land cover category. While convolutional neural network (CNN)-based methods for HSIC have significantly enhanced performance, they encounter challenges in learning the relevance of deep semantic features and grappling with escalating computational costs as network depth increases. In contrast, the transformer framework is adept at capturing the relevance of high-level semantic features, presenting an effective solution to address the limitations encountered by CNN-based approaches. This article introduces a novel adaptive learnable spectral–spatial fusion transformer (ALSST) to enhance HSI classification. The model incorporates a dual-branch adaptive spectral–spatial fusion gating mechanism (ASSF), which captures spectral–spatial fusion features effectively from images. The ASSF comprises two key components: the point depthwise attention module (PDWA) for spectral feature extraction and the asymmetric depthwise attention module (ADWA) for spatial feature extraction. The model efficiently obtains spectral–spatial fusion features by multiplying the outputs of these two branches. Furthermore, we integrate the layer scale and DropKey into the traditional transformer encoder and multi-head self-attention (MHSA) to form a new transformer with a layer scale and DropKey (LD-Former). This innovation enhances data dynamics and mitigates performance degradation in deeper encoder layers. The experiments detailed in this article are executed on four renowned datasets: Trento (TR), MUUFL (MU), Augsburg (AU), and the University of Pavia (UP). The findings demonstrate that the ALSST model secures optimal performance, surpassing some existing models. The overall accuracy (OA) is 99.70%, 89.72%, 97.84%, and 99.78% on four famous datasets: Trento (TR), MUUFL (MU), Augsburg (AU), and University of Pavia (UP), respectively.

Methods based on convolutional neural networks (CNNs) have become a vital offshoot for hyperspectral image (HSI) classification. In recent years, the three-dimensional convolutional neural network (3DCNN) has become dominant due to its excellent capability of extracting features. However, the high dimension and the limited training samples of HSI usually restrict the improvement of its classification accuracy. And the parameters of conventional 3DCNN are larger so that computational complexity and running time increase. Therefore, a new model named HSSAM is proposed to solve the above problems. First, a three-dimensional residual-dense asymmetric convolution (3D-RDAC) is designed to reuse the features, while reducing the parameters. Subsequently, 3D-RDAC combined with the multiscale convolution to construct a three-dimensional multiscale residual-dense asymmetric convolution (3D-MRDAC) for avoiding the blind spots and unrecognized regions of receiving fields. Then, 3D attention SimAM is applied to 3D-MRDAC, for constituting the heterogeneous spectral-spatial attention convolutional neural (HSSAN) block, to extract spectral-spatial features of HSI adequately. Ultimately, MLP acts as the output layer of the model to better deal with the nonlinear features of HSI. Experiments in this article are carried out on four famous hyperspectral datasets: Indian Pines (IP), Pavia University (PU), WHU-Hi-LongKou (LK), and WHU-Hi-HanChuan (HC). Results show that HSSAM achieves better classification accuracy with limited training samples than several existing models. OA reaches 96.84%, 98.85%, 98.01%, and 97.18% on the four datasets, respectively.