In the wake of developments in remote sensing, the application of target detection of remote sensing is of increasing interest. Unfortunately, unlike natural image processing, remote sensing image processing involves dealing with large variations in object size, which poses a great challenge to researchers. Although traditional multi-scale detection networks have been successful in solving problems with such large variations, they still have certain limitations: (1) The traditional multi-scale detection methods note the scale of features but ignore the correlation between feature levels. Each feature map is represented by a single layer of the backbone network, and the extracted features are not comprehensive enough. For example, the SSD network uses the features extracted from the backbone network at different scales directly for detection, resulting in the loss of a large amount of contextual information. (2) These methods combine with inherent backbone classification networks to perform detection tasks. RetinaNet is just a combination of the ResNet-101 classification network and FPN network to perform the detection tasks; however, there are differences in object classification and detection tasks. To address these issues, a cross-scale feature fusion pyramid network (CF2PN) is proposed. First and foremost, a cross-scale fusion module (CSFM) is introduced to extract sufficiently comprehensive semantic information from features for performing multi-scale fusion. Moreover, a feature pyramid for target detection utilizing thinning U-shaped modules (TUMs) performs the multi-level fusion of the features. Eventually, a focal loss in the prediction section is used to control the large number of negative samples generated during the feature fusion process. The new architecture of the network proposed in this paper is verified by DIOR and RSOD dataset. The experimental results show that the performance of this method is improved by 2–12% in the DIOR dataset and RSOD dataset compared with the current SOTA target detection methods.
Recently, hyperspectral image (HSI) classification methods based on convolutional neural networks (CNN) have developed rapidly with the advance of deep learning (DL) techniques. In order to more efficiently extract spatial and spectral features, we propose an end-to-end cubic convolutional neural network (Cubic-CNN) in this paper. The proposed Cubic-CNN is a supervised DL framework that significantly improves classification accuracy and shortens training time. Specifically, Cubic-CNN employs the dimension reduction method combined with principal component analysis (PCA) and 1D convolution to remove redundant information from HSIs. Then, convolutions are performed on the planes in different directions of the feature cube data to fully extract spatial and spatial-spectral features and fuse the features from different dimensions. In addition, we performed batch normalization on the data cube after each convolutional layer to improve the performance of the network. Extensive experiments and analysis on standard datasets show that the proposed algorithm can outperform the existing state-of-the-art DL-based methods.
6 August 2020 in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
The superpixel-based multiple kernels (MK) model uses the average value of all pixels within superpixel as the spatial feature, which results in inaccurate extraction of edge pixels. To solve this problem, a local binary patterns and superpixel-based multiple kernels (LBP-SMK) method is proposed for hyperspectral image (HSI) classification. Firstly, the original hyperspectral image is segmented into multiple superpixels by using the entropy rate superpixel (ERS) segmentation algorithm. On the HSI with superpixel index, the spectral kernel is secondly obtained by combining the spectral feature map with the radial basis kernel (RBF); By introducing local binary pattern (LBP) and weighted average filtering into RBF, the spatial kernels are obtained within and among superpixels. Finally, the combined kernel containing the above three kernels is inputted into the support vector machine (SVM) classifier to generate a classification map. The experimental procedures in this paper used LBP to extract the information in superpixels, which prevents the loss of edge features in superpixels. The experimental results show that the proposed method is superior to the state-of-the-art classifiers for HSI classification.
20 September 2022 in IEEE Transactions on Geoscience and Remote Sensing
Hyperspectral image (HSI) classification refers to accurately corresponding each pixel in an HSI to a land-cover label. Recently, the successful application of multiscale and multifeature methods has greatly improved the performance of HSI classification due to their enhanced utilization of the available spectral–spatial information. However, as the number of scales and the number of features increases, it becomes more difficult to achieve an optimal degree of fusion for multiple classifiers [e.g., kernel extreme learning machine (KELM)]. On the other hand, a limited sample size of the HSI may cause overfitting problems, which seriously affects the classification accuracy. Therefore, in this article, a novel multi-structure KELM with attention fusion strategy (MSAF-KELM) is proposed to achieve accurate fusion of multiple classifiers for effective HSI classification with ultrasmall sample rates. First, a multi-structure network is built, which combines multiple scales and multiple features to extract abundant spectral–spatial information. Second, a fast and efficient KELM is employed to enable rapid classification. Finally, a weighted self-attention fusion strategy (WSAFS) is introduced, which combines the output weights of each KELM subbranch and the self-attention mechanism to achieve an efficient fusion result on multi-structure networks. We conducted experiments on four types of HSI datasets with different evaluation methods and compared them with several classical and state-of-the-art methods, which demonstrate the excellent performance of our method on ultrasmall sample rates. The code is available at https://github.com/Fang666666/MSAF-KELM for reproducibility.
18 December 2023 in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
The objects in optical high-resolution remote sensing images (HRRSIs) are usually tiny, dense, and exist in complex backgrounds, which brings great challenges to accurate object detection. This article presents an improved YOLOv5s network-based technique for remote sensing object recognition to overcome these difficulties. First, unnecessary residual modules are pruned from the cross-stage partial layer of conventional YOLOv5s and a refined residual coordinate attention module is incorporated to enhance the feature representation of the densely packed small objects in HRRSIs by introducing the residual structure and the mix pooling operation instead of the existing average pooling. Second, since various scales of objects are present in HRRSIs, the algorithm of differential evolution is adopted to replace the traditional K-means for generating a variety of anchor boxes in different sizes. Third, we replace the commonly used complete intersection over union (IoU) loss function in YOLOv5s with the AW-IoU loss function based on both α-IoU and wise-IoU. AW-IoU could expedite bounding box regression and focus more on regular anchor boxes. Finally, instead of nonmaximum suppression (NMS), the SCYLLA (S-IoU) soft-NMS is employed to eliminate the redundant duplicate boxes to detect the dense objects in remote sensing images. Experimental results on the NWPU VHR-10 dataset demonstrate that the proposed YOLOv5s method performs well compared with state-of-the-art algorithms.