We study the problem of object detection in remote sensing images. As a simple but effective feature extractor, Feature Pyramid Network (FPN) has been widely used in several generic vision tasks. However, it still faces some challenges when used for remote sensing object detection, as the objects in remote sensing images usually exhibit variable shapes, orientations, and sizes. To this end, we propose a dedicated object detector based on the FPN architecture to achieve accurate object detection in remote sensing images. Specifically, considering the variable shapes and orientations of remote sensing objects, we first replace the original lateral connections of FPN with Deformable Convolution Lateral Connection Modules (DCLCMs), each of which includes a $3\times 3$ deformable convolution to generate feature maps with deformable receptive fields. Additionally, we further introduce several Attention-based Multi-Level Feature Fusion Modules (A-MLFFMs) to integrate the multi-level outputs of FPN adaptively, further enabling multi-scale object detection. Extensive experimental results on the DIOR dataset demonstrated the state-of-the-art performance achieved by the proposed method, with the highest mean Average Precision (mAP) of 73.6%.

In this letter, a novel object detection method based on feature pyramid network (FPN) is proposed to improve the detection performance of remote sensing objects. First, since the information in the background regions may interfere with object detection, a novel multiscale deformable attention module (MSDAM) is designed and added on the top of the backbone of FPN to make the network suppress the background features while highlighting the target features. The proposed MSDAM generates attention maps from feature maps with multiscale deformable receptive fields, thus can fit remote sensing objects of various shapes and sizes better and predict more precise attention maps for remote sensing images. Second, in the original FPN, each proposal is predicted based on feature grids pooled from only one feature level. This process is suboptimal as the information discarded in other feature levels and the global contextual information are also meaningful to object detection. Thus, a multilevel features aggregation module (MLFAM) is proposed to aggregate the multilevel outputs of FPN and the global context of the whole image, generating more powerful pyramidal representations for the subsequent object detection. The experiments conducted on the object detection in optical remote sensing images (DIOR) and remote sensing object detection (RSOD) datasets demonstrate the superiority of the proposed method over the considered state-of-the-art baseline methods in terms of detection accuracy.

Due to the rapid development of deep learning techniques and the collection of large-scale remote sensing datasets, convolutional neural networks (CNNs) have made significant progress in remote sensing object detection. However, due to the diversity of objects in remote sensing images, multiscale object detection is still a challenging task. In this letter, a novel object detection framework based on feature pyramid network (FPN) is proposed to improve the detection performance of multiscale objects. First, a receptive field expansion block (RFEB) is designed and added on the top of the backbone to expand the receptive field of FPN adaptively. In this way, the context information around each object is well captured. Then, the features obtained via RFEB are delivered to feature maps at all pyramid levels, remedying the drawback of FPN that semantic information captured by deep layers is gradually diluted when transmitted to lower layers. Third, since the classic backbone of FPN, which produces large receptive fields based on large downsampling factors, may limit the effectiveness of RFEB, the backbone of the original FPN is modified using dilated convolution to ease the resolution drop of feature maps while maintaining a large receptive field. As a feature extractor, the proposed framework can be easily deployed in other FPN-based methods. The experiments on the benchmark for object DetectIon in Optical Remote sensing images (DIOR) dataset demonstrate the proposed method’s superiority over considered state-of-the-art baseline methods in terms of detection accuracy.

Recently, deep learning algorithms, especially feature pyramid network (FPN), have achieved significant progress in object detection of natural scene images. However, due to the complex scenes of remote sensing images and the diversity of remote sensing objects, FPN still faces the following drawback when applied to remote sensing object detection. Specifically, in the original FPN, the features of each proposal are extracted by RoIAlign. However, these features have limited effective receptive fields, making FPN lack of crucial contextual information to accurately classify and locate objects, as well as filter some background noises that possess similar appearance with objects. To alleviate the above problem, in this letter, we propose a gated context aware module (G-CAM), and replace the original RoIAlign in FPN with the proposed G-CAM to adaptively incorporate the useful local context surrounding each proposal and the global context of the whole image into FPN, enabling FPN to effectively detect objects in remote sensing images Extensive experiments have been conducted on the DIOR and RSOD datasets, which validates that the proposed method achieves superior performance to the considered state-of-the-art methods in terms of detection accuracy.

To have a better understanding and usage of Convolution Neural Networks (CNNs), the visualization and interpretation of CNNs has attracted increasing attention in recent years. In particular, several Class Activation Mapping (CAM) methods have been proposed to discover the connection between CNN's decision and image regions. In spite of the reasonable visualization, lack of clear and sufficient theoretical support is the main limitation of these methods. In this paper, we introduce two axioms -- Conservation and Sensitivity -- to the visualization paradigm of the CAM methods. Meanwhile, a dedicated Axiom-based Grad-CAM (XGrad-CAM) is proposed to satisfy these axioms as much as possible. Experiments demonstrate that XGrad-CAM is an enhanced version of Grad-CAM in terms of conservation and sensitivity. It is able to achieve better visualization performance than Grad-CAM, while also be class-discriminative and easy-to-implement compared with Grad-CAM++ and Ablation-CAM. The code is available at https://github.com/Fu0511/XGrad-CAM.