Object detection is an essential task in computer vision. Recently, several convolution neural network (CNN)-based detectors have achieved a great success in natural scenes. However, for optical remote sensing images with a large scale of view, lower proportion of foreground target pixels and drastic differences in object scale present considerable challenges. To address these problems, we propose a novel one-stage detector called the full-scale object detection network (FSoD-Net) which consists of proposed multiscale enhancement network (MSE-Net) backbone cascaded with scale-invariant regression layers (SIRLs). First, MSE-Net provides the multiscale description enhancement by integrated the Laplace kernel with fewer parallel multiscale convolution layers. Second, SIRLs contain three different isolated regression branch layers (i.e., corresponding to small, medium, and large scales), which make default discrete scale bounding boxes (bboxes) cover full-scale object information in regression procedure. A novel specific scale joint loss is also designed that uses the softmax function combined with a strong $L_{1}$ -norm constraint in each regression branch layer. It can further speed up the convergence and improve the classification scores of predicted bboxes. Finally, extensive experiments are carried on challenge data sets of large-scale dataset for object detection in aerial images (DOTA) and object detection in optical remote sensing images (DIOR) which contain multiple instances from different imaging platforms, and these results demonstrate that FSoD-Net can achieve better performance than other state-of-the-art one-stage detectors, and it can reach a mean average precision (mAP) of 75.33% on DOTA and 71.80% mAP on DIOR, respectively. Especially, the average precision (AP) of tiny object detection can improve 10%–20% approximately.

Deep learning (DL) technique is widely applied in remote sensing (RS) applications because of its outstanding nonlinear feature extraction ability. However, with regard to the issues of large-scale and very high-resolution (VHR) land cover classification, multi-object distributions and clear appearance with large intraclass difference become challenges for refined pixelwise land cover mapping. Focusing on these problems, the letter proposed a novel encoding-to-decoding method called the full receptive field (RF) network (FRF-Net) based on two types of attention mechanism. In the FRF-Net, ResNet-101 is used as the basic backbone. Then, the ensemble feature is generated by encoding the high-level features based on the self-attention mechanism which could achieve full RF to capture long-range semantic. Next, the encoding result is decoded by the fusion attention mechanism combined with the low-level feature to produce a fusion feature which contains a refined semantic description for accurate land cover mapping. Extensive experiments based on the GID and ISPRS data sets proved that the proposed network outperforms the state-of-the-art methods. The FRF-Net achieved 66.71% and 64.17% of the mean of classwise Intersection over Union (mIOU) with smaller computation cost on ISPRS and GID, respectively.

Deep Convolutional Neural Network (DCNN)-based image scene classification models play an important role in a wide variety of remote sensing applications and achieve great success. However, the large-scale remote sensing images and the intensive computations make the deployment of these DCNN-based models on low-power processing systems (e.g., spaceborne or airborne) a challenging problem. To solve this problem, this paper proposes a high-performance Field-Programmable Gate Array (FPGA)-based DCNN accelerator by combining an efficient network compression scheme and reasonable hardware architecture. Firstly, this paper applies the network quantization to a high-accuracy remote sensing scene classification network, an improved oriented response network (IORN). The volume of the parameters and feature maps in the network is greatly reduced. Secondly, an efficient hardware architecture for network implementation is proposed. The architecture employs dual-channel Double Data Rate Synchronous Dynamic Random-Access Memory (DDR) access mode, rational on-chip data processing scheme and efficient processing engine design. Finally, we implement the quantized IORN (Q-IORN) with the proposed architecture on a Xilinx VC709 development board. The experimental results show that the proposed accelerator has 88.31% top-1 classification accuracy and achieves a throughput of 209.60 Giga-Operations Per Second (GOP/s) with a 6.32 W on-chip power consumption at 200 MHz. The comparison results with off-the-shelf devices and recent state-of-the-art implementations illustrate that the proposed accelerator has obvious advantages in terms of energy efficiency.

Land cover classification has achieved significant advances by employing deep convolutional network (ConvNet) based methods. Following the paradigm of learning deep models, land cover classification is modeled as semantic segmentation of very high resolution (VHR) remote sensing images. In order to obtain accurate segmentation results, high-level categorical semantics and low-level spatial details should be effectively fused. To this end, we propose a novel bidirectional gird fusion network to aggregate the multi-level features across the ConvNet. Specifically, the proposed model is characterized by a bidirectional fusion architecture, which enriches diversity of feature interaction by encouraging bidirectional information flow. In this way, our model gains mutual benefits between top-down and bottom-up information flows. Moreover, a grid fusion architecture is then followed for further feature refinement in a dense and hierarchical fusion manner. Finally, effective feature up-sampling is also critical for the multiple fusion operations. Consequently, a content-aware feature upsampling kernel is incorporated for further improvement. Our whole model consistently achieves significant improvement over state-of-the-art methods on two major datasets, ISPRS and GID.

Development of very high resolution (VHR) remote sensing imaging platforms have resulted in a requirement for developing refined land cover classification maps for various applications. Therefore, aiming at exploring the accurate boundary and complex interior texture retrieval in VHR optical remote sensing images, a novel detail injection network (DI-Net) is proposed in this paper, which is composed of three aspects. First, the decoupling refinement module (DRM) embedded with a multiscale representation is designed to improve the feature extraction capabilities that precede the encoding-to-decoding process. Second, we pay attention to the hard examples of boundary and complex interior texture in land cover classification and design two detail injection attention modules to solve the feature inactivation phenomenon in gradually convolutional encoding-to-decoding process. Third, a specific stage grading (SG) loss is proposed to adaptively regulate the structural-level weights of the encoding and decoding stages, which facilitates the details retrieval and produce refined land cover classification results. Finally, various datasets (incl. ISPRS and GID) are employed to demonstrate that the proposed DI-Net achieves better performance than state-of-the-art methods. DI-Net provides more accurate boundaries and more consistent interior textures, and it achieves 86.86% PA and 68.37% mIoU on ISPRS dataset as well as 77.04% PA and 64.38% mIoU on GID dataset, respectively.