Although the orientation and scale properties of the objects in remote sensing images have been widely considered in the modern deep learning-based object detection methods, the spatial distribution property of objects has rarely been investigated. There is a distinct spatial distribution difference between close-range objects and remote sensing objects: the former may exhibit extensive mutual occlusion and overlap, whereas the latter rarely overlap. A current remote sensing object detection algorithm that ignores the spatial distribution difference may unnecessarily apply the massive anchor-based proposal bounding box generation and nonmaximum suppression (NMS) operations. In this article, considering the unique spatial distribution of remote sensing objects, and also the other spatial properties, we propose a novel, compact, and spatial-oriented object detection framework for remote sensing images. The proposed two-stage convolutional neural network (CNN) framework, which we call the Remote-sensing Spatial Adaptation DETector (RSADet), considers the spatial distribution, scale, and orientation/shape varieties of the objects in remote sensing images. In the first stage, each object instance is inferred on the scale-attention boosted CNN heatmaps to generate candidate bounding boxes, instead of using the anchor-based proposal box generation and NMS. In the second stage, deformable convolutions are introduced to adapt to the geometric variations of different object instances and to avoid the impact of complex and changeable backgrounds. A new bounding box confidence (IoU score) prediction branch is introduced as a convenient constraint for eliminating unreliable boxes and improving performance. Experiments were conducted on a large single-class remote sensing object detection dataset (the Ningbo Pylon dataset) built as part of this study and an open-source extraordinarily large multiclass dataset (the object DetectIon in Optical Remote sensing image (DIOR) dataset). Compared with the advanced detectors from both the computer vision and remote sensing communities, the proposed RSADet achieved state-of-the-art performance on both datasets.

Recently proposed spherical convolutional neural networks (SCNNs) have shown advantages over conventional planar CNNs on classifying spherical images. However, two factors hamper their application in an objection detection task. First, a convolution in S2 (a two-dimensional sphere in three-dimensional space) or SO(3) (three-dimensional special orthogonal group) space results in the loss of an object’s location. Second, overlarge bandwidth is required to preserve a small object’s information on a sphere because the S2/SO(3) convolution must be performed on the whole sphere, instead of a local image patch. In this study, we propose a novel grid-based spherical CNN (G-SCNN) for detecting objects from spherical images. According to input bandwidth, a sphere image is transformed to a conformal grid map to be the input of the S2/SO3 convolution, and an object’s bounding box is scaled to cover an adequate area of the grid map. This solves the second problem. For the first problem, we utilize a planar region proposal network (RPN) with a data augmentation strategy that increases rotation invariance. We have also created a dataset including 600 street view panoramic images captured from a vehicle-borne panoramic camera. The dataset contains 5636 objects of interest annotated with class and bounding box and is named as WHU (Wuhan University) panoramic dataset. Results on the dataset proved our grid-based method is extremely better than the original SCNN in detecting objects from spherical images, and it outperformed several mainstream object detection networks, such as Faster R-CNN and SSD.

Detecting the terrain surface cracks caused by earthquakes, which are termed coseismic ruptures, has important significance for discovering concealed faults, monitoring their movements, and forecasting possible follow-on earthquakes. On May 22, 2021, Maduo County in Qinghai province, China, suffered an earthquake with a magnitude of 7.4, which created densely distributed cracks. In this study, we designed an automatic crack detection framework based on remote sensing technology. With the use of low-altitude unmanned aerial vehicles (UAVs), we obtained very high-resolution aerial images of the area affected by the earthquake, which were further processed by photogrammetric software to produce digital orthophoto maps (DOMs). We then designed a novel terrain surface crack detection neural network, which differs from the previous methods that focus on detecting cracks in man-made object surfaces such as flat roads. We investigated the spatial property of the sinuous linear cracks and handled this by introducing adaptive deformable convolutions with a context-channel-space boosted mechanism. The feature extraction stage, feature optimization stage, and upsampling stage were embedded with the deformable convolutions to form a compact and powerful crack detector, named Crack-CADNet [the Context-chAnnel-space boosted Deformable convolutional neural network (CNN) for crack detection]. The postprocessing included filtering out the nontectonic cracks, aided by annotations from experts, and grouping and vectorizing the generated binary segmentation map as crack polygons, which were evaluated at the instance level. In addition to the first in-depth investigation of detecting earthquake cracks with aerial remote sensing and a deep learning-based process, the crack detection network we propose outperformed the recent CNN-based methods designed for general semantic segmentation and crack detection. Source code and the Maduo earthquake crack dataset will be available at http://gpcv.whu.edu.cn/data/ .

We develop a deep learning-based matching method between an RGB (red, green and blue) and an infrared image that were captured from satellite sensors. The method includes a convolutional neural network (CNN) that compares the RGB and infrared image pair and a template searching strategy that searches the correspondent point within a search window in the target image to a given point in the reference image. A densely-connected CNN is developed to extract common features from different spectral bands. The network consists of a series of densely-connected convolutions to make full use of low-level features and an augmented cross entropy loss to avoid model overfitting. The network takes band-wise concatenated RGB and infrared images as the input and outputs a similarity score of the RGB and infrared image pair. For a given reference point, the similarity scores within the search window are calculated pixel-by-pixel, and the pixel with the highest score becomes the matching candidate. Experiments on a satellite RGB and infrared image dataset demonstrated that our method obtained more than 75% improvement on matching rate (the ratio of the successfully matched points to all the reference points) over conventional methods such as SURF, RIFT, and PSO-SIFT, and more than 10% improvement compared to other most recent CNN-based structures. Our experiments also demonstrated high performance and generalization ability of our method applying to multitemporal remote sensing images and close-range images.

Long-range dependency modeling has been widely considered in modern deep learning based semantic segmentation methods, especially those designed for large-size remote sensing images, to compensate the intrinsic locality of standard convolutions. However, in previous studies, the long-range dependency, modeled with an attention mechanism or transformer model, has been based on unsupervised learning, instead of explicit supervision from the objective ground truth. In this paper, we propose a novel supervised long-range correlation method for land-cover classification, called the supervised long-range correlation network (SLCNet), which is shown to be superior to the currently used unsupervised strategies. In SLCNet, pixels sharing the same category are considered highly correlated and those having different categories are less relevant, which can be easily supervised by the category consistency information available in the ground truth semantic segmentation map. Under such supervision, the recalibrated features are more consistent for pixels of the same category and more discriminative for pixels of other categories, regardless of their proximity. To complement the detailed information lacking in the global long-range correlation, we introduce an auxiliary adaptive receptive field feature extraction module, parallel to the long-range correlation module in the encoder, to capture finely detailed feature representations for multi-size objects in multi-scale remote sensing images. In addition, we apply multi-scale side-output supervision and a hybrid loss function as local and global constraints to further boost the segmentation accuracy. Experiments were conducted on three public remote sensing datasets (the ISPRS Vaihingen dataset, the ISPRS Potsdam dataset, and the DeepGlobe dataset). Compared with the advanced segmentation methods from the computer vision, medicine, and remote sensing communities, the proposed SLCNet method achieved a state-of-the-art performance on all the datasets. The code will be made available at gpcv.whu.edu.cn/data.