Remote sensing image analysis has drawn more attentions in the field of computer vision. At present, the methods commonly used in remote sensing image analysis are mainly at a low level semantically. The scene graph is an abstraction of objects and their relationships, which is a high-level image understanding task. To fully comprehend the meanings of remote sensing images, in this article, we propose a novel segmentation-based model to generate remote sensing image scene graphs (SRSG). In the SRSG model, a more complete and accurate scene graph is generated with the segmentation results as inputs, while the shapes of objects are reasonably coded. The morphological features of object pairs are embedded together by different branches of the SRSG model and then mapped to semantic space to predicate their relationships. Furthermore, a new dataset for scene graph generation of remote sensing images, namely, segmentation results to scene graphs (S2SG), is constructed based on pixel-level segmentation results. Experimental results demonstrate that the performance of the SRSG model is far superior to the previous methods in the task of generating remote sensing image scene graphs. The proposed SRSG model opens up new possibilities for remote sensing image analysis at a high level. Moreover, the S2SG dataset further allows for the evaluation of different approaches and is provided for the benefit of the research community.

To semantically understand remote sensing images, it is not only necessary to detect the objects in them but also to recognize the semantic relationships between the instances. Scene graph generation aims to represent the image as a semantic structural graph, where objects and relationships between them are described as nodes and edges, respectively. Some existing methods rely only on visual features to sequentially predict the relationships between objects, ignoring contextual information and making it difficult to generate high-quality scene graphs, especially for remote sensing images. Therefore, we propose a novel model for remote sensing image scene graph generation by fusing contextual information and statistical knowledge, namely RSSGG_CS. To integrate contextual information and calculate attention among all objects, the RSSGG_CS model adopts a filter module (FiM) that is based on adjusted transformer architecture. Moreover, to reduce the blindness of the model when searching semantic space, statistical knowledge of relational predicates between objects from the training dataset and the cleaned Wikipedia text is used as supervision when training the model. Experiments show that fusing contextual information and statistical knowledge allows the model to generate more complete scene graphs of remote sensing images and facilitates the semantic understanding of remote sensing images.

In the field of aerial image object detection based on deep learning, it’s difficult to extract features because the images are obtained from a top-down perspective. Therefore, there are numerous false detection boxes. The existing post-processing methods mainly remove overlapped detection boxes, but it’s hard to eliminate false detection boxes. The proposed dual non-maximum suppression (dual-NMS) combines the density of detection boxes that are generated for each detected object with the corresponding classification confidence to autonomously remove the false detection boxes. With the dual-NMS as a post-processing method, the precision is greatly improved under the premise of keeping recall unchanged. In vehicle detection in aerial imagery (VEDAI) and dataset for object detection in aerial images (DOTA) datasets, the removal rate of false detection boxes is over 50%. Additionally, according to the characteristics of aerial images, the correlation calculation layer for feature channel separation and the dilated convolution guidance structure are proposed to enhance the feature extraction ability of the network, and these structures constitute the correlation network (CorrNet). Compared with you only look once (YOLOv3), the mean average precision (mAP) of the CorrNet for DOTA increased by 9.78%. Commingled with dual-NMS, the detection effect in aerial images is significantly improved.

Deep learning has contributed to the rapid development of building extraction tasks from remote sensing(RS) images. Existing models typically leverage a segmentation-head to predict results, where multi-channel feature maps extracted by the network are directly output as single-channel predictions. However, it is rarely noticed that this process results in a loss of features, which can lead to incomplete extraction of smaller buildings. Besides, boundary-blurring is also a common problem in the task. Therefore, in this letter, we propose a Siamese Prototype Contrastive Learning Network (SPCL-Net) to address these two problems. In the network, a novel Prototype Contrastive Learning (PCL) module is proposed to alleviate feature loss problem by applying contrastive learning between prototype vectors. In addition, a Reverse Boundary Enhancement (RBE) module is proposed to facilitate the representation of building boundaries and mitigate the boundary-blurring problem. Experiments are conducted on two datasets, INRIA and WHU. Compared with existing models, the final results show that the proposed approach is better than theirs in terms of evaluation metrics IOU.

View planning (VP) is a technique that guides the adjustment of the sensor’s postures in multi-view perception tasks. It converts the perception process into active perception, which improves the intelligence and reduces the resource consumption of the robot. We propose a generic VP system for multiple kinds of visual perception. The VP system is built on the basis of the formal description of the visual task, and the next best view is calculated by the system. When dealing with a given visual task, we can simply update its description as the input of the VP system, and obtain the defined best view in real time. Formal description of the perception task includes the task’s status, the objects’ prior information library, the visual representation status and the optimization goal. The task’s status and the visual representation status are updated when data are received at a new view. If the task’s status has not reached its goal, candidate views are sorted based on the updated visual representation status, and the next best view that can minimize the entropy of the model space is chosen as the output of the VP system. Experiments of view planning for 3D recognition and reconstruction tasks are conducted, and the result shows that our algorithm has good performance on different tasks.