Recent studies in remote sensing object detection have made excellent progress and shown promising performance. However, most current detectors only explore rotation-invariant feature extraction but disregard the valuable spatial and semantic prior knowledge in remote sensing images (RSIs), which limits the detection performance when encountering blurred or heavy occluded objects. To address this issue, we propose a mask-reconstruction relation learning (MRRL) framework to learn such prior knowledge among objects and a consistency-reasoning transformer over relation proposals (CTRP) to recognize objects with limited visual features via consistency reasoning. Specifically, MRRL framework applies random mask to some objects in the training dataset and performs masked objects reconstruction to guide the network to learn the distribution consistency of objects. CTRP is the core component of the MRRL framework, which models the interaction between spatial and semantic priors, and uses easy detected objects to reason hard detected objects. The trained CTRP can be integrated into the existing detector to improve the ability of object detection with limited visual features in RSIs. Extensive experiments on widely-used datasets for two distinct tasks, namely remote sensing object detection task and occluded object detection task, demonstrate the effectiveness of the proposed method. Source code is available at https://github.com/sunpeng96/CTRP_mmrotate.

Oriented object detection has attained remarkable progress in addressing the challenges associated with rotating invariant feature extraction. However, most existing object detection most existing encounter serious performance degradation when processing objects with tiny size, blurring and occlusion. One of the reasons is that existing methods mainly focus on local features, while overlooking the correlation among objects and common sense knowledge, which is inconsistent with the human visual system. To address this issue, we propose a dual-stream global relationship learning (DGRL) method consisting of two modules: a dynamic correlation learning (DCL) method and a global knowledge mapping (GKM) module. The DCL method can construct a region-to-region dynamic relationship graph based on feature correlations and implicitly guides the detection network to learn more powerful class representation by updating nodes of the graph. The GKM module generates a class-to-class global semantic relationship graph via external knowledge and achieve more stable representation learning by dynamically global relational mapping. Extensive experiments are performed on oriented object detection datasets DOTA, HRSC2016, DIOR-R as well as horizontal object detection datasets NWPU VHR-10, RSOD. The results demonstrate that the proposed method achieves the state-of-the-art detection accuracy.

Arbitrary-oriented object detection in remote sensing optical images is an emerging yet challenging task due to the diversity of the visual appearance of objects. Although considerable progress has been made, existing methods often suffer from insufficient positive samples and critical feature misalignment caused by objects with considerable diversity. In this paper, we propose a critical feature learning (CFL) method to address the above issues. First, to obtain a more accurate location, we introduce the spatial transform to model the rotating bounding box regression and construct a spatial transform selection (STS) strategy, which is used to implement dynamic sample selection metrics to guarantee sufficient positive samples. Then, we propose a scale-controlled smooth L ₁ loss (SC-Loss) to change the form of regression loss function based on the statistics of proposals so that the training can focus more on high-quality samples. Finally, to capture critical features, we propose a critical feature sampling (CFS) module to perform location refinement of classification feature samples to extract accurate features. Comprehensive and extensive experimental results based on three benchmarks, DOTA, FAIR1M-1.0, and HRSC2016, demonstrate that the proposed method is superior to existing methods in terms of accuracy and effectiveness.

Image fusion is the process of combining multiple input images from single or multiple imaging modalities into a fused image, which is expected to be more informative for human or machine perception as compared to any of the input images. In this paper, we propose a novel method based on deep learning for fusing infrared images and visible images, named the local binary pattern (LBP)-based proportional input generative adversarial network (LPGAN). In the image fusion task, the preservation of structural similarity and image gradient information is contradictory, and it is difficult for both to achieve good performance at the same time. To solve this problem, we innovatively introduce LBP into GANs, enabling the network to have stronger texture feature extraction and utilization capabilities, as well as anti-interference capabilities. In the feature extraction stage, we introduce a pseudo-Siamese network for the generator to extract the detailed features and the contrast features. At the same time, considering the characteristic distribution of different modal images, we propose a 1:4 scale input mode. Extensive experiments on the publicly available TNO dataset and CVC14 dataset show that the proposed method achieves the state-of-the-art performance. We also test the universality of LPGAN by fusing RGB and infrared images on the RoadScene dataset and medical images. In addition, LPGAN is applied to multi-spectral remote sensing image fusion. Both qualitative and quantitative experiments demonstrate that our LPGAN can not only achieve good structural similarity, but also retain richly detailed information.

Low, small, and slow (LSS) unmanned aerial vehicles (UAVs) pose great challenges for conventional guidance methods. However, existing deep reinforcement learning (DRL)-based interception guidance law has mostly focused on simplified two-dimensional planes and requires strict initial launch scenarios (constructing collision triangles). Designing more robust guidance laws has therefore become a key research focus. In this paper, we propose a novel recurrent proximal policy optimization (RPPO)-based guidance law framework. Specifically, we first design initial launch conditions in three-dimensional space that are more applicable and realistic, without requiring to form a collision triangle at the initial launch. Then, considering the temporal continuity of the seeker’s observations, we introduce the long short-term memory (LSTM) networks into the proximal policy optimization (PPO) algorithm to extract hidden temporal information from the observation sequences, thus supporting the policy training. Finally, we propose a reward function based on velocity prediction and overload constraints. Simulation experiments show that the proposed RPPO framework achieves an interception rate of 95.3% and a miss distance of 1.2935 m under broader launch conditions. Moreover, the framework demonstrates strong generalization ability, effectively coping with unknown maneuvers of UAVs.

Image fusion is the process of combining multiple input images from single or multiple imaging modalities into a fused image, which is expected to be more informative for human or machine perception as compared to any of the input images. In this paper, we propose a novel method based on deep learning for fusing infrared images and visible images, named the LBP-based proportional input generative adversarial network (LPGAN). In the image fusion task, the preservation of structural similarity and image gradient information is contradictory, and it is difficult for both to achieve good performance at the same time. To solve this problem, we innovatively introduce Local Binary Patterns (LBP) into Generative Adversarial Networks (GANs), which effectively utilize the texture features of the source images, so that the network has stronger feature extraction ability and anti-interference ability. In the feature extraction stage, we introduce a pseudo-siamese network for the generator to extract the detailed features and the contrast features. At the same time, considering the characteristic distribution of different modal images, we propose a 1:4 scale input mode. Extensive experiments on the publicly available TNO dataset and CVC14 dataset show that the proposed method achieves the state-of-the-art performance. We also test the universality of LPGAN through the fusion of RGB and infrared images on the RoadScene dataset. In addition, LPGAN is applied to multi-spectral remote sensing image fusion. Both qualitative and quantitative experiments demonstrate that our LPGAN can not only achieve good structural similarity, but also retain rich detailed information.