Aerial visible-to-infrared image translation aims to transfer aerial visible images to their corresponding infrared images, which can effectively generate the infrared images of specific targets. Although some image-to-image translation algorithms have been applied to color-to-thermal natural images and achieved impressive results, they cannot be directly applied to aerial visible-to-infrared image translation due to the substantial differences between natural images and aerial images, including shooting angles, multi-scale targets, and complicated backgrounds. In order to verify the performance of existing image-to-image translation algorithms on aerial scenes as well as advance the development of aerial visible-to-infrared image translation, an Aerial Visible-to-Infrared Image Dataset (AVIID) is created, which is the first specialized dataset for aerial visible-to-infrared image translation and consists of over 3,000 paired visible-infrared images. Over the constructed AVIID, a complete evaluation system is presented to evaluate the generated infrared images from 2 aspects: overall appearance and target quality. In addition, a comprehensive survey of existing image-to-image translation approaches that could be applied to aerial visible-to-infrared image translation is given. We then provide a performance analysis of a set of representative methods under our proposed evaluation system on AVIID, which can serve as baseline results for future work. Finally, we summarize some meaningful conclusions, problems of existing methods, and future research directions to advance state-of-the-art algorithms for aerial visible-to-infrared image translation.
5 July 2024 in IEEE Transactions on Geoscience and Remote Sensing
Object detection in remote sensing images identifies and extracts the acquired earth surface information, providing data support and research basis for multiple fields. Remote sensing image object detection based on knowledge distillation (KD) can transfer the knowledge of a large teacher model to a smaller student model, achieving the effect of low parameter volume and high accuracy. Mainstream methods directly imitate teacher features to improve student performance, ignoring the generation of high-ranking features through teacher features instructing student feature maps in this knowledge transfer process. In this paper, an Adaptive Composite Feature Generation (ACFG) strategy is proposed to achieve end-to-end trainable knowledge distillation for object detection in remote sensing images, in which the robustness of feature points under composite masks is improved through adaptive feature mapping. Specially, a composite mask generator (CMG) module is proposed to select student instance-related features and point background features. Furthermore, a global and local projection layer (GLPL) module is proposed to connect the local information and global information of the feature map under the mask generator to adaptively realize the global recovery mapping of the feature map with partial feature points. Finally, balanced decoupling loss (BDL) is improved to handle foreground and background loss separately, so that the two decoupled features can better enable the student model to learn instance-related information. Note that the proposed ACFG is capable of conducting knowledge distillation for both single-stage and two-stage object detectors. Experimental results using both anchor-based and anchor-free detectors on the DIOR dataset and DOTA dataset demonstrate that the proposed ACFG clearly achieved better performance than several state-of-the-art algorithms for knowledge distillation.