Multispectral image pairs can provide combined information, making object detection applications more reliable and robust in the open world. To fully exploit the different modalities, a simple yet effective cross-modality feature fusion approach, named CrossModality Fusion Transformer (CFT) is presented in this paper. Unlike prior CNNs-based works, our network learns long-range dependencies and integrates global contextual information in the feature extraction stage under the guidance of the Transformer scheme. More importantly, by leveraging self attention of the Transformer, the network can naturally execute intra-modality and inter-modality fusion simultaneously, and robustly capture the latent interactions between RGB and thermal domains. Thereby the performance of multispectral object detection is improved significantly. Extensive experiments and ablation studies on multiple datasets demonstrate that the proposed scheme is effective and obtains state-of-the-art detection performance. Our code and models are available at https://github.com/DocF/multispectral-object-detection .

CubeSats provide a low-cost, convenient, and effective way of acquiring remote sensing data, and have great potential for remote sensing object detection. Although deep learning-based models have achieved excellent performance in object detection, they suffer from the problem of numerous parameters, making them difficult to deploy on CubeSats with limited memory and computational power. Existing approaches attempt to prune redundant parameters, but this inevitably causes a degradation in detection accuracy. In this paper, the novel Context-aware Dense Feature Distillation (CDFD) is proposed, guiding a small student network to integrate features extracted from multi-teacher networks to train a lightweight and superior detector for onboard remote sensing object detection. Specifically, a Contextual Feature Generation Module (CFGM) is designed to rebuild the non-local relationships between different pixels and transfer them from teacher to student, thus guiding students to extract rich contextual features to assist in remote sensing object detection. In addition, an Adaptive Dense Multi-teacher Distillation (ADMD) strategy is proposed, which performs adaptive weighted loss fusion of students with multiple well-trained teachers, guiding students to integrate the learning of helpful knowledge from multiple teachers. Extensive experiments were conducted on two large-scale remote sensing object detection datasets with various network structures; the results demonstrate that the trained lightweight network achieves auspicious performance. Our approach also shows good generality for existing state-of-the-art remote sensing object detectors. Furthermore, by experimenting on large general object datasets, we demonstrate that our approach is equally practical for general object detection distillation.

Scale diversity, small target, and power limitation have made remote sensing imagery a challenging field in object detection on satellites. Aiming at the aspects of scale diversity and small target, this paper provides a novel feature pyramid network with Adaptive Residual Spatial Bi-Fusion (ARSF) as a solution. ARSF nets introduce a robust fusion of multi-scale semantic information and fine spatial details. A spatial feature fusion module designed in networks with ARSF adapts to object size variation by learning the most crucial feature maps. Comparing to the original feature pyramid network, a shorter critical path for information transmission is formed in our method. Experiments show that a validation instance of YOLOv3-ARSF can achieve a state-of-the-art performance of 85.8 mAP on the NWPU-VHR10 dataset. YOLOv3-ARSF only 3MB larger than YOLOv3 but far exceeds YOLOv3 by 2.3% mAP, which shows our ARSF is efficient. As for the last challenge, two lightweight versions, ARSF(lite) and ARSF(lite+) are also validated for future research of online object detection on satellites in aerospace engineering. Visualizations and details are provided for a more comprehensive understanding.

Multispectral image pairs can provide the combined information, making object detection applications more reliable and robust in the open world. To fully exploit the different modalities, we present a simple yet effective cross-modality feature fusion approach, named Cross-Modality Fusion Transformer (CFT) in this paper. Unlike prior CNNs-based works, guided by the transformer scheme, our network learns long-range dependencies and integrates global contextual information in the feature extraction stage. More importantly, by leveraging the self attention of the transformer, the network can naturally carry out simultaneous intra-modality and inter-modality fusion, and robustly capture the latent interactions between RGB and Thermal domains, thereby significantly improving the performance of multispectral object detection. Extensive experiments and ablation studies on multiple datasets demonstrate that our approach is effective and achieves state-of-the-art detection performance. Our code and models are available at https://github.com/DocF/multispectral-object-detection.

In recent years, increasing attention has been paid to object detection in remote sensing imagery. However, traditional optical detection is highly susceptible to illumination and weather anomalies. It is a challenge to effectively utilize the cross-modality information from multisource remote sensing images, especially from optical and synthetic aperture radar images, to achieve all-day and all-weather detection with high accuracy and speed. Toward this end, a fast multisource fusion detection framework FusionDet is proposed in the current paper. A novel distance-decay intersection over union is employed to encode the shape properties of the targets with scale invariance. Therefore, the same target in multisource images can be paired accurately. Furthermore, the weighted Dempster–Shafer evidence theory is utilized to combine the optical and synthetic aperture radar detection, which overcomes the drawback in a feature-level fusion that requires a large amount of paired data. In addition, the paired optical and synthetic aperture radar images are taken to demonstrate our fusion algorithm. Considerable experiments and ablation studies on multiple datasets show that our method is generally effective and achieves state-of-the-art detection performance.

Remote sensing object detection is a critical task in Earth observation. Despite the remarkable progress made in general object detection, existing detectors struggle with remote sensing scenarios due to the prevalence of numerous small objects with limited discriminative cues. Cutting-edge studies have shown that incorporating contextual information effectively enhances the detection performance for small objects. Meanwhile, recent research has revealed that convolution in the frequency domain is capable of capturing long-range spatial dependencies with high efficiency. Inspired by this, we propose a Frequency-aware Feature Pyramid Framework (FFPF) for remote sensing object detection, which consists of a novel Frequency-aware ResNet (F-ResNet) and a Bilateral Spectral-aware Feature Pyramid Network (BS-FPN). Specifically, the F-ResNet is proposed to extract the spectral context information by plugging the frequency domain convolution into each stage of the backbone, thereby enriching features of small objects. In addition, the BS-FPN employs a bilateral sampling strategy and skipping connection to model the association of object features at different scales, enabling the contextual information extracted by the F-ResNet to be fully leveraged. Extensive experiments are conducted for object detection in the public remote sensing image dataset and natural image dataset. The experimental results demonstrate the excellent performance of the FFPF, achieving 73.8% mAP on the DIOR dataset without using any additional training tricks.