Recently, deep learning in synthetic aperture radar (SAR) automatic target recognition (ATR) has made significant progress, but the sample limitation problem in the SAR field is still obvious. Compared with the optical remote sensing images, the SAR images are insufficient, especially those containing the geospatial targets with certain target attitude angles (TAAs). To solve these problems, a novel few-shot learning framework named scattering characteristics analysis network (SCAN) is proposed in this article. First, a scattering extraction module (SEM) is designed to combine the target imaging mechanism with the network, which learns the number and distribution of the scattering points for each target type via explicit supervision. Besides, considering the imaging variability of SAR targets, a TAA-guided metalearning network consisting of an angle self-adaption classifier (ASC) and a frequency embedded module (FEM) is designed. ASC guides the network to focus on the positive sample pairs with different TAAs. FEM combines pulse cosine transform (PCT) with the network training process effectively to enrich frequency-domain information. In addition, a new dataset named SAR aircraft category dataset is constructed for the experiments. Compared with other few-shot SAR target classification approaches, our model efficiently integrates the scattering characteristics with the learning process, and the test accuracy for 5-way 1-shot has been improved by 4.74%. Finally, the experimental results are provided to demonstrate the validity of the proposed method.
18 July 2023 in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
Intelligent processing of satellite video focuses on extracting specific information of ground objects and scenes from Earth observation videos through intelligent image/video processing technology, which has important applications in fields such as traffic monitoring, resource and environmental monitoring. The integration of deep learning technology in satellite video processing has led to significant advancements in tasks like object detection and object tracking, expanding into emerging research areas such as satellite video scene classification, object segmentation. However, there is no comprehensive review and summary in the intelligent processing of satellite video. This paper presents a systematic review and quantitative analysis of the results published over the last decade, intending to further promote the development of various intelligent processing tasks for satellite video. It analyzes the current difficulties, challenges and the methodological system for each task. Additionally, it provides an in-depth analysis and summary of publicly available datasets and evaluation benchmarks for each task, as well as classic algorithm performance and application scenarios. Finally, this paper summarizes the current research status and looks forward to the future development trend, hoping to inspire researchers in related fields and jointly promote the development of intelligent processing of satellite video.
16 October 2023 in IEEE Geoscience and Remote Sensing Letters
Synthetic aperture radar (SAR) image ship detection is a popular area of ocean remote sensing, which has broad application prospects in ocean monitoring, maritime rescue and other tasks. Recently, deep learning has been used in this field, but convolutional neural network (CNN) based SAR ship detection still faces some challenges. First, due to the characteristic of CNN’s local convolution, the global information of the ship is not sufficiently learned and the detection is vulnerable to complex background interference. Second, SAR ship imaging varies greatly under different imaging conditions and postures, so CNN is difficult to adapt to scattering change imaging. To solve these problems, we propose a Scattering Information Fusion Network (SIFNet) for oriented ship detection in SAR Images consisting of a multi-scale contextual semantic information fusion (MCSIF) module and a scattering points information learning (SPIL) module. The MCSIF module enhances the acquisition of global information, enabling the network to extract more efficient feature maps. The SPIL module takes advantage of the fact that scattering points can stably represent the key features of the ship under different imaging conditions to make detection more robust through scattering information learning. Experiments show that our method achieves the highest F1-score and AP50 on both HRSID and RSDD-SAR datasets.
8 April 2024 in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
Video satellites provide dynamic real-time monitoring of hotspot areas and objects by continuously imaging objects within a specified sequence, providing dynamic information over large areas. Satellite video object segmentation aims to separate foreground and background continuously. Compared with satellite video single-object segmentation, satellite video multiobject segmentation can separate multiple foreground objects of interest in the scene simultaneously. So far, the research on multiobject segmentation mainly focuses on extracting deeper informative features. However, this does not cope well with the problems of weak object texture information, foreground-background disproportion, and small object scale in satellite videos. Therefore, this article proposes an end-to-end semisupervised segmentation network named background interference suppression strategy for satellite video multiobject segmentation, which is the first research about satellite video multiobject segmentation. First, spatial and temporal matching templates integrate spatio-temporal information to enhance texture features within a sequence. Second, a novel matching strategy is also proposed to suppress complex background interference in satellite video. Finally, both intuitive and quantitative experimental results indicate that our proposed method outperforms other multiobject segmentation models.
Maritime ship detection is a critical task for security and traffic management. To advance research in this area, we constructed a new high-resolution, spatially aligned optical-SAR dataset, named MOS-Ship. Building on this, we propose MOS-DETR, a novel query-based framework. This model incorporates an innovative multi-modal Swin Transformer backbone to extract unified feature pyramids from both RGB and SAR images. This design allows the model to jointly exploit optical textures and SAR scattering signatures for precise, oriented bounding box prediction. We also introduce an adaptive probabilistic fusion mechanism. This post-processing module dynamically integrates the detection results generated by our model from the optical and SAR inputs, synergistically combining their complementary strengths. Experiments validate that MOS-DETR achieves highly competitive accuracy and significantly outperforms unimodal baselines, demonstrating superior robustness across diverse conditions. This work provides a robust framework and methodology for advancing multimodal maritime surveillance.
In space environments, microgravity, high radiation, and weak magnetic fields induce behavioral alterations in animals, resulting in erratic movement patterns that complicate tracking. These challenges impede accurate behavioral analysis, especially in multi-object scenarios. To address this issue, this study proposes a deep learning-based multi-object tracking (MOT) framework specifically designed for space animals. The proposed method decouples appearance and motion features through dual-stream inputs and employs modality-specific encoders (MSEs), which are fused via a heterogeneous graph network to model cross-modal spatio-temporal relationships. Additionally, an object re-detection module is integrated to maintain identity continuity during occlusions or rapid movements. This approach is validated using public datasets of space-observed Drosophila and zebrafish, with experimental results demonstrating superior performance compared with existing tracking methods. This work highlights the potential of artificial intelligence as a valuable tool in behavioral studies, enabling reliable animal tracking and analysis under extreme space conditions and supporting future research in space life sciences.