In recent years, the task of salient object detection in optical remote sensing images (RSI-SOD) has received extensive attention. Benefiting from the development of deep learning, much progress has been made in RSI-SOD field. However, existing methods still face challenges in addressing various issues present in optical RSI, including uncertain numbers of salient objects, cluttered backgrounds, and interference from shadows. To address these challenges, we propose a novel approach, Adaptive Edge-aware Semantic Interaction Network (AESINet) for efficient salient object detection. Specifically, to improve the extraction of complex edge information, we design a Local Detail Aggregation Module (LDAM). This module can adaptively enhance the edge information of salient objects by leveraging our proposed difference perception mechanism. Notably, our difference perception mechanism is a novel edge enhancement method without the supervision of edge groundtruth. Additionally, to accurately locate salient objects of varying numbers and scales, we design a Multi-scale Feature Enhancement Module (MFEM), which effectively captures and utilizes multi-scale information. Moreover, we design the Deep Semantic Interaction Module (DSIM) to identify salient objects amidst cluttered backgrounds and effectively mitigate the interference of shadows. We conduct extensive experiments on three well-established optical RSI datasets and the results demonstrate that our proposed model outperforms 14 state-of-the-art methods. All codes and detection results are available at https://github.com/xumingzhu989/AESINet-TGRS.

Recent advances in deep convolution neural networks (CNNs) boost the development of video salient object detection (SOD), and many remarkable deep-CNNs video SOD models have been proposed. However, many existing deep-CNNs video SOD models still suffer from coarse boundaries of the salient object, which may be attributed to the loss of high-frequency information. The traditional graph-based video SOD models can preserve object boundaries well by conducting superpixels/supervoxels segmentation in advance, but they perform weaker in highlighting the whole object than the latest deep-CNNs models, limited by heuristic graph clustering algorithms. To tackle this problem, we find a new way to address this issue under the framework of graph convolution networks (GCNs), taking advantage of graph model and deep neural network. Specifically, a superpixel-level spatiotemporal graph is first constructed among multiple frame-pairs by exploiting the motion cues implied in the frame-pairs. Then the graph data is imported into the devised multi-stream attention-aware GCN, where a novel Edge-Gated graph convolution (GC) operation is proposed to boost the saliency information aggregation on the graph data. A novel attention module is designed to encode the spatiotemporal sematic information via adaptive selection of graph nodes and fusion of the static-specific and the motion-specific graph embedding. Finally, a smoothness-aware regularization term is proposed to enhance the uniformity of salient object. Graph nodes (superpixels) inherently belonging to the same class will be ideally clustered together in the learned embedding space. Extensive experiments have been conducted on three widely used datasets. Compared with fourteen state-of-the-art video SOD models, our proposed method can well retain the salient object boundaries and possess a strong learning ability, which shows that this work is a good practice for designing GCNs for video SOD.

The InfraRed Small Target Detection (IRSTD) task involves identifying and separating small targets from complex backgrounds. However, these targets pose significant challenges due to their small, variable sizes and dim appearance with a low signal-to-noise ratio, often obscured by cluttered backgrounds. Standard spatial-domain Convolutional Neural Networks (CNNs) act as low-pass filters, hindering their ability to detect small, variably sized, low-contrast targets against complex backgrounds. Infrared images also exhibit diverse spectral energy distributions, yet CNNs lack a global spectral view to discern these patterns, making them susceptible to background clutter. To address these shortcomings, we propose a novel Hybrid Domain Network (HDNet), which fuses frequency-domain features with conventional spatial-domain CNN features to markedly enhance target-background contrast and explicitly suppress background interference. Specifically, HDNet comprises two main branches: the spatial domain branch and the frequency domain branch. In the spatial domain, we innovatively introduce a Multi-scale Atrous Contrast convolution (MAC) module, utilizing multiple parallel atrous contrast convolutions with varying kernel sizes to enhance perception of small, variably sized targets. In the frequency domain, we have specifically designed the Dynamic High-Pass Filter (DHPF) module, hierarchically calculating low-frequency signal energy and dynamically removing specific low-frequency information to preserve high-frequency image details. This effectively filters out slowly varying low-frequency backgrounds, highlighting small targets. Comprehensive ablation studies and experimental analysis on three datasets (IRSTD-1K, NUAA-SIRST, NUDT-SIRST) validate HD-Net’s effectiveness and superiority compared to 26 state-of-the-art (SOTA) methods. The source code is available at: https://github.com/xumingzhu989/HDNet-TGRS.

Convolutional neural networks (CNN)-based salient object detection (SOD) models have achieved promising performance in optical remote sensing images (ORSIs) in recent years. However, the restriction concerning the local sliding window operation of CNN has caused many existing CNN-based ORSI SOD models to still struggle with learning long-range relationships. To this end, a novel transformer framework is proposed for ORSI SOD which is inspired by the powerful global dependency relationships of transformer networks. This is the first attempt to explore global and local details using transformer architecture for SOD in ORSIs. Concretely, we design an adaptive spatial tokenization transformer encoder to extract global-local features, which can accurately sparsify tokens for each input image and achieve competitive performance in ORSI SOD tasks. Then, a specific dense token aggregation decoder (DTAD) is proposed to generate saliency results, including three cascade decoders to integrate the global-local tokens and contextual dependencies. Extensive experiments indicate that the proposed model greatly surpasses twenty state-of-the-art (SOTA) SOD approaches on two standard ORSI SOD datasets under seven evaluation metrics. We also report a comparison results to demonstrate the generalization capacity on the latest challenging ORSI datasets. In addition, we validate the contributions of different modules through a series of ablation analyses, especially the proposed adaptive spatial tokenization module (ASTM), which can halve the computational budget.

We present a novel, robust estimation method to distinguish salient objects from complicated, dynamic backgrounds in videos. In this method, we propose a novel approach to model motion energy based on motion magnitude, motion orientation, gradient flow field, and spatial gradient of the video frame. Furthermore, an effective spatiotemporal objectness map is also proposed to estimate a compact object-like region in the current video frame leveraging both the objectness proposals and the saliency map of the previous frame. Then the current video frame is oversegmented into the granularity of superpixels using the simple linear iterative clustering algorithm. Each superpixel is designated as a node of a graph. The similarity between adjacent superpixels will be assigned as the weight of an edge that connects these two nodes. The feature values of motion energy and spatiotemporal objectness within each superpixel will be averaged respectively, and used to graphically cluster similar superpixels to form the detected salient object. Extensive experiments comparing this proposed new method against twelve existing salient object detection (SOD) methods have been performed using the benchmark datasets unconstrained video saliency detection and densely annotated video segmentation. Superior performance of this proposed SOD method has been observed through three well-known performance metrics: precision-recall curves, F-measure curves, and the mean absolute error.

In recent years, the task of Salient Object Detection in optical Remote Sensing Images (RSI-SOD) has gained increasing interest. Despite some advancements in current methods, challenges such as the irregular topology of salient objects and cluttered backgrounds in optical RSI are remaining. To tackle these issues, we propose a novel Heterogeneous Feature Collaboration Network (HFCNet). Specifically, we design a new hybrid heterogeneous encoder that combines CNN and Transformer to extract a set of heterogeneous features, famous in modeling local and global information, respectively. Subsequently, the Adaptive Global-Local Integration module (AGLI) is devised to integrate these complementary heterogeneous features through our feature alignment methods at global and local level, so the global irregular topology structure and local details can be well modeled. Furthermore, the proposed Sailency-Guided Attention Enhanced Decoder (SGAED) leverages deep salient cues to guide the shallow decoders to pay more attention to important areas and suppress irrelevant areas, reducing the interference of cluttered backgrounds. Extensive experiments on three benchmark datasets have confirmed the significant superiority of our method compared to 18 state-of-the-art methods. All codes and results of our method are available at https://github.com/xumingzhu989/HFCNet-TGRS.