Due to the continuous booming of surveillance and Web videos, video moment localization, as an important branch of video content analysis, has attracted wide attention from both industry and academia in recent years. It is, however, a non-trivial task due to the following challenges: temporal context modeling, intelligent moment candidate generation, as well as the necessary efficiency and scalability in practice. To address these impediments, we present a deep end-to-end cross-modal hashing network. To be specific, we first design a video encoder relying on a bidirectional temporal convolutional network to simultaneously generate moment candidates and learn their representations. Considering that the video encoder characterizes temporal contextual structures at multiple scales of time windows, we can thus obtain enhanced moment representations. As a counterpart, we design an independent query encoder towards user intention understanding. Thereafter, a cross-model hashing module is developed to project these two heterogeneous representations into a shared isomorphic Hamming space for compact hash code learning. After that, we can effectively estimate the relevance score of each “ moment-query ” pair via the Hamming distance. Besides effectiveness, our model is far more efficient and scalable since the hash codes of videos can be learned offline. Experimental results on real-world datasets have justified the superiority of our model over several state-of-the-art competitors.

In recent years, the task of salient object detection in optical remote sensing images (RSI-SOD) has received extensive attention. Benefiting from the development of deep learning, much progress has been made in RSI-SOD field. However, existing methods still face challenges in addressing various issues present in optical RSI, including uncertain numbers of salient objects, cluttered backgrounds, and interference from shadows. To address these challenges, we propose a novel approach, Adaptive Edge-aware Semantic Interaction Network (AESINet) for efficient salient object detection. Specifically, to improve the extraction of complex edge information, we design a Local Detail Aggregation Module (LDAM). This module can adaptively enhance the edge information of salient objects by leveraging our proposed difference perception mechanism. Notably, our difference perception mechanism is a novel edge enhancement method without the supervision of edge groundtruth. Additionally, to accurately locate salient objects of varying numbers and scales, we design a Multi-scale Feature Enhancement Module (MFEM), which effectively captures and utilizes multi-scale information. Moreover, we design the Deep Semantic Interaction Module (DSIM) to identify salient objects amidst cluttered backgrounds and effectively mitigate the interference of shadows. We conduct extensive experiments on three well-established optical RSI datasets and the results demonstrate that our proposed model outperforms 14 state-of-the-art methods. All codes and detection results are available at https://github.com/xumingzhu989/AESINet-TGRS.

Video moment localization, as an important branch of video content analysis, has attracted extensive attention in recent years. However, it is still in its infancy due to the following challenges: cross-modal semantic alignment and localization efficiency. To address these impediments, we present a cross-modal semantic alignment network. To be specific, we first design a video encoder to generate moment candidates, learn their representations, as well as model their semantic relevance. Meanwhile, we design a query encoder for diverse query intention understanding. Thereafter, we introduce a multi-granularity interaction module to deeply explore the semantic correlation between multi-modalities. Thereby, we can effectively complete target moment localization via sufficient cross-modal semantic understanding. Moreover, we introduce a semantic pruning strategy to reduce cross-modal retrieval overhead, improving localization efficiency. Experimental results on two benchmark datasets have justified the superiority of our model over several state-of-the-art competitors.

The InfraRed Small Target Detection (IRSTD) task involves identifying and separating small targets from complex backgrounds. However, these targets pose significant challenges due to their small, variable sizes and dim appearance with a low signal-to-noise ratio, often obscured by cluttered backgrounds. Standard spatial-domain Convolutional Neural Networks (CNNs) act as low-pass filters, hindering their ability to detect small, variably sized, low-contrast targets against complex backgrounds. Infrared images also exhibit diverse spectral energy distributions, yet CNNs lack a global spectral view to discern these patterns, making them susceptible to background clutter. To address these shortcomings, we propose a novel Hybrid Domain Network (HDNet), which fuses frequency-domain features with conventional spatial-domain CNN features to markedly enhance target-background contrast and explicitly suppress background interference. Specifically, HDNet comprises two main branches: the spatial domain branch and the frequency domain branch. In the spatial domain, we innovatively introduce a Multi-scale Atrous Contrast convolution (MAC) module, utilizing multiple parallel atrous contrast convolutions with varying kernel sizes to enhance perception of small, variably sized targets. In the frequency domain, we have specifically designed the Dynamic High-Pass Filter (DHPF) module, hierarchically calculating low-frequency signal energy and dynamically removing specific low-frequency information to preserve high-frequency image details. This effectively filters out slowly varying low-frequency backgrounds, highlighting small targets. Comprehensive ablation studies and experimental analysis on three datasets (IRSTD-1K, NUAA-SIRST, NUDT-SIRST) validate HD-Net’s effectiveness and superiority compared to 26 state-of-the-art (SOTA) methods. The source code is available at: https://github.com/xumingzhu989/HDNet-TGRS.

In recent years, the task of Salient Object Detection in optical Remote Sensing Images (RSI-SOD) has gained increasing interest. Despite some advancements in current methods, challenges such as the irregular topology of salient objects and cluttered backgrounds in optical RSI are remaining. To tackle these issues, we propose a novel Heterogeneous Feature Collaboration Network (HFCNet). Specifically, we design a new hybrid heterogeneous encoder that combines CNN and Transformer to extract a set of heterogeneous features, famous in modeling local and global information, respectively. Subsequently, the Adaptive Global-Local Integration module (AGLI) is devised to integrate these complementary heterogeneous features through our feature alignment methods at global and local level, so the global irregular topology structure and local details can be well modeled. Furthermore, the proposed Sailency-Guided Attention Enhanced Decoder (SGAED) leverages deep salient cues to guide the shallow decoders to pay more attention to important areas and suppress irrelevant areas, reducing the interference of cluttered backgrounds. Extensive experiments on three benchmark datasets have confirmed the significant superiority of our method compared to 18 state-of-the-art methods. All codes and results of our method are available at https://github.com/xumingzhu989/HFCNet-TGRS.