Image deblurring aims to restore the detailed texture information or structures from the blurry images, which has become an indispensable step in many computer vision tasks. Although various methods have been proposed to deal with the image deblurring problem, most of them treated the blurry image as a whole and neglected the characteristics of different image frequencies. In this paper, we present a new method called multi-scale frequency separation network (MSFS-Net) for image deblurring. MSFS-Net introduces the frequency separation module (FSM) into an encoder-decoder network architecture to capture the low- and high-frequency information of image at multiple scales. Then, a simple cycle-consistency strategy and a sophisticated contrastive learning module (CLM) are respectively designed to retain the low-frequency information and recover the high-frequency information during deblurring. At last, the features of different scales are fused by a cross-scale feature fusion module (CSFFM). Extensive experiments on benchmark datasets show that the proposed network achieves state-of-the-art performance.
20 February 2024 in IEEE Transactions on Multimedia
Temporal action localization (TAL), which aims to identify and localize actions in long untrimmed videos, is a challenging task in video understanding. Recent studies have shown that the Transformer and its variants are effective at improving the performance of TAL. The success of the Transformer can be attributed to the use of multi-head self-attention (MHSA) as a token mixer to capture long-term temporal dependencies within the video sequence. However, in the existing Transformer architecture, the features obtained by multiple token mixing (i.e., self-attention) heads are treated equally, which neglects the distinct characteristics of different heads and hampers the exploitation of discriminative information. To this end, we present a new method called the adaptive dual selective Transformer (ADSFormer) for TAL in this paper. The key component in ADSFormer is the dual selective multihead token mixer (DSMHTM), which integrates multiple feature representations from different token mixing heads by adaptively selecting important features across both the head and channel dimensions. Moreover, we also incorporate our ADSFormer into a pyramid structure so that the multi-scale features obtained can be effectively combined to improve TAL performance. Benefiting from the dual selective multi-head token mixer (DSMHTM) and pyramid feature combination, ADSFormer outperforms several state-of-the-art methods on four challenging benchmark datasets: THUMOS14, MultiTHUMOS, EPIC-KITCHENS-100 and ActivityNet-1.3. We publicly released our codes and pretrained models at: https://github.com/LiQiang0307/ADSFormer.
Object detection has been playing a significant role in computer vision for a long time, but it is still full of challenges. In this paper, we propose a novel object detection framework based on relationship among different objects and the scene-level information of the whole image to cope with the problem that some strongly correlated objects are difficult to be recognized. Our motivation is to enrich the semantics of object detection feature by a scene-level information branch and a relationship branch. There are three important changes of our framework over traditional detection methods: representation of relationship, scene-level information as the prior knowledge and the fusion of the above two information. Extensive experiments are carried out on PASCAL VOC and MS COCO databases. The experimental results show that the detection performance can be improved by introducing relationship and scene-level information, and our proposed model achieve better performance than several classical and state-of-the-art methods.
Recently, deep learning-based image deblurring and deraining have been well developed. However, most of these methods fail to distill the useful features. What is more, exploiting the detailed image features in a deep learning framework always requires a mass of parameters, which inevitably makes the network suffer from a high computational burden. We propose a lightweight fusion distillation network (LFDN) for image deblurring and deraining to solve the above problems. The proposed LFDN is designed as an encoder–decoder architecture. In the encoding stage, the image feature is reduced to various small-scale spaces for multi-scale information extraction and fusion without much information loss. Then, a feature distillation normalization block is designed at the beginning of the decoding stage, which enables the network to distill and screen valuable channel information of feature maps continuously. Besides, an information fusion strategy between distillation modules and feature channels is also carried out by the attention mechanism. By fusing different information in the proposed approach, our network can achieve state-of-the-art image deblurring and deraining results with a smaller number of parameters and outperform the existing methods in model complexity.
21 August 2025 in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
With the rapid advancement of deep learning techniques, CNN-based change detection (CD) methods have achieved remarkable progress in remote sensing image analysis, primarily attributed to their exceptional ability to extract localized spatial features. However, these methods remain constrained by two fundamental limitations: (1) restricted receptive field that hinder comprehensive contextual understanding, and (2) inadequate capacity for capturing long-range global dependencies. To address these challenges, we propose a Deep Supervised Change Detection Network based on Context-Rich Information (DSCRNet) featuring three innovative components: a CNN TransConv Block (CTCB), a Hybrid Dual Attention Block (HDAB), and a Multiscale Supervised Module (MSM). The CTCB module synergistically integrates lightweight convolutional operations with Transformer architectures through a novel dual-branch framework. This hybrid design not only enhances CNN's ability in local feature extraction but also leverages Transformer's self-attention mechanism to effectively model global contextual relationships. The HDAB module enhances discriminative feature learning through two key innovations: 1) spatial-channel transformation operations enabling cross-scale feature interaction, and 2) a parallel attention mechanism combining channel-wise and spatial attention to produce more discriminative feature representations. The MSM introduces a parameter-free multiscale supervision mechanism that enables progressive refinement of change features through hierarchical feature fusion while maintaining computational efficiency. The experimental results and ablation studies conducted on the LEVIR-CD, CDD and DSIFN-CD datasets indicate that the proposed DSCRNet achieves superior performance than the existing methods, attributed to the integration of the CNN TransConv Block and other key modules. The code has been released at: https://github.com/joyeuxni/DSCRNet.