25 May 2022 in IEEE Transactions on Instrumentation and Measurement
Due to the discrepancy between training and test data, autoencoder-based image fusion methods trained by natural images easily lose vital information of infrared and visible images. Generative adversarial network (GAN)-based methods usually conduct their adversarial learning in image domain, which are difficult to optimize and ultimately affect the image fusion performance. To address these problems, this article proposes an autoencoder-based network, extraction-and-reconstruction network (ERNet), to fuse infrared and visible images. To weaken the discrepancy problem, ERNet’s encoder is trained using infrared and visible images. To stably train the encoder, we conduct adversarial learning in the feature domain of infrared and visible images, which makes the well-trained encoder that could efficiently extract vital features from them. ERNet’s decoder is trained using natural images with a supervised mode, which could reconstruct the final fused image from the vital features extracted by the encoder. The encoder and decoder are alternately trained to further weaken the discrepancy problem. The experimental results show that ERNet can effectively extract vital features of infrared and visible images and obtain higher quality fused images. Our fusion results can be found at https://github.com/suweijian1996/ERNet .
19 July 2024 in IEEE Transactions on Mobile Computing
Federated learning (FL) offers a credible solution for distributed data trading since it could train machine learning models in a distributed manner thereby enhancing data privacy without sharing local data. However, it is still challenging to trade data through FL due to unfair model allocation issues arising from the unreliability of user-provided data. To tackle it, we propose F2UL, a Fairness-aware Federated UnLearning solution that distributes models to users commensurate with their data quality. F2UL is trained in a two-stage (TST) way and contains three main components: 1) Label-free model quality assessment (LMQA) promotes fairness by evaluating models without user-specific data, ensuring uniform assessment standards. 2) Fair model distribution (FMD) addresses the issue of unfair model distribution by allocating models with feature mapping deviation, ensuring that users who contribute low-quality models do not receive enhanced models. 3) User data federated unlearning (UDFU) ensures fairness in model distribution by employing rapid recovery federated unlearning, safeguarding regular users from the adverse effects of low-quality data on model performance. In experiments on CIFAR10, F2UL reduces the low-quality data user accuracy to 9.09% and increases regular users’ accuracy by 3.02%, thereby demonstrating F2UL's capacity to ensure fairness in data trading.
30 October 2023 in IEEE Transactions on Instrumentation and Measurement
The existing infrared and visible image fusion methods typically apply small kernel convolution that can extract local information or details of the source images but cannot easily perceive their global information. This paper proposes a lightweight large kernel convolution network for infrared and visible image fusion, L 2 Net, primarily comprising three parts: shallow feature extraction, deep feature extraction, and fused image reconstruction. The shallow feature extraction part extracts the shallow initial features of the input images, and the deep feature extraction part outputs the final fused features from the initial features using large kernel feature extraction modules (LKFE). The LKFE contains a large kernel attention block (LKA) and a small kernel skip block (SKS), which help L 2 Net to simultaneously extract the global and local information from the source images. The large kernel convolution in L 2 Net is decomposed into three lightweight convolutions to reduce the number of parameters and computational cost. In addition, an edge enhancement module (EEM) is designed to extract vital edge details. The fused image reconstruction part transformed the final fused features into the final fused image. Experimental analysis shows that our L 2 Net obtains better fusion performance and higher running efficiency than the previous state-of-the-art methods. Moreover, object detection experiments demonstrate that our L 2 Net’s fused images can effectively improve detection performance.
10 February 2025 in IEEE Internet of Things Journal
Recent model recovery methods in federated unlearning (FUL) either rely on additional communication with the remaining clients or require large amounts of high-quality data from the server for training, overlooking scenarios with limited data resources. Currently, Contrastive Language-Image Pretraining (CLIP) has demonstrated remarkable performance across a wide range of tasks, particularly excelling in few-shot learning scenarios. In this paper, inspired by CLIP, we explore the scenario of few-shot knowledge distillation and propose CLIP-Guided Few-shot Knowledge Distillation (CGKD) for Model Recovery in FUL. CGKD mainly consists of three components: 1) The unlearning module constructs the unlearning model by erasing all historical contributions of the target client, and this model is treated as the student model; 2) Fine-tuning the pre-trained CLIP model using few-shot data from the server side to obtain a more robust teacher model (CLIP*); and 3) Model recovery is achieved through knowledge distillation, leveraging the rich visual and semantic knowledge of CLIP* to enhance the student model’s understanding of image semantic context, thereby improving the performance of the unlearning model. Extensive experimental results demonstrate that CGKD outperforms the compared FUL method in recovery performance across four standard datasets, validating the effectiveness of our approach.
2 March 2026 in IEEE Transactions on Circuits and Systems for Video Technology
Infrared–visible image fusion leverages the complementary strengths of both modalities to enhance visual perception in challenging environments. While image-level fusion has achieved promising results, extending it to video remains challenging due to temporal illumination variations, brightness flickering, and visual inconsistency caused by motion under non-uniform illumination. To address these issues, we propose Seq-IF, a sequential fusion framework that ensures visual consistency and structural fidelity when processing video sequences. The framework comprises a static–dynamic decoupling module for robust foreground–background separation. For the background, fusion is performed by selecting the frame with the highest contrast to ensure clarity and stability. For the dynamic objects, the pixel intensity is adaptively adjusted across consecutive frames by a lightweight MLP-based illumination-consistency fine-tuning module that performs online adaptation and dynamically optimizes brightness in response to scene changes. Later we introduce a spatial–frequency fusion module integrating multi-scale encoder and edge-guided decoder to ensure structural consistency. Extensive experiments demonstrate that the fusion results produced by Seq-IF outperform baseline methods in terms of clarity, detail preservation, and illumination stability, achieving smoother temporal transitions and enhanced perceptual quality. Furthermore, the effectiveness of Seq-IF is validated on downstream tasks, including object detection and optical flow estimation, highlighting its applicability to real-world scenarios.
16 January 2026 in IEEE Transactions on Multimedia
Image fusion is a common approach to enhancing image quality by combining the strengths of multi-modal visual signals, thereby improving the performance of computer vision tasks. However, privacy concerns regarding raw images leave data from multiple domains and clients isolated, preventing centralized training of a universal image fusion model. Federated learning (FL) is a distributed framework that enables collaborative model training by aggregating parameters from multiple nodes without sharing raw data, offering a privacy-preserving solution for lightweight and generalizable models. In this paper, we propose the federated multi-modal image fusion (FedMIF), a general multi-modal image fusion solution across multiple domains that operates without raw data sharing. We notice existing FL methods lack task-specific strategies tailored for general fusion tasks, leading to suboptimal fusion performance. To address this, we propose a four-stage framework that enhances detail features while maintaining overall visual quality, which are 1) Server initialization stage, dual decoders are trained on natural data to regulate feature distribution. 2) Task-agnostic stage, all clients collaboratively train the encoder to improve feature extraction performance. 3) Task-specific stage, training is conducted within relevant tasks to address the negative impact of inconsistent optimization objectives across different tasks. 4) Local-specific stage, local adapters are trained on each client to help mitigate detail loss resulting from model aggregation. Complete subjective and objective comparisons prove that FedMIF could generate information-rich fusion results on four fusion tasks while requiring less communication overhead. Our code is available at: https://github.com/suweijian1996/FedMIF.