Federated machine learning which enables resource-constrained node devices (e.g., Internet of Things (IoT) devices and smartphones) to establish a knowledge-shared model while keeping the raw data local, could provide privacy preservation, and economic benefit by designing an effective communication protocol. However, this communication protocol can be adopted by attackers to launch data poisoning attacks for different nodes, which has been shown as a big threat to most machine learning models. Therefore, we in this article intend to study the model vulnerability of federated machine learning, and even on IoT systems. To be specific, we here attempt to attacking a popular federated multitask learning framework, which uses a general multitask learning framework to handle statistical challenges in the federated learning setting. The problem of calculating optimal poisoning attacks on federated multitask learning is formulated as a bilevel program, which is adaptive to the arbitrary selection of target nodes and source attacking nodes. We then propose a novel systems-aware optimization method, called as attack on federated learning (AT 2 FL), to efficiently derive the implicit gradients for poisoned data, and further attain optimal attack strategies in the federated machine learning. This is an earlier work, to our knowledge, that explores attacking federated machine learning via data poisoning. Finally, experiments on several real-world data sets demonstrate that when the attackers directly poison the target nodes or indirectly poison the related nodes via using the communication protocol, the federated multitask learning model is sensitive to both poisoning attacks.
7 March 2022 in IEEE Robotics and Automation Letters
Underwater images suffer from color casts and low illumination due to the scattering and absorption of light as it propagates in water. These problems can interfere with underwater vision tasks, such as recognition and detection. We propose an adaptive learning attention network for underwater image enhancement based on supervised learning, named LANet, to solve these degradation issues. First, a multiscale fusion module is proposed to combine different spatial information. Second, we design a novel parallel attention module(PAM) to focus on the illuminated features and more significant color information coupled with the pixel and channel attention. Then, an adaptive learning module(ALM) can retain the shallow information and adaptively learn important feature information. Further, we utilize a multinomial loss function that is formed by mean absolute error and perceptual loss. Finally, we introduce an asynchronous training mode to promote the network’s performance of multinomial loss function. Qualitative analysis and quantitative evaluations show the excellent performance of our method on different underwater datasets. The code is available at: https://github.com/LiuShiBen/LANet .
8 January 2025 in IEEE Transactions on Geoscience and Remote Sensing
With the development of deep learning, impressive progress has been made in the field of image restoration. The existing methods mainly rely on CNN and Transformer to obtain multi-scale feature information. However, these methods rarely integrate frequency domain information effectively during feature extraction, limiting their performance in image restoration. Additionally, few have combined Mamba with the Fourier domain for image restoration, which limits Mamba’s ability to perceive global degradation in the frequency domain. Therefore, we propose a new image restoration model called FMambaIR, which utilizes the complementarity between frequency and Mamba for image restoration. The core of FMambaIR is the F-Mamba block, which combines Fourier transform and Mamba for global degradation perception modeling. Specifically, F-Mamba adopts a dual branch complementary structure, including spatial Mamba branches and Fourier frequency domain global modeling. Mamba models the long-range dependencies of the entire image features, and the frequency branch utilizes Fourier to extract global degraded features from the image. Finally, we use a forward feedback network to integrate local information, which is beneficial for improving the recovery details. We comprehensively evaluate FMambaIR on several image restoration tasks, including underwater image enhancement, remote sensing image dehazing, and low-light image enhancement. The experimental results demonstrate that FMambaIR not only achieves superior performance compared to state-of-the-art methods but also significantly reduces computational complexity. Our code is available at https://github.com/mickoluan/FMambaIR.
Shadow removal, which aims to restore the illumination in shadow regions, is challenging due to the diversity of shadows in terms of location, intensity, shape, and size. Different from most multi-task methods, which design elaborate multi-branch or multi-stage structures for better shadow removal, we introduce feature decomposition to learn better feature representations. Specifically, we propose a single-stage and decoupled multi-task network (DMTN) to explicitly learn the decomposed features for shadow removal, shadow matte estimation, and shadow image reconstruction. First, we propose several coarse-to-fine semi-convolution (SMC) modules to capture features sufficient for joint learning of these three tasks. Second, we design a theoretically supported feature decoupling layer to explicitly decouple the learned features into shadow image features and shadow matte features via weight reassignment. Last, these features are converted to a target shadow-free image, affiliated shadow matte, and shadow image, supervised by multi-task joint loss functions. With multi-task collaboration, DMTN effectively recovers the illumination in shadow areas while ensuring the fidelity of non-shadow areas. Experimental results show that DMTN competes favorably with state-of-the-art multi-branch/multi-stage shadow removal methods, while maintaining the simplicity of single-stage methods. We have released our code to encourage future exploration in powerful feature representation for shadow removal https://github.com/nachifur/DMTN
30 April 2024 in IEEE Transactions on Image Processing
Existing RGB-Thermal trackers usually treat intra-modal feature extraction and inter-modal feature fusion as two separate processes, therefore the mutual promotion of extraction and fusion is neglected. Then, the complementary advantages of RGB-T fusion are not fully exploited, and the independent feature extraction is not adaptive to modal quality fluctuation during tracking. To address the limitations, we design a joint-modality query fusion network, in which the intra-modal feature extraction and the inter-modal fusion are coupled together and promote each other via joint-modality queries. The queries are initialized based on the multimodal features of the current frame, making the subsequent fusion adaptive to modal quality fluctuation during tracking. Then the joint-modality query fusion (JQF) utilizes the queries to interact with RGB-T features, allowing the intra-modal enhancement and the inter-modal interactions to be unified for mutual promotion. In this way, JQF can distinguish and enhance the complementary modality features, while filtering out redundant information. For real-time tracking, we propose regional cross-attention for cross-modal interactions to reduce computational cost. Our end-to-end tracker sets a new state-of-the-art performance on multiple RGBT tracking benchmarks including LasHeR, VTUAV, RGBT234 and GTOT, while running at a real-time speed.