31 May 2023 in IEEE Transactions on Image Processing
Saturation information in hazy images is conducive to effective haze removal, However, existing saturation-based dehazing methods just focus on the saturation value of each pixel itself, while the higher-level distribution characteristic between pixels regarding saturation remains to be harnessed. In this paper, we observe that the pixels, which share the same surface reflectance coefficient in the local patches of haze-free images, exhibit a linear relationship between their saturation component and the reciprocal of their brightness component in the corresponding hazy images normalized by atmospheric light. Furthermore, the intercept of the line described by this linear relationship on the saturation axis is exactly the saturation value of these pixels in the haze-free images. Using this characteristic of saturation, termed saturation line prior (SLP), the transmission estimation is translated into the construction of saturation lines. Accordingly, a new dehazing framework using SLP is proposed, which employs the intrinsic relevance between pixels to achieve a reliable saturation line construction for transmission estimation. This approach can recover the fine details and attain realistic colors from hazy scenes, resulting in a remarkable visibility improvement. Extensive experiments in real-world and synthetic hazy images show that the proposed method performs favorably against state-of-the-art dehazing methods. Code is available on https://github.com/LPengYang/Saturation-Line-Prior.
7 January 2022 in IEEE Transactions on Circuits and Systems for Video Technology
Global structure and local detailed texture have different effects on image enhancement tasks. However, most existing works treated these two components in the same way, without fully considering the characteristics of the global structure and local detailed texture. In this work, we propose a structure-texture aware network (STANet) that successfully exploits structure and texture features of low-light images to improve perceptual quality. To construct STANet, a fine-scale contour map guided filter is introduced to decompose the image into a structure component and a texture component. Then, structure-attention and texture-attention subnetworks are designed to fully exploit the characteristics of these two components. Finally, a fusion subnetwork with attention mechanisms is utilized to explore the internal correlations among the global and local features. Furthermore, to optimize the proposed STANet model, we propose a hybrid loss function; specifically, a color loss function is introduced to alleviate color distortion in the enhanced image. Extensive experiments demonstrate that the proposed method improves the visual quality of images; moreover, STANet outperforms most other state-of-the-art approaches.
29 March 2021 in IEEE Transactions on Neural Networks and Learning Systems
Most of the recent image segmentation methods have tried to achieve the utmost segmentation results using large-scale pixel-level annotated data sets. However, obtaining these pixel-level annotated training data is usually tedious and expensive. In this work, we address the task of semisupervised semantic segmentation, which reduces the need for large numbers of pixel-level annotated images. We propose a method for semisupervised semantic segmentation by improving the confidence of the predicted class probability map via two parts. First, we build an adversarial framework that regards the segmentation network as the generator and uses a fully convolutional network as the discriminator. The adversarial learning makes the prediction class probability closer to 1. Second, the information entropy of the predicted class probability map is computed to represent the unpredictability of the segmentation prediction. Then, we infer the label-error map of the segmentation prediction and minimize the uncertainty on misclassified regions for unlabeled images. In contrast to existing semisupervised and weakly supervised semantic segmentation methods, the proposed method results in more confident predictions by focusing on the misclassified regions, especially the boundary regions. Our experimental results on the PASCAL VOC 2012 and PASCAL-CONTEXT data sets show that the proposed method achieves competitive segmentation performance.
3 May 2021 in IEEE Transactions on Multimedia
Most existing studies performed video denoising by using multiple adjacent noisy frames to recover one clean frame; however, despite achieving relatively good quality for each individual frame, these approaches may result in visual flickering when the denoised frames are considered in sequence. In this paper, instead of separately restoring each clean frame, we propose a multiframe-to-multiframe (MM) denoising scheme that simultaneously recovers multiple clean frames from consecutive noisy frames. The proposed MM denoising scheme uses a training strategy that optimizes the denoised video from both the spatial and temporal dimensions, enabling better temporal consistency in the denoised video. Furthermore, we present an MM network (MMNet), which adopts a spatiotemporal convolutional architecture that considers both the interframe similarity and single-frame characteristics. Benefiting from the underlying parallel mechanism of the MM denoising scheme, MMNet achieves a highly competitive denoising efficiency. Extensive analyses and experiments demonstrate that MMNet outperforms the state-of-the-art video denoising methods, yielding temporal consistency improvements of at least 13.3 $\%$ and running more than 2 times faster than the other methods.
19 September 2023 in IEEE Transactions on Image Processing
Recently, learning-based multi-exposure fusion (MEF) methods have made significant improvements. However, these methods mainly focus on static scenes and are prone to generate ghosting artifacts when tackling a more common scenario, i.e., the input images include motion, due to the lack of a benchmark dataset and solution for dynamic scenes. In this paper, we fill this gap by creating an MEF dataset of dynamic scenes, which contains multi-exposure image sequences and their corresponding high-quality reference images. To construct such a dataset, we propose a ‘static-for-dynamic’ strategy to obtain multi-exposure sequences with motions and their corresponding reference images. To the best of our knowledge, this is the first MEF dataset of dynamic scenes. Correspondingly, we propose a deep dynamic MEF (DDMEF) framework to reconstruct a ghost-free high-quality image from only two differently exposed images of a dynamic scene. DDMEF is achieved through two steps: pre-enhancement-based alignment and privilege-information-guided fusion. The former pre-enhances the input images before alignment, which helps to address the misalignments caused by the significant exposure difference. The latter introduces a privilege distillation scheme with an information attention transfer loss, which effectively improves the deghosting ability of the fusion network. Extensive qualitative and quantitative experimental results show that the proposed method outperforms state-of-the-art dynamic MEF methods. The source code and dataset are released at https://github.com/Tx000/Deep_dynamicMEF .
24 March 2021 in Measurement Science and Technology
Recently, deep neural networks have achieved great success in bearing fault diagnosis. Most existing methods are developed under the assumption that the bearing vibration signals are collected under the same machine operating conditions. However, bearing fault diagnosis under cross-domain conditions will suffer from domain shift problems if the neural network is only trained with the source domain data. Moreover, acquiring enough labeled data from the target domain will be expensive and time-consuming. To address the above problems, this paper proposes an end-to-end multi-adversarial cross-domain neural network for bearing fault diagnosis, which takes labeled source domain data and unlabeled target domain data to achieve the cross-domain bearing fault diagnosis under cross-load conditions and cross-machine conditions. The proposed method employs multi-adversarial training to automatically extract the domain-invariant features from source and target domains instead of manually designing features, which combines domain-adversarial learning and mini-max entropy adversarial learning to adversarially reduce the domain discrepancy between the source and target domains and alleviate the class misalignment problem. The results of the cross-load and the cross-machine experiments prove the effectiveness of the proposed method, and the proposed method provides a promising tool for cross-domain bearing fault diagnosis.