Hyperspectral image (HSI) possesses three intrinsic characteristics: the global correlation across spectral domain, the nonlocal self-similarity across spatial domain, and the local smooth structure across both spatial and spectral domains. This paper proposes a novel tensor based approach to handle the problem of HSI spatial super-resolution by modeling such three underlying characteristics. Specifically, a noncovex tensor penalty is used to exploit the former two intrinsic characteristics hidden in several 4D tensors formed by nonlocal similar patches within the 3D HSI. In addition, the local smoothness in both spatial and spectral modes of the HSI cube is characterized by a 3D total variation (TV) term. Then, we develop an effective algorithm for solving the resulting optimization by using the local linear approximation (LLA) strategy and the alternative direction method of multipliers (ADMM). A series of experiments are carried out to illustrate the superiority of the proposed approach over some state-of-the-art approaches.
18 April 2024 in IEEE Transactions on Emerging Topics in Computational Intelligence
Most existing dehazing networks rely on synthetic hazy-clear image pairs for training, and thus fail to work well in real-world scenes. In this paper, we deduce a reformulated atmospheric scattering model for a hazy image and propose a novel lightweight two-branch dehazing network. In the model, we use a Transformation Map to represent the dehazing transformation and use a Compensation Map to represent variable illumination compensation. Based on this model, we design a T wo- B ranch N etwork (TBN) to jointly estimate the Transformation Map and Compensation Map. Our TBN is designed with a shared Feature Extraction Module and two Adaptive Weight Modules. The Feature Extraction Module is used to extract shared features from hazy images. The two Adaptive Weight Modules generate two groups of adaptive weighted features for the Transformation Map and Compensation Map, respectively. This design allows for a targeted conversion of features to the Transformation Map and Compensation Map. To further improve the dehazing performance in the real-world, we propose a semi-supervised learning strategy for TBN. Specifically, by performing supervised pre-training based on synthetic image pairs, we propose a Self-Enhancement method to generate pseudo-labels, and then further train our TBN with the pseudo-labels in a semi-supervised way. Extensive experiments demonstrate that the model-based TBN outperforms the state-of-the-art methods on various real-world datasets.
8 June 2022 in IEEE Transactions on Circuits and Systems for Video Technology
Tensor completion methods based on the tensor train (TT) have the issues of inaccurate weight assignment and ineffective tensor augmentation pre-processing. In this work, we propose a novel tensor completion approach via the element-wise weighted technique. Accordingly, a novel formulation for tensor completion and an effective optimization algorithm, called tensor completion by parallel weighted matrix factorization via tensor train (TWMac-TT), is proposed. In addition, we specifically consider the recovery quality of edge elements from adjacent blocks. Different from traditional reshaping and ket augmentation, we utilize a new tensor augmentation technique called overlapping ket augmentation, which can further avoid blocking artifacts. We then conduct extensive performance evaluations on synthetic data and several real image data sets. Our experimental results demonstrate that the proposed algorithm TWMac-TT outperforms several other competing tensor completion methods. The code is available at https://github.com/yzcv/TWMac-TT-OKA.
26 July 2024 in Engineering Applications of Artificial Intelligence
Noisy labels make a profoundly negative impact on Facial Expression Recognition (FER) due to inter-class similarity and subjective annotation. Recent works mainly focus on distinguishing the clean samples or mining the latent truth, which not only need extra computational overhead, but also have the error risk of relabeling. In this work, we propose a SNEFER model without elaborately discriminating noisy labels, which can adaptively stop the negative effect by a novel contrastive regularization term. Specifically, we establish two branches: class prediction and contrastive regularization, in a parallel manner. Though the class prediction branch learns facial expression representations over the noisy labels, it keeps the ability to measure the similarity of two facial images but is not robust for FER. If the similarity exceeds a given threshold, the regularization branch pulls them tightly in the embedding space through a Bilateral Contrastive Regularization (BCR) loss, which prevents noisy labels from degrading FER model adaptively. Experimental results demonstrate that our SNEFER model outperforms the state-of-the-art methods on three FER benchmarks under different noisy levels.
Due to the absorption and scattering of light in water, captured underwater images often suffer from some degradation, such as color cast, blur, and low contrast. These types of degradation usually affect and degrade the performance of computer vision methods and tasks under water. In order to solve these problems, in this paper, we propose a multi-stage and gradually optimized underwater image enhancement deep network, named DLRNet, based on dual layers regression. Our network emphasizes important information by aggregating different depth features in the channel attention module, and the dual-layer regression module is designed with regression to obtain the ambient light and scene light transmission for an underwater image. Then, with the underwater imaging model, the enhanced underwater image for a degraded image can be obtained with normal color, higher clarity, and contrast. The experiments on some different datasets with qualitative analysis and quantitative evaluations validate our network, and show that it outperforms some state-of-the-art approaches.