Due to the scattering and attenuation of light into the water, the underwater image usually appears with color distortion, blurred details, and low contrast. To address these problems, a novel two-stage underwater image convolutional neural network (CNN) based on structure decomposition (UWCNN-SD) for underwater image enhancement is proposed by considering the characteristics of underwater imaging. Specifically, the raw underwater image is decomposed into high-frequency and low-frequency based on theoretical analysis of the underwater imaging. Then, a two-stage underwater enhancement network including a preliminary enhancement network and a refinement network is proposed. In the first stage, the preliminary enhancement network, which contains the high-frequency and the low-frequency enhancement networks, is proposed. The high-frequency part is enhanced directly by a deep learning network, and the low-frequency enhancement network is based on the underwater imaging, which is integrated transmission map and background light into joint component map. In the second stage, the refinement network is designed to further optimize the color of the underwater image by considering complexity of underwater imaging. The experimental results of synthetic and real-world underwater images/videos demonstrate that the proposed UWCNN-SD method can perform color correction and enhancement on different types of underwater images. The ablation study verifies the effectiveness of each component, and application tests further illustrate that the proposed UWCNN-SD method can obtain underwater images with higher visual quality. The trained model is available at: https://github.com/wushengcong/UWCNN-SD .

Due to the light scattering and absorption of impurities, the quality of underwater imaging is poor, which seriously affects underwater exploration and research. To address the problem, a novel underwater image enhancement method integrating the convolutional neural network (CNN) with a feature fusion Transformer (UIE-Convformer) is proposed. Specifically, the proposed UIE-Convformer adopts a multi-scale U-Net structure to fully mine rich texture information and semantic information. Firstly, considering that CNN is more efficient and comprehensive in extracting local feature information of underwater images, the ConvBlock module based on CNN is proposed to extract local features of images and ensure the efficiency and integrity of feature extraction. Furthermore, considering the serious color deviation caused by the absorption and scattering of light in water, as well as the large-scale blur and diffusion effects in the underwater environment, the feature fusion transformer module (Feaformer) for global information fusion and reconstruction is proposed to establish long-distance feature dependency. Additionally, the Jump Fusion Connection Module (JFCM) is built between the encoder and the decoder to fuse multi-scale features through effective bidirectional cross-connection and weighted fusion, which helps to provide richer feature information for the reconstruction of the decoder. Finally, the refinement module is designed to further optimize the details of the underwater images and achieve better visual effects. Experimental results on available datasets show the effectiveness of the proposed UIE-Convformer.

Point cloud (PC) compression inevitably introduces distortion during communication, which can affect users’ visual experience. Thus, efficient point cloud quality measurement (PCQM) tools are highly desired to measure the PC’s visual quality. In this paper, a pseudo-reference PCQM metric based on joint two-dimensional (2D) and three-dimensional (3D) distortion description is proposed. In 2D description, aiming at the visual quality degradation reflected cooperatively by PC’s texture distortion and geometry distortion, a joint texture-geometry distribution with texture projection map and geometry projection map of the video-based point cloud compression (V-PCC) standard is constructed to measure the joint texture-geometry distortion of PC. Since the geometry distortion of PC results in the similar distortion phenomena in the geometry projection map and texture projection map, a self-reference geometry-texture structural similarity (SGT-SSIM) is proposed. The separate statistical features of texture projection map and geometry projection map are also considered. In 3D description, considering the limitations of using full-reference metric and the difficulty of directly reflecting PC cracks and outliers only by the V-PCC projection, a pseudo-reference PC is constructed by performing Poisson surface reconstruction on the distorted PC. Then, the point-to-distribution is used to directly characterize pseudo-referenced geometry distortion, while gray level-gradient co-occurrence matrix based on key points of PC is constructed to measure the texture distortion. Finally, the features with joint 2D and 3D distortion description are combined to measure the PC visual quality more comprehensively. Experimental results on five PC datasets demonstrate that the proposed metric has comparable performance to the existing full-reference metrics.

Cell-phone use while driving results in potentially severe safety hazards. In this paper, a scheme for detecting cell-phone use that is based on deep learning is proposed, which can eliminate the potential risk by detecting the driver behavior and issuing an early warning. The proposed scheme consists of two stages: model training and practical testing. In the former, a multi-angle arrangement of cameras is first designed. Then, based on self-established data set, two independent convolutional neural networks (CNNs) are trained by optimizing the size and number of the convolution kernels, which can efficiently recognize cell-phones and hands in real time. In the testing stage, dynamic region extraction and skin color detection are employed as preprocessing to improve the accuracy of target recognition. Then, with the trained CNNs, the detection of cell-phone and hand targets is carried out, and the corresponding early warning is issued based on the distance of the interaction between the cell-phone and the hand. Numerous experiments are conducted and the results demonstrate that the proposed scheme can accurately detect cell-phone use during driving in real time, with a running time of 144 fps and an accuracy of 95.7%.

Screen content images (SCIs) have been rapidly and widely applied in interactive multimedia applications. The problem of quality assessment for SCIs is an interesting research topic. Most of the existing methods use subjective and independent features in gray domain to predict the image quality, which can't comprehensively characterize the image properties or lack unified mathematical explanation for SCIs. To address these problems, we propose a novel blind quality assessment method based on macro-micro modeling of tensor domain dictionary for SCIs in this paper. In the proposed method, the tensor decomposition is explored first to avoid the loss of color information, and then a target dictionary is learned more effectively with the principal components. Furthermore, a macro-micro model is established to characterize the micro and macro features in the target dictionary space, which is capable of providing a systematic mathematical interpretation for feature extraction. For the micro features, a log-normal pooling scheme is designed to enhance the effectiveness of feature aggregation by analyzing the particularity of the statistical distribution of sparse codes. Additionally, the statistical properties are mainly discussed and studied based on the Bernoulli law of large numbers, and then a reliable macro feature is generated to describe the relationship between the statistical distribution and quality degradation of SCIs. Experimental results determined by using three public SCI databases show that the proposed method can perform better than relevant existing methods in the prediction of the visual quality of SCIs, especially in terms of the generalization for distortion type and interpretability for feature generation.

Considering multidimensional structure of the multi-exposure images, a new Tensor product and Tensor-singular value decomposition based Multi-Exposure image Fusion (TT-MEF) method is proposed. The main innovation of this work is to explore a new feature representation of multi-exposure images in the new tensor domain and design the fusion strategy on this basis. Specifically, the luminance and the chrominance channels are fused separately to maintain color consistency. For the luminance fusion, the luminance channel of multi-exposure images is divided into two parts, that is, de-mean term and mean term. The de-mean term is represented as a tensor to extract the feature. Then, the tensor product and tensor-singular value decomposition (T-SVD) are used to design a tensor feature extractor. Furthermore, a fusion strategy of the de-mean term is presented according to the visual saliency model, and a fusion strategy of the mean term is defined by the local and the global visual weights to control counterpoise between the local and global luminance. For the chrominance fusion, a new fusion strategy is also designed by the tensor product and T-SVD, similar to the luminance fusion. Finally, the fused image is obtained by combining the luminance and chrominance fusion. Experimental results show that the proposed TT-MEF method generally outperforms the existing state-of-the-art in terms of subjective visual quality and objective evaluation.