Image quality assessment potentially benefits from the addition of visual attention. However, incorporating aspects of visual attention in image quality models by means of a perceptually optimized strategy is largely unexplored. Fundamental challenges, such as how visual attention is affected by the concurrence of visual signals and their distortions; whether visual attention affected by distortion or that driven by the original scene only should be included in an image quality model; and how to select visual attention models for the image quality application context, remain. To shed light on the above unsolved issues, designing and performing eye-tracking experiments are essential. Collecting eye-tracking data for the purpose of image quality study is so far confronted with a bias due to the involvement of stimulus repetition. In this paper, we propose a new experimental methodology to eliminate such inherent bias. This allows obtaining reliable eye-tracking data with a large degree of stimulus variability. In fact, we first conducted 5760 eye movement trials that included 160 human observers freely viewing 288 images of varying quality. We then made use of the resulting eye-tracking data to provide insights into the optimal use of visual attention in image quality research. The new eye-tracking data are made publicly available to the research community.

Reliably predicting video quality as perceived by humans remains challenging and is of high practical relevance. A significant research trend is to investigate visual saliency and its implications for video quality assessment. Fundamental problems regarding how to acquire reliable eye-tracking data for the purpose of video quality research and how saliency should be incorporated in objective video quality metrics (VQMs) are largely unsolved. In this paper, we propose a refined methodology for reliably collecting eye-tracking data, which essentially eliminates bias induced by each subject having to view multiple variations of the same scene in a conventional experiment. We performed a large-scale eye-tracking experiment that involved 160 human observers and 160 video stimuli distorted with different distortion types at various degradation levels. The measured saliency was integrated into several best known VQMs in the literature. With the assurance of the reliability of the saliency data, we thoroughly assessed the capabilities of saliency in improving the performance of VQMs, and devised a novel approach for optimal use of saliency in VQMs. We also evaluated to what extent the state-of-the-art computational saliency models can improve VQMs in comparison to the improvement achieved by using “ground truth” eye-tracking data. The eye-tracking database is made publicly available to the research community.

Objective image quality metrics (IQMs) potentially benefit from the addition of visual saliency. However, challenges to optimizing the performance of saliency-based IQMs remain. A previous eye-tracking study has shown that gaze is concentrated in fewer places in images with highly salient features than in images lacking salient features. From this, it can be inferred that the former are more likely to benefit from adding a saliency term to an IQM. To understand whether these ideas still hold when using computational saliency instead of eye-tracking data, we first conducted a statistical evaluation using 15 state-of-the-art saliency models and 10 well-known IQMs. We then used the results to devise an algorithm, which adaptively incorporates saliency in IQMs for natural scenes, based on saliency dispersion. Experimental results demonstrate that this can give significant improvements.

With the development of multimedia technologies and virtual reality display devices, omnidirectional videos have gained popularity nowadays. To reduce the bandwidth requirement for omnidirectional video transmission, tile-based viewport adaptive streaming methods have been proposed in the literatures. Challenges related to decoding the tiles simultaneously with limited number of decoders, and ensuring user's viewing experience during the viewport switch are still to be solved. In this paper, a two-layer fast viewport switching dynamic adaptive streaming over HTTP (DASH) system based on tiling mechanism is proposed, which incorporates the viewing trajectory of end users. To deal with the simultaneously decoding problem, an open group of picture (GOP) technique is proposed to enable merging different types of tiles into a composite stream at the client side. To reduce the quality recovery duration after the viewport change, a fast-switching strategy is also proposed. Moreover, considering the priorities of different types of chunks, a download strategy is further proposed to adapt the bandwidth fluctuations and viewport changes. Experimental results showed that the proposed system can significantly reduce the recovery duration of high quality video by approximately 90%, which can provide a better viewing experience to end users.

Efficient compression for 3D point clouds is crucial due to their massive data volume. Quality enhancement can significantly improve the compression efficiency of 3D point clouds. In this study, we propose a neural network-based quality enhancement method for color attributes of dense 3D point cloud. Our approach involves the extraction of 3D patches from an input distorted point cloud, followed by their conversion into 2D images using a specific scan order. We then propose an efficient U-Net-inspired neural network, namely SSIU-Net, to enhance the quality of these 2D images. Finally, the processed 2D images are converted back to 3D patches, allowing for the point cloud reconstruction. Experimental results demonstrate that the proposed method, when implemented in both G-PCC and V-PCC, achieves competitive results. For example, 0.146 dB, 0.417dB and 0.270 dB PSNR gains can be achieved for Luma (Y), Chroma difference of blue (Cb) and Chroma difference of red (Cr) components compared with G-PCC, corresponding to 4.1%, 9.4% and 12.5% BD-rate savings. The source code is available at https://github.com/xjr998/SSIU.

Due to the ability to depict large-scale 3D scenes, point clouds acquired by the Light Detection And Ranging (LiDAR) devices have played an indispensable role in various fields. The growing data amount of point cloud, however, brings huge challenges to existing point cloud processing networks. Developing point cloud compression algorithms has become an active research area in recent years. Representative compression frameworks include the MPEG Geometry-based Point Cloud Compression (G-PCC) standard in which a dedicated profile is designed for spinning LiDAR point clouds. In that design, prior knowledge of the LiDAR device is used to project points to nodes in a predictive structure which better reflects the spatial correlation of LiDAR point clouds. In this paper, an analysis has been conducted to explain the observed irregular point distribution in the predictive structure. A regularized projection algorithm is then proposed to construct a reliable prediction relationship in the predictive structure. Simplified geometry prediction techniques are further proposed based on the regularized projection pattern. Experimental results show that an average BD-rate gain of 18% can be achieved with lower encoding runtime if compared with MPEG G-PCC.