The use of variable block-size motion estimation (ME), disparity estimation (DE), and multiple reference frames selection aims to improve the coding efficiency of multiview video coding (MVC), however, this is at the cost of high computational complexity of these advanced coding techniques, which are not suitable for real-time video broadcasting applications. In this paper, we propose an efficient ME and DE algorithm for reducing the computational complexity of MVC. Firstly, according to the characteristics of the coded block pattern and rate distortion (RD) cost, an early DIRECT mode decision algorithm is proposed. Then, based on the characteristics of the initial search point in the ME/DE process and the observation that the best point is center-biased, an early ME/DE termination strategy is proposed. If the ME/DE early termination is not satisfied, the ME/DE search window will be reduced by applying the optimal theory. At last, two block matching search strategies are proposed to predict the best point for the ME/DE. Experimental results show that the proposed algorithm can achieve 50.05% to 77.61%, 64.83% on average encoding time saving. Meanwhile, the RD performance degradation is negligible. Especially, the proposed algorithm can be applied to not only the odd views but also the even views.

The high definition (HD) and ultra HD videos can be widely applied in broadcasting applications. However, with the increased resolution of video, the volume of the raw HD visual information data increases significantly, which becomes a challenge for storage, processing, and transmitting the HD visual data. The state-of-the-art video compression standard-H.265/High Efficiency Video Coding (HEVC) compresses the raw HD visual data efficiently, while the high compression rate comes at the cost of heavy computation load. Hence, reducing the encoding complexity becomes vital for the H.265/HEVC encoder to be used in broadcasting applications. In this paper, based on the best motion vector selection correlation among the different size prediction modes, we propose a fast motion estimation (ME) method to reduce the encoding complexity of the H.265/HEVC encoder. First, according to the prediction unit (PU) partition type, all PUs are classified into two classes, parent PU and children PUs, respectively. Then, based on the best motion vector selection correlation between the parent PU and children PUs, the block matching search process of the children PUs is adaptively skipped if their parent PU chooses the initial search point as its final optimal motion vector in the ME process. Experimental results show that the proposed method achieves an average of 20% ME time saving as compared with the original HM-TZSearch. Meanwhile, the rate distortion performance degradation is negligible.

A new methodology to measure coded image/video quality using the just-noticeable-difference (JND) idea was proposed in Lin et al. (2015). Several small JND-based image/video quality datasets were released by the Media Communications Lab at the University of Southern California in Jin et al. (2016) and Wang et al. (2016) [3]. In this work, we present an effort to build a large-scale JND-based coded video quality dataset. The dataset consists of 220 5-s sequences in four resolutions (i.e., 1920×1080,1280×720,960×540 and 640×360). For each of the 880 video clips, we encode it using the H.264/AVC codec with QP=1,…,51 and measure the first three JND points with 30 + subjects. The dataset is called the “VideoSet”, which is an acronym for “Video Subject Evaluation Test (SET)”. This work describes the subjective test procedure, detection and removal of outlying measured data, and the properties of collected JND data. Finally, the significance and implications of the VideoSet to future video coding research and standardization efforts are pointed out. All source/coded video clips as well as measured JND data included in the VideoSet are available to the public in the IEEE DataPort (Wang et al., 2016 [4]). © 2017

In this paper, we propose a machine learning-based fast coding unit (CU) depth decision method for High Efficiency Video Coding (HEVC), which optimizes the complexity allocation at CU level with given rate-distortion (RD) cost constraints. First, we analyze quad-tree CU depth decision process in HEVC and model it as a three-level of hierarchical binary decision problem. Second, a flexible CU depth decision structure is presented, which allows the performances of each CU depth decision be smoothly transferred between the coding complexity and RD performance. Then, a three-output joint classifier consists of multiple binary classifiers with different parameters is designed to control the risk of false prediction. Finally, a sophisticated RD-complexity model is derived to determine the optimal parameters for the joint classifier, which is capable of minimizing the complexity in each CU depth at given RD degradation constraints. Comparative experiments over various sequences show that the proposed CU depth decision algorithm can reduce the computational complexity from 28.82% to 70.93%, and 51.45% on average when compared with the original HEVC test model. The Bjøntegaard delta peak signal-to-noise ratio and Bjøntegaard delta bit rate are -0.061 dB and 1.98% on average, which is negligible. The overall performance of the proposed algorithm outperforms those of the state-of-the-art schemes.

High efficiency video coding (HEVC) standard achieves half bit-rate reduction while keeping the same quality compared with AVC. However, it still cannot satisfy the demand of higher quality in real applications, especially at low bit rates. To further improve the quality of reconstructed frame while reducing the bitrates, a residual highway convolutional neural network (RHCNN) is proposed in this paper for in-loop filtering in HEVC. The RHCNN is composed of several residual highway units and convolutional layers. In the highway units, there are some paths that could allow unimpeded information across several layers. Moreover, there also exists one identity skip connection (shortcut) from the beginning to the end, which is followed by one small convolutional layer. Without conflicting with deblocking filter (DF) and sample adaptive offset (SAO) filter in HEVC, RHCNN is employed as a high-dimension filter following DF and SAO to enhance the quality of reconstructed frames. To facilitate the real application, we apply the proposed method to I frame, P frame, and B frame, respectively. For obtaining better performance, the entire quantization parameter (QP) range is divided into several QP bands, where a dedicated RHCNN is trained for each QP band. Furthermore, we adopt a progressive training scheme for the RHCNN where the QP band with lower value is used for early training and their weights are used as initial weights for QP band of higher values in a progressive manner. Experimental results demonstrate that the proposed method is able to not only raise the PSNR of reconstructed frame but also prominently reduce the bit-rate compared with HEVC reference software.

Picture Wise Just Noticeable Difference (PW-JND), which accounts for the minimum difference of a picture that human visual system can perceive, can be widely used in perception-oriented image and video processing. However, the conventional Just Noticeable Difference (JND) models calculate the JND threshold for each pixel or sub-band separately, which may not reflect the total masking effect of a picture accurately. In this paper, we propose a deep learning based PW-JND prediction model for image compression. Firstly, we formulate the task of predicting PW-JND as a multi-class classification problem, and propose a framework to transform the multi-class classification problem to a binary classification problem solved by just one binary classifier. Secondly, we construct a deep learning based binary classifier named perceptually lossy/lossless predictor which can predict whether an image is perceptually lossy to another or not. Finally, we propose a sliding window based search strategy to predict PW-JND based on the prediction results of the perceptually lossy/lossless predictor. Experimental results show that the mean accuracy of the perceptually lossy/lossless predictor reaches 92%, and the absolute prediction error of the proposed PW-JND model is 0.79 dB on average, which show the superiority of the proposed PW-JND model to the conventional JND models.