31 March 2015 in IEEE Transactions on Circuits and Systems for Video Technology
With a lot of enhanced coding tools introduced, High Efficiency Video Coding (HEVC) achieves significant improvement in coding efficiency at the cost of increased computational complexity. To efficiently reduce the encoding time of HEVC, a wavefront-based high parallel (WHP) solution integrating novel data-level and task-level methods is proposed in this paper. On data level, optimal single-instruction-multiple-data algorithms are designed for the enhanced coding tools, i.e., replacing the multiplication in motion compensation by add and shift operations with reduced instruction cycles, removing the transpose in transform via realignment of coefficients, and minimizing the memory access in sum of absolute difference/sum of squared differences calculation by fully reusing the registers. On task level, a novel inter-frame wavefront (IFW) method is developed by effectively decreasing the dependence of wavefront parallel processing (WPP). In addition, a coding tree block level parallelism analysis method is presented to prove the superior of IFW method compared with other HEVC representative parallel methods. Besides, a three-level thread management scheme is proposed to best exploit the parallelism of IFW method and achieve corresponding encoding speedup. Extensive experimental results show that, the overall WHP solution can bring up to $57.65\times $ , $65.55\times $ , and $88.17\times $ speedup for HEVC encoding of Wide Video Graphics Array, 720p and 1080p standard test sequences, while maintaining the same coding performance as with WPP. The proposed solution is also applied in several leading video companies in China, providing HEVC video service for more than 1.3 million users everyday.
25 February 2023 in ACM Transactions on Multimedia Computing, Communications, and Applications
Deep neural networks have achieved remarkable success in HEVC compressed video quality enhancement. However, most existing multiframe-based methods either deliver unsatisfactory results or consume a significant amount of resources to leverage temporal information of neighboring frames. For the sake of practicality, a thorough investigation of the architecture design of the video quality enhancement network regarding enhancement performance, model parameters, and running speed is essential. In this article, we first propose an efficient alignment module that can quickly and accurately aggregate the spatiotemporal information of neighboring frames. The proposed module estimates deformable offsets progressively in lower-resolution space motivated by the observation of offset correlations between adjacent pixels. Then, the quantization parameter (QP) that represents compression level prior knowledge is utilized to guide aligned feature enhancement. By combining alignment feature distillation with residual feature correction, we obtain an efficient QP attention block. To save the storage space of the network, we design a hash buffer to store QP embedding features. These efficient components allow our network to effectively exploit temporal redundancies and obtain favorable enhancement capability while maintaining a lightweight structure and fast running speed. Extensive experiments demonstrate that the proposed approach outperforms state-of-the-art methods over different QPs by up to 0.09 to 0.11 dB, whereas the inference time can be reduced by up to 69%.
27 August 2012 in IEEE Transactions on Image Processing
This paper provides a systematic rate-distortion (R-D) analysis of the dead-zone plus uniform threshold scalar quantization (DZ+UTSQ) with nearly uniform reconstruction quantization (NURQ) for generalized Gaussian distribution (GGD), which consists of two aspects: R-D performance analysis and R-D modeling. In R-D performance analysis, we first derive the preliminary constraint of optimum entropy-constrained DZ+UTSQ/NURQ for GGD, under which the property of the GGD distortion-rate (D-R) function is elucidated. Then for the GGD source of actual transform coefficients, the refined constraint and precise conditions of optimum DZ+UTSQ/NURQ are rigorously deduced in the real coding bit rate range, and efficient DZ+UTSQ/NURQ design criteria are proposed to reasonably simplify the utilization of effective quantizers in practice. In R-D modeling, inspired by R-D performance analysis, the D-R function is first developed, followed by the novel rate-quantization (R-Q) and distortion-quantization (D-Q) models derived using analytical and heuristic methods. The D-R, R-Q, and D-Q models form the source model describing the relationship between the rate, distortion, and quantization steps. One application of the proposed source model is the effective two-pass VBR coding algorithm design on an encoder of H.264/AVC reference software, which achieves constant video quality and desirable rate control accuracy.
15 July 2024 in IEEE Transactions on Circuits and Systems for Video Technology
The Versatile Video Coding (VVC) standard introduces a quad-tree with a nested multi-type tree (QTMT) partition structure to improve the rate-distortion (RD) performance, but this leads to a substantial increase in encoding complexity. Previous studies have labeled partition modes of CUs using hard targets (i.e., one-hot labels) generated by VVC reference software (VTM), which is challenging for neural networks to predict accurately. Furthermore, in earlier works, the VVC restrictions are not incorporated into convolutional neural network (CNN), not fully exploiting the predicting capacity of CNN. In this paper, we propose a novel soft-target and restriction-aware neural network (STRANet) to address these issues. Firstly, inspired by the observation that a CU may split differently under various circumstances, we collect these RD costs and precisely estimate the probability of each partition mode to generate a soft target. Secondly, our neural network incorporates QP and restriction type through attention modules so as to output predictions that are standard-compliant with simple post-processing. Thirdly, Window Attention Module, a combination of CNN and attention mechanism, is adopted to further enhance performance on GPU. Through the application of these methods, STRANet reduces encoding time by 51.84% and 61.00% with 0.44% and 0.84% Bjøntegaard delta bit-rate (BD-BR) increase, superior to state-of-the-art methods. The code has been released at https://github.com/cppppp/STRANet.