15 July 2024 in IEEE Transactions on Circuits and Systems for Video Technology
The Versatile Video Coding (VVC) standard introduces a quad-tree with a nested multi-type tree (QTMT) partition structure to improve the rate-distortion (RD) performance, but this leads to a substantial increase in encoding complexity. Previous studies have labeled partition modes of CUs using hard targets (i.e., one-hot labels) generated by VVC reference software (VTM), which is challenging for neural networks to predict accurately. Furthermore, in earlier works, the VVC restrictions are not incorporated into convolutional neural network (CNN), not fully exploiting the predicting capacity of CNN. In this paper, we propose a novel soft-target and restriction-aware neural network (STRANet) to address these issues. Firstly, inspired by the observation that a CU may split differently under various circumstances, we collect these RD costs and precisely estimate the probability of each partition mode to generate a soft target. Secondly, our neural network incorporates QP and restriction type through attention modules so as to output predictions that are standard-compliant with simple post-processing. Thirdly, Window Attention Module, a combination of CNN and attention mechanism, is adopted to further enhance performance on GPU. Through the application of these methods, STRANet reduces encoding time by 51.84% and 61.00% with 0.44% and 0.84% Bjøntegaard delta bit-rate (BD-BR) increase, superior to state-of-the-art methods. The code has been released at https://github.com/cppppp/STRANet.
27 February 2026 in IEEE Transactions on Multimedia
Versatile Video Coding (VVC) achieves significant compression efficiency by adopting various techniques, especially the Quad-Tree Plus Multi-Type Tree (QTMT) partition structure, but this leads to a dramatic rise in encoding complexity. Some accelerating methods have relatively low accuracy in predicting partition modes, while others increase accuracy at the cost of larger network inference time. When setting thresholds for skipping partition modes, it is common to determine them empirically or based on prediction accuracy, often neglecting their impact on encoding complexity. To achieve a balance in accuracy and efficiency, we propose to only extract the feature map of each 32×32 CU once and share it among sub-CUs. Padding between layers is removed to keep consistency between training and inference. Considering that encoding complexity of each partition mode differs, an algorithm is designed to search for the optimal thresholds with a balance between RD performance and encoding complexity. Then, we integrate a gradient-based method and a novel partition mode reuse method into VTM, which achieve a better RD-complexity trade-off at a large time-saving rate. Lastly, we remove several existing accelerating methods in VTM and find that our method is a good substitute. Experiments implemented single-threaded on CPU demonstrate that our method can reduce 25.29%$\sim$73.98% complexity of VVC intra encoding with -0.04%$\sim$1.93% BDRate increase. All the code is available at https://github.com/cppppp/FastIntra.