Rate control plays an important role in video coding systems, which makes the output bitrate of a video encoder equal to the target bitrate while minimizing the distortion of the compressed video. However, most of the existing rate control schemes achieve accurate bitrate control at the loss of rate-distortion (R-D) performance. This paper proposes an effective frame level bit allocation method to improve the R-D performance whilst maintaining the high accuracy of the bitrate control in HEVC. First, an improved R-D model is presented at the frame level, which, by making use of the information of encoded frames more completely, achieves higher accuracy with lower bitrate mismatch rate. Second, a group of picture (GOP) level bit allocation approach is introduced with the consideration of the temporal R-D dependency among different GOPs, which can further enhance the R-D performance. Finally, to achieve optimal R-D performance globally, a formulation for optimal frame level bit allocation is developed with a GOP level Lagrange multiplier introduced, which takes into account the coding effects of a current frame on the other frames within the same GOP. A scheme of recursive Taylor expansion is employed to find the GOP level Lagrange multiplier. Experimental results demonstrate that the proposed method can achieve considerable R-D performance improvement and accurate bitrate control. Specifically, our method shows 3.9% and 3.8% BD-rate saving in average compared against the HEVC reference software HM16.7 with rate control in the low-delay B and low-delay P coding structures, respectively.

Although high efficiency video coding (HEVC) has achieved high coding performance, it has been shown that dependent rate-distortion optimization (RDO) can still further improve the coding performance at the encoder side. Inspired by our previous work on temporal-dependent RDO, this paper proposes a rate control method at coding tree unit (CTU) level to obtain both higher rate-distortion (R-D) performance and lower bitrate errors. We first formulate the global optimization problem in hybrid video coding and present a general solution framework based on a weighted Lagrange multiplier, where the weighted coefficient is related to the strength of temporal dependency among coding units, and a propagation factor is introduced for quantitative measurement of the temporal dependency. Second, an estimation method of CTU level R-D model is proposed to characterize R-D relationship more accurately. Finally, under the constraint of frame level bit budget, a formulation combining inter-block dependency and R-D characteristic is developed for optimal bit allocation at the CTU level. In the process of encoding a frame, a relaxed constraint algorithm (RCA) and a strict constraint algorithm (SCA) are employed to reduce the difference between numbers of actual and target bits for the current frame, respectively. Experimental results demonstrate that, on the HEVC test model (HM-16.7), our methods including RCA and SCA achieve considerable R-D performance improvement with lower bitrate errors, outperforming the state-of-the-art schemes. It is worth mentioning that even compared with HEVC without rate control, the RCA can still achieve bitrate savings of 5.4% and 5.1% on average under configurations of low delay P and B frames, respectively.

Versatile Video Coding (VVC) significantly improves the coding efficiency over the preceding high efficiency video coding (HEVC) standard, but at the expense of much higher computational complexity. Specifically for intra coding of VVC, the computational burden is mainly on the brute-force recursive rate-distortion optimization (RDO) search of quadtree with nested multi-type tree (QTMT) based coding unit (CU) partition structure. Consequently, we propose a random forest based algorithm to reduce the complexity of CU partition. The CUs are first divided into three categories, namely simple, fuzzy, and complex CUs. For simple and complex CUs, one random forest classifier is trained to directly predict the optimal partition mode. For fuzzy CUs, another random forest is trained to predict whether the partition process is terminated or not. The experimental results show that the complexity reduction of the proposed algorithm is up to 69% as compared to the VVC reference software (VTM 7.0), and averagely 57% encoding time saving is achieved with 1.21 % BDBR increase.

To save transmission bandwidth, there exists an approach to down-sample a video and then up-sample the compressed video to save bit rates. The existing Super Resolution (SR) methods generally design powerful networks to compensate the loss information introduced by down-sampling. But the information of entire video is not fully utilized and effectively fused, resulting in the learned context information that is not enough for the high quality reconstruction. We propose a multi high-quality frames Referenced Cross-scale compressed Video Super Resolution method (RCVSR) that wisely uses past-and-future information, to pursue higher compression efficiency. Specifically, a joint reference motion alignment module is proposed. Low resolution (LR) frame after up-sampling is separately aligned with past-and-future reference frames to preserve more spatial details; at the same time this LR frame is aligned with neighborhood frames to get continuous motion information and similar contents. Then, a reference based refinement module is applied to compensate motion and lost texture details by computing similarity matrix across channel dimensions. Finally, an attention guided dual-branch residual module is employed to enhance the reconstructed result concurrently. Compared with the HEVC anchor, the average gain of Bjontegaard Delta Rate (BD-Rate) under the Low-Delay-P (LDP) setting is 24.86%. In addition, an experimental comparison is made with the advanced SR methods and compressed video quality enhancement (VQE) methods, and the superior efficiency and generalization of the proposed algorithm are further reported.

With the evolution of video coding, balancing video compression efficiency with quality has become a critical challenge for researchers and the industry. The development of the next-generation video coding standards, such as Versatile Video Coding (VVC), signifies a significant leap in supporting high-resolution formats including 8K, HDR, and WCG. Currently, machine vision has emerged as a rising research focus, driven by breakthrough in Artificial Intelligence and its growing role in content generation, production, distribution, and storage in multimedia applications. This paper presents a comprehensive survey of the video coding tools in the VVC standard. Additionally, we examine recent research in next-generation video coding, particularly in Beyond VVC and end-to-end coding frameworks. Developments in shared human-machine vision systems are also discussed, emphasizing their relevance in evolving multimedia applications. Finally, this paper provides an outlook on video coding standards, considering their potential to drive next-generation multimedia technologies.