30 November 2023 in IEEE Transactions on Multimedia
Recently, many compression algorithms are applied to decrease the cost of video storage and transmission. This will introduce undesirable artifacts, which severely degrade visual quality. Therefore, Video Compression Artifacts Removal (VCAR) aims at reconstructing a high-quality video from its corrupted version of compression. Generally, this task is considered as a vision-related instead of media-related problem. In vision-related research, the visual quality has been significantly improved while the computational complexity and bitrate issues are less considered. In this work, we review the performance constraints of video coding and transfer to evaluate the VCAR outputs. Based on the analyses, we propose a Spatial-Temporal Attention-Guided Enhancement Network (STAGE-Net). First, we employ dynamic filter processing, instead of conventional optical flow method, to reduce the computational cost of VCAR. Second, we introduce self-attention mechanism to design Sequential Residual Attention Blocks (SRABs) to improve visual quality of enhanced video frames with bitrate constraints. Both quantitative and qualitative experimental results have demonstrated the superiority of our proposed method, which achieves high visual qualities and low computational costs.
19 May 2022 in IEEE Transactions on Broadcasting
The emerging 360-degree video has a good practical prospect in video broadcasting due to its more immersive experience than traditional videos. To maintain a high fidelity with panoramic vision, 360-degree video adopts a significantly increased resolution than planar videos, which also inevitably leads to higher encoding complexity. In this paper, we propose a flexible complexity optimization method for 360-degree video coding, in which the encoder is capable of adapting to different complexity constraints in diversified broadcasting scenarios. In particular, the proposed method is realized with a latitude-based computation complexity allocation. Firstly, it investigates the Coding Unit (CU) partition variation along the latitude (i.e., vertical axis) and divides each video frame into Largest CU (LCU)-based latitude regions. Then, it formulates the complexity allocation problem from the global complexity target to all latitude regions, by considering the intraframe dependencies and Rate-Distortion (RD) models. Finally, it calculates the solution to this complexity allocation problem and further employs it in 360-degree video coding. Comprehensive experiments on the standard dataset reveal the superiority of our method, which achieves a flexible computational time reduction from 14.53% to 68.36% whilst maintaining a high visual quality of compressed videos.
21 December 2022 in IEEE Transactions on Circuits and Systems for Video Technology
With a significant Rate-Distortion (RD) improvement than H.265/HEVC, Versatile Video Coding (VVC) has set a new milestone in lossy video compression. It also incorporates the emerging λ-domain rate control technique, aiming at a higher visual quality under a fixed bit constraint. However, the challenge remains how to efficiently allocate bits to all frames and Coding Tree Units (CTUs). In this paper, we propose an effective solution by formulating the above task as a Nash equilibrium problem, where all CTUs are treated as players that bargains with each other. By introducing λ-domain RD models, a constrained optimization is derived with no closed-form solution. We then propose a two-step strategy to address this issue: a Newton method to iteratively calculate an intermediate variable, and a final solution of Nash equilibrium to obtain an approximately optimal λ. Finally, we utilize the derived λ to perform an effective CTU-level bit allocation, which is the very first attempt to introduce Nash equilibrium in λ-domain rate control. Experimental results with Common Test Conditions (CTC) demonstrate the effectiveness and superiority of our method, which outperforms the state-of-the-art CTU-level rate allocation algorithms for VVC.
8 December 2022 in IEEE Signal Processing Letters
Hitherto, Versatile Video Coding (VVC) has a more magnificent overall performance than High Efficiency Video Coding (HEVC). The Quadtree with Nested Multi-Type Tree (QTMT) coding block structure can substantially enhance video coding quality in VVC. However, the coding gain also leads to a greater coding complexity. Therefore, this letter proposes a Fast Decision Scheme Based on Structural Similarity Index Metric Variation (FDS-SSIMV) to solve this problem. Firstly, the Structural Similarity Index Metric Variation (SSIMV) characteristic among the sub coding units of the spit mode is illustrated. Next, to evaluate the SSIMV value, SSIMV measure strategies are designed for different split modes in this letter. Then, the desired split modes are selected by the SSIMV values. Experimental results show that the proposed method achieves an average encoding Time Saving (TS) and Bjøntegaard Delta Bit Rate (BDBR) with 64.74% and 2.79%, respectively, outperforming the benchmarks
17 February 2025 in IEEE Transactions on Multimedia
Deep video coding has paved a way to break through the performance bottleneck of reigning hybrid video coding. However, unlike hybrid video codecs, existing deep video codecs cannot offer both flexible rates and regulable complexities within one single codec, which limits their applications. In this paper, we propose a Regulable Deep Video Codec (RDVC) to address the above issue. First, we propose an Adaptive Feature Compression (AFC) network that generates variable rates while ensuring Rate-Distortion (RD) performance. The network introduces a two-stage coarse-to-fine rate adjustment that can be controlled by a user-specified rate level. Second, we propose a Spatio-Temporal Feature Propagation (STFP) mechanism to provide high-quality reference information for AFC process. Third, we also utilize slimmable convolutional components in our framework to adjust decoding complexity constrained by user configuration. Experimental results demonstrate that RDVC can adjust the codec structure flexibly according to different user configurations while maintaining advanced performance. On average, it reduces the bit-per-pixel (bpp) by 9.35%$/$58.12% while maintaining the same PSNR/MS-SSIM as the reference software VTM-13.2. Sourcecode is available at: https://github.com/WXJ0001/RDVC.