20 August 2020 in IEEE Transactions on Circuits and Systems for Video Technology
The paper proposes a novel frame-wise filtering method based on Convolutional Neural Networks (CNNs) for enhancing the quality of HEVC decoded videos. A novel deep neural network architecture is proposed for post-filtering the entire intra-coded videos. A novel scheme utilizing frame-size patches is employed for training the network. The proposed method filters the luminance channel separately from the pair of chrominance channels. A novel patch generation paradigm is proposed where, for each color channel, the corresponding mode map is generated based on the HEVC intra-prediction mode index and block segmentation. The proposed CNN-based filtering method is an alternative to the traditional HEVC built-in in-loop filtering module for intra-coded frames. Experimental results on standard test sequences show that the proposed method outperforms the HEVC standard with average BD-rate savings of 11.1% and an average BD-PSNR improvement of 0.602 dB. The average relative improvement in ΔPSNR is around 105% at QP=42 and around 85% at QP=32 compared with state-of-the-art machine-learning-based methods.
9 September 2019 in IEEE Transactions on Circuits and Systems for Video Technology
The paper proposes a novel block-wise prediction paradigm based on Convolutional Neural Networks (CNNs) for lossless video coding. A deep neural network model which follows a multi-resolution design is employed for block-wise prediction. Several contributions are proposed to improve neural network training. A first contribution proposes a novel loss function formulation for an efficient network training based on a new approach for patch selection. Another contribution consists in replacing all HEVC-based angular intra-prediction modes with a CNN-based intra-prediction method, where each angular prediction mode is complemented by a CNN-based prediction mode using a specifically trained model. Another contribution consists in an efficient adaptation of the CNN-based intra-prediction residual for lossless video coding. Experimental results on standard test sequences show that the proposed coding system outperforms the HEVC standard with an average bitrate improvement of around 5%. To our knowledge, the paper is the first to replace all the traditional HEVC-based angular intra-prediction modes with an intra-prediction method based on modern Machine Learning techniques for lossless video coding applications.
4 November 2020 in Electronics Letters
The Letter introduces a novel filtering method based on convolutional neural networks (CNNs) for quality enhancement of light field (LF) images captured by a plenoptic camera and compressed using high‐efficiency video coding (HEVC). The method takes advantage of the macro‐pixel (MP) structure specific to the LF images and proposes a novel MP‐wise filtering approach based on a novel deep neural network architecture. The proposed CNN‐based method achieves an outstanding performance when HEVC is employed without its in‐loop filters. Experimental results show high luminance‐peak signal‐to‐noise ratio (Y‐PSNR) gains and average Y‐Bjøntegaard delta (BD)‐rate savings of over HEVC on a large data set.
31 October 2024 in Journal of Visual Communication and Image Representation
1 January 2026 in IEEE Transactions on Geoscience and Remote Sensing