Quality assessment is essential to creating and comparing video compression algorithms. Despite the development of many new quality-assessment methods, well-known and generally accepted codecs comparisons mainly employ classical methods such as PSNR, SSIM, and VMAF. These methods have different variations: temporal pooling techniques, color-component summations and versions. In this paper, we present comparison results for generally accepted video-quality metrics to determine which ones are most relevant to video codecs comparisons. For evaluation we used videos compressed by codecs of different standards at three bitrates, and subjective scores were collected for these videos. Evaluation dataset consists of 789 encoded streams and 320294 subjective scores. VMAF calculated for all Y, U, V color spaced showed the best correlation with subjective quality, and we also showed that the usage of smaller weighting coefficients for U and V components leads to a better correlation with subjective quality.
This paper discusses the challenges of evaluating SDR-to-HDR methods. The lack of comprehensive comparisons between existing methods prompted us to independently evaluate 14 HDR reconstruction approaches. To achieve this, we created a new closed HDR dataset. The evaluation involves objective quality metrics, such as PSNR, SSIM, MS-SSIM and NIQE adapted for HDR content. The study presents the first publicly available subjective scores for HDR reconstruction methods. The perceived quality of the reconstructed HDR videos is assessed by conducting a pairwise comparison. The research shows the strengths and weaknesses of each method, offering valuable insights for HDR video reconstruction and highlighting the need for improved quality assessment metrics in this domain. In addition, we introduce a novel quality evaluation metric, which leverages machine learning techniques to measure the effectiveness of HDR reconstruction methods.