The paper presents a new method of depth estimation, dedicated for free-viewpoint television (FTV) and virtual navigation (VN). In this method, multiple arbitrarily positioned input views are simultaneously used to produce depth maps characterized by high inter-view and temporal consistencies. The estimation is performed for segments and their size is used to control the trade-off between the quality of depth maps and the processing time of depth estimation. Additionally, an original technique is proposed for the improvement of temporal consistency of depth maps. This technique uses the temporal prediction of depth, thus depth is estimated for P-type depth frames. For such depth frames, temporal consistency is high, whereas estimation complexity is relatively low. Similarly, as for video coding, I-type depth frames with no temporal depth prediction are used in order to achieve robustness. Moreover, we propose a novel parallelization technique that significantly reduces the estimation time. The method is implemented in C++ software that is provided together with this paper, so other researchers may use it as a new reference for their future works. In performed experiments, MPEG methodology was used whenever possible. The provided results demonstrate the advantages over the Depth Estimation Reference Software (DERS) developed by MPEG. The fidelity of a depth map, measured by the quality of synthesized views, is higher on average by 2.6 dB. This significant quality improvement is obtained despite a significant reduction of the estimation time, on average 4.5 times. The application of the proposed temporal consistency enhancement method increases this reduction to 29 times. Moreover, the proposed parallelization results in the reduction of the estimation time up to 130 times (using 6 threads). As there is no commonly accepted measure of the consistency of depth maps, the application of compression efficiency of depth is proposed as a measure of depth consistency.

The extensions of Advanced Video Coding (AVC) and High-efficiency Video Coding (HEVC) for multiview video plus depth allow for coding of 3D video scenes. During the standardization of these extensions it was demonstrated that compression performance can often be improved by application of nonlinear depth transformation prior to the compression and applying the respective backward nonlinear depth transformation after decompression. Such an approach is referred to as the compression of Nonlinear Depth Representation (NDR). In this paper, a survey of the NDR technology is provided, but a large part of this survey comprises yet unpublished results. Firstly, the motivation and rationale behind the idea of NDR are discussed. The options for the choice of nonlinear transformation are provided with the respective formulations. Also, standardization of 3D video coding with NDR within ISO/IEC and ITU is discussed along with the experimental results that demonstrate benefits related to the application of NDR in 3D-AVC and 3D-HEVC. Furthermore, experimental comparisons are provided for simulcast coding of multiview plus depth video using AVC and HEVC technology with and without NDR. Similar experimental results are also provided for simulcast coding using the current version of Versatile Video Coding (VVC) technology that is under development for the forthcoming standard of ISO/IEC (MPEG-I) and ITU. All of the experimental results have been obtained for standard JVC-3V video test sequences under Common Test Conditions for 3D Video Coding.

This paper presents a detailed, quantitative analysis of video quality losses in a homogeneous HEVC video transcoder together with the analysis of the origin of those quality losses and the influence of quantization step alignment on transcoding. With the use of HM15.0 reference software and a set of test video sequences, the cascaded pixel domain video transcoder (CPDT) concept has been used to gather all the necessary data needed for the analysis. This experiment was performed for a wide range of source and target bitrates. The essential result of the work is an extensive evaluation of CPDT, commonly used as a reference in works on effective video transcoding. So far, no such extensively performed study has been made available. The quality degradation between a transcoded video and a video that would be the result of direct compression of the original video at the same bitrate as the transcoded one has been reported. The dependences between quality degradation caused by transcoding and bitrate changes of the transcoded data stream are clearly presented in graphs.

The application of machine learning to video coding is generally studied in two main approaches: end-to-end video coding using deep neural networks and classic hybrid codecs with individual tools implemented using such networks. This work exploits the latter approach, where a trained Artificial Neural Network (ANN) is used for fast implementation of the search for the partitioning of Coding Tree Units (CTU) into Coding Units (CUs) and Prediction Units (PUs). The proposed approach differs from the previous ones, among other factors, by the application of an ANN with probabilistic soft outputs, which allows for assessing the probability of particular division patterns. This is followed by a decision algorithm that selects more than one candidate division pattern from the most probable patterns provided by ANN. Two variants of the ANN model are considered, out of which the extended one involves deep partitioning at the PU-level. The number of selected division pattern candidates is controlled by a single parameter – uncertainty range – that allows for choosing a trade-off between the complexity and the loss of coding efficiency. This feature is particularly important for broadcasting applications, where processing must be adjusted to the current computational load and resource availability. The experiments demonstrate that the proposed approach yields good results using much smaller ANNs than those described in the references. This makes the proposed approach well-suited for the usage on edge servers where GPU may not be available. The source code for the respective HEVC codec software implementation is provided.

Video coding for machines is an emerging area within video compression technology that has recently attracted considerable research attention. Within the ISO/IEC standardization activities, efforts are underway to develop a new standard optimized for machine vision tasks, rather than just for human-oriented video consumption. In this context, the novel contribution of this work is the application of neural networks to perform chroma synthesis at the decoder side, thus eliminating the need for direct chroma transmission. This concept has been implemented and validated in the Video Coding for Machines Reference Software (VCM-RS), a test model developed by the MPEG Video group, which is briefly described for completeness. Experimental results reported in this paper show that our approach significantly reduces the number of bits required for video representation, achieving bitrate savings of up to 60% for certain video sequences. The presented proposal has been accepted in the current version of the specification of the upcoming VCM standard.

The work is focused on video compression for the scenarios, where the decoded video serves not only human viewers but also as input for systems implementing various machine vision tasks, such as object detection and tracking. The proposed innovative tool is based on retargeting video frames processed based on Regions of Interest (RoI), corresponding to the individual objects detected in the frames. Experimental evaluation demonstrates significant average bitrate reduction while maintaining the same quality, ranging from 3 to 57% depending on the machine vision task and encoding scenario. The proposal underwent thorough consideration within the MPEG group and was adopted for the upcoming Video Coding for Machines (VCM) technology.