Implicit neural representations (INRs) for various data types have gained popularity in the field of deep learning owing to their effectiveness. However, previous studies on INRs have only focused on recovering original representations. This paper investigated an image compression model based on INRs using a model compression technique for entropy-constrained neural networks. Specifically, the proposed model trains a multilayer perceptron (MLP) to overfit a single image and then uses its weights to optimize its compressed representation using additive uniform noise. Accordingly, the proposed model efficiently minimizes the size of the model weight in an end-to-end manner. This training optimization process is fairly desirable for adjusting the rate of distortion for image compression. In contrast to other model compression techniques, the proposed model is implemented without additional training process or memory cost. By introducing entropy loss, this paper demonstrated that the proposed model can be used to preserve high image quality while maintaining smaller model size. The experimental results demonstrated that the proposed model achieved comparable performance to conventional image compression models without incurring high storage costs.
Immersive video streaming has become very popular. To increase the quality of experience (QoE) with immersive media, user movement adaptive video streaming, three degrees of freedom plus (3DoF+), has emerged and is expected to meet this growing demand. Satisfying the limit of the bandwidth, providing high-quality immersive experience is challenging because 3DoF+ system requires high resolution, multi-view video transmission. This paper proposes a stride based 3DoF+ 360 video streaming system and introduces two main ideas: (i) a multi-view video redundancy removal method using view synthesis, (ii) a multi-view video residual packing method. The proposed multi-view video compression method removes redundancy between videos and packs them into one video, and it exhibits a BD-rate saving of 36.0%in maximum compared to the results of the high-efficiency video coding reference model. In addition, the proposed system requires fewer number of decoders for the clients, and it decreases the burden for immersive video streaming.
Recently, with the increasing demand for virtual reality (VR), experiencing immersive contents with VR has become easier. However, a tremendous amount of calculation and bandwidth is required when processing 360 videos. Moreover, additional information such as the depth of the video is required to enjoy stereoscopic 360 contents. Therefore, this paper proposes an efficient method of streaming high-quality 360 videos. To reduce the bandwidth when streaming and synthesizing the 3DoF+ 360 videos, which supports limited movements of the user, a proper down-sampling ratio and quantization parameter are offered from the analysis of the graph between bitrate and peak signal-to-noise ratio. High-efficiency video coding (HEVC) is used to encode and decode the 360 videos, and the view synthesizer produces the video of intermediate view, providing the user with an immersive experience.
The Moving Picture Experts Group (MPEG) has started an immersive media standard project to enable multi-view video and depth representation in three-dimensional (3D) scenes. The MPEG Immersive Video (MIV) standard technology is intended to provide a limited 6 degrees of freedom (DoF) based on depth map-based image rendering (DIBR). The 6DoF immersive video system is still challenging because multiple high-quality video streams require high bandwidth and computing resources. This paper proposes a group-based adaptive rendering method for 6DoF immersive video streaming. With group-based MIV, each group can be transmitted independently, which enables adaptive transmission depending on the user’s viewport. The proposed method derives weights from groups for view synthesis and allocates high-quality bitstreams according to a given viewport. This paper also discussed the results of the group-based approach in the MIV, and the advantages and drawbacks of this approach are detailed. In addition, pixel rate constraint analysis has been introduced to facilitate deployment with existing video codecs. On end-to-end evaluation metrics with TMIV anchor, the proposed method saves average 37.26% Bjontegaard-delta rate (BD-rate) on the peak signal-to-noise ratio (PSNR).
The moving picture experts group (MPEG) video coding group started an immersive video coding standard project to accomplish acceptable bandwidth and decoding resources for multiple immersive videos with texture (color) and geometry (depth) information. The MPEG immersive video (MIV) coding standard provides functionality to remove inter-view redundancy and merge the residuals into videos defined as atlases. Although MIV reduces the number of videos to decode, an efficient decoder instance reduction method has not been actively investigated. This paper proposes a frame packing implementation in MIV software. The proposed method conducts frame packing for geometry atlases to align with texture atlases, encodes them as subpicture bitstreams, and merges one texture / geometry atlas into a single bitstream with a bitstream merger. Furthermore, the proposed method provides control over the number of decoder instances without significantly increasing the computational complexity compared to existing frame packing methods. Therefore, the proposed method was accepted in MIV and will be implemented on the recent version of the reference software of MIV.