Visual information varying over time is typically captured by cameras that acquire data via images (frames) equally spaced in time. Using a different approach, Neuromorphic Vision Sensors (NVSs) are emerging visual capturing devices that only acquire information when changes occur in the scene. This results in major advantages in terms of low power consumption, wide dynamic range, high temporal resolution, and lower data rates than conventional video. Although the acquisition strategy already results in much lower data rates than conventional video, such data can be further compressed. To this end, in this paper we propose a lossless compression strategy based on point cloud compression, inspired by the observation that, by appropriately reporting NVS data in a ( x, y, t ) tridimensional space, we have a point cloud representation of NVS data. The proposed strategy outperforms the benchmark strategies resulting in a compression ratio up to 30% higher for the considered dataset.

Requirements of next-generation video applications are becoming a challenge for conventional video coding systems, although they have evolved over decades to accommodate the most demanding of current video applications. Semantic communications, built on the concept of transmitting just the semantics of a message and allowing the receiver to reconstruct the message based on a shared context, is a non-conventional approach being considered to overcome these challenges and improve performance of video coding systems. In this paper, a first such semantic communication-based video coding system in hybrid mode is proposed, which uses an autoencoder-based semantic encoder for inter coding, augmented by the intra coding capabilities of Versatile Video Coding (VVC) to encode key frames that form the context for the semantic communication and the residuals for improving the fidelity of the output frames. For a range of videos with differing levels of complexity, the proposed system consistently outperforms High Efficiency Video Coding (HEVC) and Advanced Video Coding (AVC) in terms of rate distortion metrics quantified by Bjontegaard Delta Rates. It also outperforms Versatile Video Coding with videos with low or high complexity, but slightly falls behind with videos with medium complexity, which can be improved by addressing the open research areas that stem from this work. The proposed system demonstrates the potential of semantic communication based video coding systems to consistently outperform state-of-the-art conventional video coding systems over a wide range video applications.

Consumer Electronics (CE) devices have started to support 360∘ videos in order to improve the perceived video quality by consumers. Higher compression in 360∘ videos not only lead to improved consumer perception, but also efficient energy and storage usage in CE devices. Consequently, efficient compression of such video content is becoming a critical design consideration of CE devices. To this end, this paper first proposes a residual weighting algorithm that utilizes the features of Weighted Craster Parabolic Projection Peak Signal-to-Noise Ratio (WCPPPSNR), to instigate the first stage of magnitude reduction of the residuals. The model parameters associated with the residual weighting algorithm are determined in the initial experimental stage. Next, the paper proposes a Quantization Parameter (QP) optimization technique which introduces the second stage of magnitude reduction for residuals with minimal impact on the overall video quality. The proposed method is tested on the Versatile Video Coding (VVC) test model with All-Intra configuration. In this context, the empirical results demonstrate that the proposed method improves the coding efficiency by 3.34% on average and outperforms the state-of-the-art techniques.

Multimedia applications have expanded and resulted in huge demands for high resolution videos in order to enhance the end user Quality of Experience (QoE). Therefore, QoE becomes a critical factor that is often addressed in design considerations of Consumer Electronics (CE) devices. Furthermore, error prone transmission conditions can influence the QoE, potentially degrading the end-user multimedia experience. Although video coding standards such as Versatile Video Coding (VVC) take potential mitigation measures to improve the bandwidth reductions, it fails to address other significant network Quality of Service (QoS) parameters that impact transmission based QoE such as transmission delay and Packet Loss Rate (PLR) in the encoding processes. In response, this paper first derives a relationship between QoE that addresses the network QoS parameters and VVC dependent parameters. Subsequently, the derived QoE model is used with the VVC encoding process to adaptively optimize Lagrange parameters as well as the quantization parameter. Extensive experiments report significant improvements to VVC Test Model (VTM) 7.0, achieving superior QoE gains over the state-of-the-art research and up to 3.41% over the anchor implementations.

360-degree video streaming for VR visualisation is characterised by large transmission data volume and stringent interactive latency demands; hence guaranteeing suitable transmission quality, while meeting the existing constraints, is highly challenging. This paper addresses the relevant grand challenge presented at MMSys 2024 on 360-degree video on-demand streaming, aiming at designing and implementing a 360-degree video on-demand streaming solution using the open-source evaluation platform E3PO. The proposed solution incorporates several strategies including viewport prediction, tiling, encoding tile selection for streaming and upsampling. The source code for the proposed solution will be publicly available on Github1.

The rapid development of virtual reality applications continues to urge better compression of 360° videos owing to the large volume of content. These videos are typically converted to 2-D formats using various projection techniques in order to benefit from ad-hoc coding tools designed to support conventional 2-D video compression. Although recently emerged video coding standard, Versatile Video Coding (VVC) introduces 360° video specific coding tools, it fails to prioritize the user observed regions in 360° videos, represented by the rectilinear images called the viewports. This leads to the encoding of redundant regions in the video frames, escalating the bit rate cost of the videos. In response to this issue, this paper proposes a novel 360° video coding framework for VVC which exploits user observed viewport information to alleviate pixel redundancy in 360° videos. In this regard, bidirectional optical flow, Gaussian filter and Spherical Convolutional Neural Networks (Spherical CNN) are deployed to extract perceptual features and predict user observed viewports. By appropriately fusing the predicted viewports on the 2-D projected 360° video frames, a novel Regions of Interest (ROI) aware weightmap is developed which can be used to mask the source video and introduce adaptive changes to the Lagrange and quantization parameters in VVC. Comprehensive experiments conducted in the context of VVC Test Model (VTM) 7.0 show that the proposed framework can improve bitrate reduction, achieving an average bitrate saving of 5.85% and up to 17.15% at the same perceptual quality which is measured using Viewport Peak Signal-To-Noise Ratio (VPSNR).