Omnidirectional or 360∘ video is increasingly being used, mostly due to the latest advancements in immersive Virtual Reality (VR) technology. However, its wide adoption is hindered by the higher bandwidth and lower latency requirements than associated with traditional video content delivery. Diverse researchers propose and design solutions that help support an immersive visual experience of 360∘ video, primarily when delivered over a dynamic network environment. This paper presents the state-of-the-art on adaptive 360∘ video delivery solutions considering end-to-end video streaming in general and then specifically of 360∘ video delivery. Current and emerging solutions for adaptive 360∘ video streaming, including viewport-independent, viewport-dependent, and tile-based schemes are presented. Next, solutions for network-assisted unicast and multicast streaming of 360∘ video content are discussed. Different research challenges for both on-demand and live 360∘ video streaming are also analyzed. Several proposed standards and technologies and top international research projects are then presented. We demonstrate the ongoing standardization efforts for 360∘ media services that ensure interoperability and immersive media deployment on a massive scale. Finally, the paper concludes with a discussion about future research opportunities enabled by 360∘ video.
Multimedia synchronization involves a temporal relationship between audio and visual media components. The presentation of “in-sync” data streams is essential to achieve a natural impression, as “out-of-sync” effects are often associated with user quality of experience (QoE) decrease . Recently , multi-sensory media (mulsemedia) has been demonstrated to provide a highly immersive experience for its users. Unlike traditional multimedia, mulsemedia consists of other media types (i.e., haptic, olfaction, taste, etc.) in addition to audio and visual content. Therefore, the goal of achieving high quality mulsemedia transmission is to present no or little synchronization errors between the multiple media components. In order to achieve this ideal synchronization, there is a need for comprehensive knowledge of the synchronization requirements at the user interface. This paper presents the results of a subjective study carried out to explore the temporal boundaries within which haptic and air-flow media objects can be successfully synchronized with video media. Results show that skews between sensorial media and multimedia might still give the effect that the mulsemedia sequence is “in-sync” and provide certain constraints under which synchronization errors might be tolerated. The outcomes of the paper are used to provide recommendations for mulsemedia service providers in order for their services to be associated with acceptable user experience levels, e.g. haptic media could be presented with a delay of up to 1 s behind video content, while air-flow media could be released either 5 s ahead of or 3 s behind video content.
23 September 2020 in IEEE Transactions on Multimedia
Recently, innovative technologies such as Virtual Reality (VR), Augmented Reality (AR), Mixed Reality (MR) and Multi-Sensorial Media (mulsemedia) have introduced new sensorial effects including vibration, smell, airflow, etc. to human life. These effects were mostly deployed for entertainment gaming) and have resulted in positive user satisfaction. This paper describes a novel Dynamic Adaptive Streaming over HTTP (DASH)-based Multi-sensory Media Delivery Solution (DASHMS) which supports adaptive mulsemedia content distribution based on the operational environment which includes network, device and user settings. DASHMS was evaluated in a real-life educational experiment involving 44 students in an Irish university. The evaluation focused on both learner satisfaction and impact on education results. The results showed how the adaptive multi-sensorial media delivery resulted in a statistically significant increase in user experience. In terms of benefit to learning outcomes however, it was only memory recall which was statistically improved in the student experiment.
Increasing number of researchers are focusing on the emerging communication technologies which enrich user perceived quality of experience by involving vision, auditory, tactile, olfaction, gustatory and other senses. However, there are multiple challenges related to using multiple sensorial media (i.e. mulsemedia), including synchronization with the traditional multimedia content and delivery over diverse network environments. This paper proposes MulseDASH, a novel multiple sensorial media content delivery solution based on the Dynamic Adaptive Streaming over HTTP standard (DASH). MulseDASH is described and evaluated in a real test-bed in terms of effectiveness of its adaptive streaming and synchronization mechanisms. Extensive testing involving both network emulation and subjective assessment experiments shows how MulseDASH performs excellent real-time streaming adjustment to match network conditions and improves user quality of experience.
11 April 2024 in IEEE Transactions on Broadcasting
The widespread use of live streaming necessitates low-latency requirements for the processing and transmission of virtual reality (VR) videos. This paper introduces a prototype system for low-latency VR video processing and transmission that exploits edge computing to harness the computational power of edge servers. This approach enables efficient video preprocessing and facilitates closer-to-user multicast video distribution. Despite edge computing’s potential, managing large-scale access, addressing differentiated channel conditions, and accommodating diverse user viewports pose significant challenges for VR video transcoding and scheduling. To tackle these challenges, our system utilizes dual-edge servers for video transcoding and slicing, thereby markedly improving the viewing experience compared to traditional cloud-based systems. Additionally, we devise a low-complexity greedy algorithm for multi-edge and multi-user VR video offloading distribution, employing the results of bitrate decisions to guide video transcoding inversely. Simulation results reveal that our strategy significantly enhances system utility by 44.77 $\%$ over existing state-of-the-art schemes that do not utilize edge servers while reducing processing time by 58.54 $\%$ .
In the current telecommunication environment, network operators are trying to cope with a significant increase in data traffic by adopting different solutions to expand their network capacity. One of these solutions is the convergence of next generation wireless networks (e.g., HSDPA, LTE and WiMAX) which involve closely interworking of existing 2G/2.5G/3G networks with the new next generation networks in terms of handover and network selection. However, the diversification in mobile devices and the heterogeneity of the wireless environment make the seamless always best connectivity of mobile users a challenge for the service providers. We propose RLoad, a novel Reputation-based Load-balancing Network Selection Strategy for heterogeneous wireless environments, built on top of the IEEE 802.21 Media Independent Handover (MIH) standard. The proposed solution makes use of a reputation-based mechanism to select the most appropriate set of networks for the mobile user and a load balancing mechanism to distribute the traffic load among the networks by making use of the Multipath TCP (MPTCP) protocol. Preliminary simulation results show significant benefits when using the proposed RLoad solution.