12 February 2019 in IEEE Journal on Emerging and Selected Topics in Circuits and Systems
360° video on-demand streaming is a key component of the emerging Virtual Reality (VR) and Augmented Reality (AR) applications. In such applications, sending the entire 360° video demands extremely high network bandwidth that may not be affordable by today’s networks. On the other hand, sending only the predicted user’s field of view (FoV) is not viable as it is hard to achieve perfect FoV prediction in on-demand streaming, where it is better to prefetch the video multiple seconds ahead, to absorb the network bandwidth fluctuation. This paper proposes a two-tier solution, where the base tier delivers the entire 360° span at a lower quality with a long prefetching buffer, and the enhancement tier delivers the predicted FoV at a higher quality using a short buffer. The base tier provides robustness to both network bandwidth variations and FoV prediction errors. The enhancement tier improves the video quality if it is delivered in time and FoV prediction is accurate. We study the optimal rate allocation between the two tiers and buffer provisioning for the enhancement tier to achieve the optimal trade-off between video quality and streaming robustness. We also design periodic and adaptive optimization frameworks to adapt to the bandwidth variations and FoV prediction errors in realtime. Through simulations driven by real LTE and WiGig network bandwidth traces and user FoV traces, we demonstrate that the proposed two-tier systems can achieve a high-level of Quality-of-Experience (QoE) in the face of network bandwidth and user FoV dynamics.
8 October 2018 in IEEE Transactions on Circuits and Systems for Video Technology
This paper presents a novel fast transcoding framework to efficiently bridge the state-of-art High Efficiency Video Coding (HEVC) standard and its Screen Content Coding (SCC) extension to support the bitstream compatibility over the legacy HEVC devices. By exploiting the side information from the SCC bitstream, fast mode and partition decisions are made to accurately translate the novel SCC modes to conventional HEVC modes based on statistical mode mapping techniques. Compared with the “Full-Decoding-Full-Encoding” (FDFE) solution, the proposed framework achieves on average 51% and 82% complexity reductions with 0.57% Bjontegaard-Delta Rate (BD-Rate) loss and 9.74% BD-Rate gain under All-Intra (AI) and Low-Delay (LD) configurations, respectively. Compared with the direct transcoding reusing Intra mode and Inter motion, the proposed mode mapping framework introduces additional 23% and 6% complexity reductions for AI and LD encoding configurations with 0.43% BD-Rate loss and 1.10% BD-Rate saving, respectively. The proposed solution is extended to support the Single-Input-Multiple-Output (SIMO) screen content adaptive streaming at the edge clouds, where an SCC bitstream coded in high quality is transcoded into multiple HEVC bitstreams in reduced qualities. Our proposed solution achieves on average 49% and 76% complexity reductions with 0.78% BD-Rate loss and 7.40% BD-Rate gain under AI and LD configurations, respectively.