360-degree video provides an immersive experience to end-users through Virtual Reality (VR) Head-Mounted-Displays (HMDs). However, it is not trivial to understand the Quality of Experience (QoE) of 360-degree video since user experience is influenced by various factors that affect QoE when watching a 360-degree video in VR. This manuscript presents a machine learning-based QoE prediction of 360-degree video in VR, considering the two key QoE aspects: perceptual quality and cybersickness. In addition, we proposed two new QoE-affecting factors: user’s familiarity with VR and user’s interest in 360-degree video for the QoE evaluation. To aim this, we first conduct a subjective experiment on 96 video samples and collect datasets from 29 users for perceptual quality and cybersickness. We design a new Logistic Regression (LR) based model for QoE prediction in terms of perceptual quality. The prediction accuracy of the proposed model is compared against well-known supervised machine-learning algorithms such as k-Nearest Neighbors (kNN), Support Vector Machine (SVM), and Decision Tree (DT) with respect to accuracy rate, recall, f1-score, precision, and mean absolute error (MAE). LR performs well with 86% accuracy, which is in close agreement with subjective opinion. The prediction accuracy of the proposed model is then compared with existing QoE models in terms of perceptual quality. Finally, we build a Neural Network-based model for the QoE prediction in terms of cybersickness. The proposed model performs well against the state of the art QoE prediction methods in terms of cybersickness.

Human activity recognition is an active field of research in computer vision with numerous applications. Recently, deep convolutional networks and recurrent neural networks (RNN) have received increasing attention in multimedia studies, and have yielded state-of-the-art results. In this research work, we propose a new framework which intelligently combines 3D-CNN and LSTM networks. First, we integrate discriminative information from a video into a map called a ‘motion map’ by using a deep 3-dimensional convolutional network (C3D). A motion map and the next video frame can be integrated into a new motion map, and this technique can be trained by increasing the training video length iteratively; then, the final acquired network can be used for generating the motion map of the whole video. Next, a linear weighted fusion scheme is used to fuse the network feature maps into spatio-temporal features. Finally, we use a Long-Short-Term-Memory (LSTM) encoder-decoder for final predictions. This method is simple to implement and retains discriminative and dynamic information. The improved results on benchmark public datasets prove the effectiveness and practicability of the proposed method.

360-degree Virtual Reality (VR) videos have already taken up viewers’ attention by storm. Despite the immense attractiveness and hype, VR conveys a loathsome side effect called “cybersickness” that often creates significant discomfort to the viewers. It is of great importance to evaluate the factors that induce cybersickness symptoms and its deterioration on the end user’s Quality-of-Experience (QoE) when visualizing 360-degree videos in VR. This manuscript’s intent is to subjectively investigate factors of high priority that affect a user’s QoE in terms of perceptual quality, presence, and cybersickness. The content type (fast, medium, and slow), the effect of camera motion (fixed, horizontal, and vertical), and the number of moving targets (none, single, and multiple) in a video can be the factors that may affect the QoE. The significant effect of such factors on end-user QoE under various stalling events (none, single, and multiple) is evaluated in a subjective experiment. The results from subjective experiments show a notable impact of these factors on end-user QoE. Finally, to label the viewing safety concern in VR, we propose a neural network-based QoE prediction method that can predict the degree of cybersickness influenced by 360-degree videos under various stalling events in VR. The performance accuracy of the proposed method is then compared against well-known Machine Learning (ML) algorithms and existing QoE prediction models. The proposed method achieved a 90% prediction accuracy rate and performed well against existing models and other ML methods.

The introduction of mobile edge computing (MEC) technology in unmanned aerial vehicle (UAV) networks can provide computing services for mobile users with or without communication infrastructure coverage. However, mobile users’ privacy may be leaked during the computation offloading process due to the information interaction between UAVs and the migration of computation tasks between mobile users and UAVs. To this end, we propose a secure aerial computing architecture that integrates MEC and blockchain technology for UAV networks to effectively ensure the security and privacy of computation offloading between UAVs and mobile users. Under this architecture, a problem of joint optimization of user association, UAV trajectory, block processor scheduling, and computation resource allocation is formulated to minimize the weighted sum of the energy consumption and the delay in completing computation tasks and blockchain tasks processing. To handle this intractable issue, we first decouple the optimization variables and then separate the original problem into multiple subproblems to be solved alternately. In addition, we design a block coordinate descent (BCD)-based algorithm for user association and computation resource allocation, and a successive convex approximation (SCA)-based algorithm to optimize the trajectories of UAVs. Simulation results show that the proposed algorithm has better performance.

Binaural sound source localization is an important and widely used perceptually based method and it has been applied to machine learning studies by many researchers based on head-related transfer function (HRTF). Because the HRTF is closely related to human physiological structure, the HRTFs vary between individuals. Related machine learning studies to date tend to focus on binaural localization in reverberant or noisy environments, or in conditions with multiple simultaneously active sound sources. In contrast, mismatched HRTF condition, in which the HRTFs used to generate the training and test sets are different, is rarely studied. This mismatch leads to a degradation of localization performance. A basic solution to this problem is to introduce more data to improve generalization performance, which requires a lot. However, simply increasing the data volume will result in data-inefficiency. In this paper, we propose a data-efficient method based on deep neural network (DNN) and clustering to improve binaural localization performance in the mismatched HRTF condition. Firstly, we analyze the relationship between binaural cues and the sound source localization with a classification DNN. Different HRTFs are used to generate training and test sets, respectively. On this basis, we study the localization performance of DNN model trained by each training set on different test sets. The result shows that the localization performance of the same model on different test sets is different, while the localization performance of different models on the same test set may be similar. The result also shows a clustering trend. Secondly, different HRTFs are divided into several clusters. Finally, the corresponding HRTFs of each cluster center are selected to generate a new training set and to train a more generalized DNN model. The experimental results show that the proposed method achieves better generalization performance than the baseline methods in the mismatched HRTF condition and has almost equal performance to the DNN trained with a large number of HRTFs, which means the proposed method is data-efficient.