This study introduces an FER-based machine learning framework for real-time QoE assessment in video streaming. This study’s aim is to address the challenges posed by end-to-end encryption and video advertisement while enhancing user QoE. Our proposed framework significantly outperforms the base reference, ITU-T P.1203, by up to 37.1% in terms of accuracy and 21.74% after attribute selection. Our study contributes to the field in two ways. First, we offer a promising solution to enhance user satisfaction in video streaming services via real-time user emotion and user feedback integration, providing a more holistic understanding of user experience. Second, high-quality data collection and insights are offered by collecting real data from diverse regions to minimize any potential biases and provide advertisement placement suggestions.

With end-to-end encryption for video streaming services becoming more popular, network administrators face new challenges in preserving network performance and user experience. Video ads may cause traffic congestion and poor Quality of Experience. Because of the natural variation in user interests and network situations, traditional algorithms for increasing QoE may face limitations. To solve this problem, we suggest a novel method that uses user facial emotion recognition to deduce QoE and study the effect of ads. We use open-access Face Emotion Recognition (FER) datasets and extract facial emotion information from actual observers to train machine learning models. Participants were requested to watch ad videos and provide feedback, which will be used for comparison, training, testing, and validation of our suggested technique. Our tests show that our approach beats the ITU-T P.1203 standard in terms of accuracy by 37.1%. Our method provides a hopeful answer to the problem of increasing user engagement and experience in video streaming services.

Viewer engagement with videos has consistently been low owing to excessive advertisements before and during playback, long start-up delays, pauses, blurring, poor resolution, and repetitive and intrusive advertisements. Although many well-established algorithms and techniques aim to improve user engagement, there is currently no universal solution that addresses all user requirements or adapts effectively to various network conditions. This study investigates methods for assessing user Quality of Experience (QoE) while watching ad-supported videos by analysing viewer facial expressions, video metadata, network conditions, and questionnaire feedback results. The participants in this subjective study underwent facial and screen capture, watched a series of videos with embedded advertisements, and provided feedback after every session. The collected comments and concerns were then analysed to understand how advertisements affect engagement, with the goal of making more informed advertisement placement decisions for future video streaming platforms. Our proposed method, an Adaptive Ensemble for Imbalance Classification using eXplainable Artificial Intelligence (AELIX), based on experimental data and analysis, successfully infers user QoE and offers advertisers insight into placing ads without compromising viewer engagement. AELIX achieved accuracies of 0.95 and 0.994 with the use of advanced metrics and surpassed our benchmark ITU-T P.1203 by 55.3%.

Viewer engagement in videos has consistently low due to excessive advertisements during or before playback, long startup delays, pauses, blurring, poor resolution, and repetitive, intrusive ads. Although many well-established algorithms and techniques claim to improve user engagement, there is no universal solution that addresses all user needs and adapts to varying network conditions. This study elaborates methods for assessing user Quality of Experience (QoE) while watching ad-supported videos by analyzing viewer facial expressions, video metadata, network conditions, and questionnaire feedback. Participants in this subjective study allow facial and screen capture, watch a series of videos with embedded ads, and provide feedback after each session. The collected comments and concerns are then analyzed to understand how advertisements affect engagement, with the goal of making more informed ad placement decisions for future video streaming. Our proposed method, AELIX (Adaptive Ensemble for Imbalance Classification using eXplainable artificial intelligence), based on experimental data and analysis, can infer user QoE and offer advertisers insights on placing ads without diminishing viewer engagement. AELIX achieves a no accuracy of 0.95 and 0.994 with utilization of advanced metrics. And it surpasses our benchmark, ITU-T P.1203, by 55.3%.