A student’s interest level can strongly affect the learning process, and thus, can be considered an important factor in the effort to improve learning. Presently, student interest is primarily assessed by administering questionnaires or conducting case analyses. However, this method cannot provide timely feedback in the learning environment to allow an instructor to make immediate improvements for a more effective learning process. Hence, we designed an intelligent analysis method to analyse student interest using multimodal natural sensing technology. In this study, we present a three-dimensional (3D) learning interest model designed from an educational psychology perspective which comprehensively describes student interest in a learning environment: cognitive attention, learning emotion and thinking activity. Multimodal data are compiled by head pose estimation, facial expression recognition and interactive data collection and interpreted based on this model. Then, multimodal data fusion is conducted to comprehensively gauge student interest. Experimental testing revealed that the proposed 3D model of learning interest could objectively reflect student interest, providing an effective basis for improving teaching in real time.

In recent years, the rise of location-based service applications such as cashier-less shopping, mobile advertisement targeting, and geo-based augmented reality (AR) has been remarkable. These applications offer convenient and interactive experiences by utilizing indoor localization technology. One popular research area in indoor localization is passive fingerprinting localization based on Channel State Information (CSI), which uses general-purpose Wi-Fi platforms and “unconscious cooperative sensing” to achieve device-free localization. However, existing studies face challenges related to inadequate fingerprint richness, limited distinguishability, and inconsistent fingerprint features in real-world dynamic environments. To address these challenges, we prpose MFFLoc in this paper. MFFLoc extracts and processes amplitude and phase information from CSI in a 2D manner. It then fuses the amplitude and phase information using multimodal fusion representation, resulting in rich and distinguishable fused fingerprint features. This approach allows MFFLoc to achieve satisfactory accuracy with just one communication link, reducing deployment costs. To overcome the issue of inconsistent fingerprint features in dynamic environments, MFFLoc proposes an unsupervised domain adaptation method. It employs a dual-flow structure, with one flow operating in the source domain and the other in the target domain. The adaptation layer, with correlated weights, remains unshared between the two flows. Meta-learning is also used to automatically determine the most suitable adaptation layer. Through extensive 6-day experiments conducted in a dynamic indoor environment, MFFLoc showcases superior performance compared to state-of-the-art systems. It demonstrates higher localization accuracy and robustness, making it a promising solution for indoor localization applications.

Positive emotional experiences can improve learning efficiency and cognitive ability, stimulate students' interest in learning, and improve teacher-student relationships. However, positive emotions in the classroom are primarily identified through teachers' observations and postclass questionnaires or interviews. The expression intensity of students, which is extremely important for fine-grained emotion analysis, is not considered. Hence, a novel method based on smile intensity estimation using sequence-relative key-frame labeling is presented. This method aims to recognize the positive emotion levels of a student in an end-to-end framework. First, the intensity label is generated robustly for each frame in the expression sequence based on the relative key frames to address the lack of annotations for smile intensity. Then, a deep-asymmetric convolutional neural network learns the expression model through dual neural networks, to enhance the stability of the network model and avoid the extreme attention region learned. Further, dual neural networks and the dual attention mechanism are integrated using the intensity label based on the relative key frames as the supervised information. Thus, diverse features are effectively extracted and subtle appearance differences between different smiles are perceived based on different perspectives. Finally, comparative experiments for the convergence speed, model-training parameters, confusion matrix, and classification probability are performed. The proposed method was applied to a real classroom scene to analyze the emotions of students. Numerous experiments validated that the proposed method is promising for analyzing the differences in the positive emotion of students while learning in a classroom.

In this paper, we present a novel unsupervised feature selection method termed robust matrix factorization with robust adaptive structure learning (RMFRASL), which can select discriminative features from a large amount of multimedia data to improve the performance of classification and clustering tasks. RMFRASL integrates three models (robust matrix factorization, adaptive structure learning, and structure regularization) into a unified framework. More specifically, a robust matrix factorization-based feature selection (RMFFS) model is proposed by introducing an indicator matrix to measure the importance of features, and the L₂₁-norm is adopted as a metric to enhance the robustness of feature selection. Furthermore, a robust adaptive structure learning (RASL) model based on the self-representation capability of the samples is designed to discover the geometric structure relationships of original data. Lastly, a structure regularization (SR) term is designed on the learned graph structure, which constrains the selected features to preserve the structure information in the selected feature space. To solve the objective function of our proposed RMFRASL, an iterative optimization algorithm is proposed. By comparing our method with some state-of-the-art unsupervised feature selection approaches on several publicly available databases, the advantage of the proposed RMFRASL is demonstrated.