Self-supervised learning for semi-supervised facial expression recognition aims to avoid the need to collect expensive labeled facial expression data. Existing methods demonstrate an impressive performance boost, but they artificially assume that small labeled facial expression data and large unlabeled facial expression data are from the same class distribution. In a more realistic scenario, when utilizing facial expression data from a large face recognition database as unlabeled data, there will be a mismatch distribution between the two sets of data. This often results in severe performance degradation due to incorrect propagation of unlabeled data from unrelated sources. In this paper, we address the class distribution mismatch problem in deep semi-supervised learning-based facial expression recognition. Specifically, we propose a silhouette coefficient-based contrast clustering algorithm, which determines the degree of separation between clusters by examining the intra-cluster and inter-cluster distances to accurately detect out-of-distribution data. In addition, we propose a pseudo-labeling rethinking strategy that matches the soft pseudo-labels estimated from a fine-tuned network to the contrast clustering to produce reliable pseudo-labels. Experiments on three in-the-wild datasets, RAF-DB, FERPlus and AffectNet, demonstrate the effectiveness of our method, and our approach performs well compared to state-of-the-art methods.
Facial expression analysis aims to understand human emotions by analyzing visual face information and is a popular topic in the computer vision community. In educational research, the analyzed students’ affect states can be used by faculty members as feedback to improve their teaching style and strategy so that the learning rate of all the students present can be enhanced. Facial expression analysis has attracted much attention in educational research, and a few reviews on this topic have emerged. However, previous reviews on facial expression recognition methods in educational research focus mostly on summarizing the existing literature on emotion models from a theoretical perspective, neglecting technical summaries of facial expression recognition. In order to advance the development of facial expression analysis in educational research, this paper outlines the tasks, progress, challenges, and future trends related to facial expression analysis. First, facial expression recognition methods in educational research lack an overall framework. Second, studies based on the latest machine learning methods are not mentioned in previous reviews. Finally, some key challenges have not been fully explored. Therefore, unlike previous reviews, this systematic review summarizes two kinds of educational research methods based on facial expression recognition and their application scenarios. Then, an overall framework is proposed, along with various kinds of machine learning methods and published datasets. Finally, the key challenges of face occlusion and the expression uncertainty problem are presented. This study aims to capture the full picture of facial expression recognition methods in educational research from a machine learning perspective.
The development of artificial intelligence (AI) significantly improves the effectiveness of classroom dialogue systems, but their integration into the learning environment remains challenging. To address this gap, this research presents a framework for automatic intelligent dialogue analysis, intending to promote high-quality classroom dialogue and facilitate teaching and learning. The proposed framework includes two main components: a dialogue-oriented interactive classroom and an artificial intelligence-powered analysis system. We present a synthesis of essential principles that ought to be adhered to in the dialogue-oriented interactive classroom, as viewed through the lens of three key domains: the environment, the community and the teaching–learning. The AI system will analyse the dialogues generated from the interactive classroom. The utilization of feedback obtained from the AI system assists educators who adjust their pedagogical strategies, consequently improving the quality of classroom dialogues. Elevated-quality dialogues will reciprocally boost the performance of the AI system, engendering a sustainable improvement for the entire framework. Moreover, we also propose “Guide of AI”, a union of classroom participants and experts, which serves as the bridge between the classroom and technology to guide the operation of AI system. For the validation of the framework, we conduct an empirical study that mainly investigates the effectiveness of processed essential principles and AI systems. We select 6 pre-service teachers who are randomly divided into three groups. Three groups have different levels of involvement in AI system and each teacher gives three lessons. We record and analyse all teaching dialogue records and also use questionnaires to obtain teachers’ attitudes. The results show that timely feedback from AI system can promote the improvement of dialogue quality, which demonstrates the effectiveness of AI dialogue analysis system. In addition, the proposed essential principles also show a constructive impact.
Hyperspectral image (HSI) classification has been marked by exceptional progress in recent years. Much of this progess has come from advances in convolutional neural networks (CNNs). Different from the RGB images, HSI images are captured by various remote sensors with different spectral configurations. Moreover, each HSI dataset only contains very limited training samples and thus the model is prone to overfitting when using deep CNNs. In this paper, we first propose a 3D asymmetric inception network, AINet, to overcome the overfitting problem. With the emphasis on spectral signatures over spatial contexts of HSI data, the 3D convolution layer of AINet is replaced with two asymmetric inception units, i.e., a space inception unit and spectrum inception unit, to convey and classify the features effectively. In addition, we exploited a data-fusion transfer learning strategy to improve model initialization and classification performance. Extensive experiments show that the proposed approach outperforms all of the state-of-the-art methods via several HSI benchmarks, including Pavia University, Indian Pines and Kennedy Space Center (KSC).
The recognition of human activity is crucial as the Internet of Things (IoT) progresses toward future smart homes. Wi-Fi-based motion-recognition stands out due to its non-contact nature and widespread applicability. However, the channel state information (CSI) related to human movement in indoor environments changes with the direction of movement, which poses challenges for existing Wi-Fi movement-recognition methods. These challenges include limited directions of movement that can be detected, short detection distances, and inaccurate feature extraction, all of which significantly constrain the wide-scale application of Wi-Fi action-recognition. To address this issue, we propose a direction-independent CSI fusion and sharing model named CSI-F, one which combines Convolutional Neural Networks (CNN) and Gated Recurrent Units (GRU). Specifically, we have introduced a series of signal-processing techniques that utilize antenna diversity to eliminate random phase shifts, thereby removing noise influences unrelated to motion information. Later, by amplifying the Doppler frequency shift effect through cyclic actions and generating a spectrogram, we further enhance the impact of actions on CSI. To demonstrate the effectiveness of this method, we conducted experiments on datasets collected in natural environments. We confirmed that the superposition of periodic actions on CSI can improve the accuracy of the process. CSI-F can achieve higher recognition accuracy compared with other methods and a monitoring coverage of up to 6 m.