Effective emotion recognition algorithms can help machines better understand people and promote the development of human-computer interaction applications. In recent years, many research efforts have used benchmark expression data to train deep neural network models to achieve state-of-art results. These high-accuracy models usually contain hundreds of layers, so they require complex calculations and may not be suitable for real-world scenarios. This paper proposes a lightweight emotion recognition (LER) model to handle the latency problem under natural conditions. The three main contributions of this paper are as follows. 1) The LER model incorporates a densely connected convolution layer and model compression techniques into a framework that eliminates redundancy parameters. 2) Multichannel input is introduced in our work to preprocess the image data, which improves the learning ability of the model. 3) Experiments show that the proposed LER model has better performance on the FER2013 and FERPLUS datasets compared with other lightweight models. Compared with the VGG13 used in previous work, the LER model achieves higher accuracy and reduces the number of parameters by 97 times. Finally, the FERFIN dataset is created, which had fewer noise data and more accurate labels than the FERPLUS dataset.

Multimodal sentiment analysis (MSA) is important for quickly and accurately understanding people’s attitudes and opinions about an event. However, existing sentiment analysis methods suffer from the dominant contribution of text modality in the dataset; this is called text dominance. In this context, we emphasize that weakening the dominant role of text modality is important for MSA tasks. To solve the above two problems, from the perspective of datasets, we first propose the Chinese multimodal opinion-level sentiment intensity (CMOSI) dataset. Three different versions of the dataset were constructed: manually proofreading subtitles, generating subtitles using machine speech transcription, and generating subtitles using human cross-language translation. The latter two versions radically weaken the dominant role of the textual model. We randomly collected 144 real videos from the Bilibili video site and manually edited 2557 clips containing emotions from them. From the perspective of network modeling, we propose a multimodal semantic enhancement network (MSEN) based on a multiheaded attention mechanism by taking advantage of the multiple versions of the CMOSI dataset. Experiments with our proposed CMOSI show that the network performs best with the text-unweakened version of the dataset. The loss of performance is minimal on both versions of the text-weakened dataset, indicating that our network can fully exploit the latent semantics in nontext patterns. In addition, we conducted model generalization experiments with MSEN on MOSI, MOSEI, and CH-SIMS datasets, and the results show that our approach is also very competitive and has good cross-language robustness.

The high incidence of traffic accidents brings immeasurable losses to life. In order to avoid such crises, researchers and automakers have used many methods to solve this problem. Among them, technology based on visual features is widely used in driver fatigue detection. As fatigue detection plays a vital role in the driving process, the high accuracy of fatigue monitoring is very important. This paper focuses on the method based on convolutional neural network to detect driver fatigue. First, in the face detection part, the Single‐Shot Multi‐Box Detector algorithm is used to improve the speed and accuracy of face detection to extract the eye and mouth regions; second, the VGG16 network is used to learn fatigue features, which is performed on the NTHU‐Drowsy Driver Detection (NTHU‐DDD) data set and the other two modified data sets Training test. The main result of this work is that the accuracy of fatigue monitoring is higher than other methods including the original method, with an accuracy rate of over 90%. And it has better generalization ability than the multi‐physical feature fusion detection method. At the same time, we propose the fatigue detection method based on convolutional neural network to improve the advanced driver assistance system (ADAS) to make it more robust and reliable decision making.

Falling is a common and dangerous event for the elderly population. Fall detection is an interesting scientific problem and an ill-defined process that can be solved through various methods. In this paper, various methods and detection algorithms based on wearable sensors are introduced. A benchmark database, namely, a fall detection database, is presented to evaluate the performance of detection algorithms. This database collects sample data from 26 males and 24 females performing 15 kinds of activities, including falls and activities of daily life, such as walking, running, and walking upstairs. The subjects comprise 50 males and females ranging from 21 to 60 years of age, 1.55 to 1.90 m in height, and 40 to 85 kg in weight. A full comparison between the existing databases and the proposed database is presented. Four baseline algorithms (the artificial neutral network, k nearest neighbor, support vector machine, and kernel Fisher discriminant) are used to evaluate the databases’ reliabilities. Evaluation protocols based on the fall detection database are discussed, and the performance of the four algorithms as a baseline are presented for the following objectives: 1) to elementarily assess the database through the fall detection algorithms; 2) to identify the strengths and weaknesses of the common algorithms; and 3) to obtain the evaluation results for two classes and multiple classes based on the database. In hospitals, it is helpful to build a fall detection system for some patients who are unconscious and unable to call for help after falling down. Therefore, this paper has implications for the construction of a medical IOT platform.