Electroencephalogram (EEG) emotion recognition based on a hybrid feature extraction method in Empirical Mode Decomposition (EMD) domain combining with optimal feature selection based on Sequence Backward Selection (SBS) is proposed, which can reflect subtle information of multi-scale components of unstable and non-linear EEG signals and remove the reductant features to improve the performance of emotion recognition. The proposal is tested on DEAP dataset, in which the emotional states in the Valance dimension and Arousal dimension are classified by both K-nearest neighbor (KNN) and support vector machine (SVM), respectively. In the experiments, temporal windows of different length and three kinds of rhythms of EEG signal are taken into account for comparison, from which the results show that EEG signal with 1s temporal window achieves highest recognition accuracy of 86.46% in Valence dimension and 84.90% in Arousal dimension, respectively, which is superior to some state-of-the-art works. The proposed method would be applied to real-time emotion recognition in multimodal emotional communication based humans-robots interaction system.

The weight-adapted convolution neural network (WACNN) is proposed to extract discriminative expression representations for recognizing facial expression. It aims to make good use of the convolution neural network's (CNN's) potential performance in avoiding local optima and speeding up convergence by the hybrid genetic algorithm (HGA) with optimal initial population, in such a way that it realizes deep and global emotion understanding in human-robot interaction. Moreover, the idea of novelty search is introduced to solve the deception problem in the HGA, which can expend the search space to help genetic algorithm jump out of local optimum and optimize large-scale parameters. In the proposal, the facial expression image preprocessing is conducted first, then the low-level expression features are extracted by using a principal component analysis. Finally, the high-level expression semantic features are extracted and recognized by WACNN which is optimized by HGA. In order to evaluate the effectiveness of WACNN, experiments on JAFFE, CK+, and static facial expressions in the wild 2.0 databases are carried out by using k-fold cross validation, and experimental results show the recognition accuracies of the proposal are superior to that of the state-of-the-art, such as local directional ternary pattern and weighted mixture deep neural network (DNN), which aim to extract discriminative and are the DNN-based methods. Moreover, recognition accuracies of the proposal are also higher than the deep CNN without HGA, which indicates that the proposal has better global optimization ability. Meanwhile, preliminary application experiments are also carried out by using the proposed algorithm on the emotional social robot system, where seven volunteers and two-wheeled robots experience the scenario of emotion understanding. Application results indicate that the wheeled robots can recognize basic expressions, such as happy, surprise, and so on.

A three-layer weighted fuzzy support vector regression (TLWFSVR) model is proposed for understanding human intention, and it is based on the emotion-identification information in human-robot interaction. The TLWFSVR model consists of three layers, including adjusted weighted kernel fuzzy c-means for data clustering, fuzzy support vector regressions (FSVR) for information understanding, and weighted fusion for intention understanding. It aims to guarantee the quick convergence and satisfactory performance of the local FSVR via adjusting the weights of each feature in each cluster, in such a way that importance of different emotion-identification information is represented. Moreover, smooth human-oriented interaction can be obtained by endowing robot with human intention understanding capability. Experimental results show that the proposed TLWFSVR model obtains higher intention understanding accuracy and less computational time than that of two-layer fuzzy support vector regression, support vector regression, and back propagation neural network (BPNN), respectively. Additionally, the preliminary application experiments are performed in the developing human-robot interaction system, called emotional social robot system, where 12 volunteers and 2 mobile robots experience a scenario of “drinking at a bar.” Application results indicate that the bartender robot is able to understand customers' order intentions.

Two-layer fuzzy support vector regression-Takagi-Sugeno (TLFSVR-TS) model is proposed for emotion understanding in human-robot interaction (HRI), where the real-time dynamic emotion is recognized according to facial expression, and emotional intention understanding is obtained mainly based on human emotions and identification information. It aims to make robots capable of recognizing and understanding human emotions, in such a way that make HRI run smoothly. TLFSVR-TS considers about the priori knowledge inferred from human personal preference to reduce the uncertainty of various people, and multiple support vector regression (SVR) corresponding to different genders/provinces/ages of human to guarantee the local learning ability. Preliminary application experiments are performed in the developing emotional social robot system, where 30 volunteers experience the scenario of ``drinking in the bar.'' Results show that the proposal receives higher understanding accuracy than that of TLFSVR, kernel fuzzy c-means clustering is fused with SVR, and SVR.

This article presents a modified equivalent-input-disturbance (MEID) approach that actively rejects exogenous disturbances. A dynamic compensator is newly introduced in a conventional equivalent-input-disturbance (EID) estimator to further suppress the disturbances. The mechanism of the MEID approach to improving disturbance rejection is analyzed based on the transfer characteristic from the EID to the output of the system, and a stability condition of the MEID control system is presented and is used to design the parameters of the system. A comparison with the conventional EID approach and the disturbance-observer method through experiments demonstrates the validity of the presented method.

The emotion perception capability could endow machines with deeper feedback evaluation of human–machine interaction (HMI), which makes the experience of HMI more harmonious and natural. Currently, how to obtain the robust surface and semantic facial representation features has not been fully explored, moreover, the parameters and computation of conventional convolutional neural network models, which are adopted in facial expression recognition (FER) are too heavy, it is not conducive to the deployment of the HMI system. Therefore, this article proposes a representation reinforcement network (RRN) and transfer self-training (TST) based efficient FER method. Our designed RRN is mainly composed of two modules, i.e., surface representation reinforcement (SurRR) module and semantic representation reinforcement (SemaRR) module. SurRR module fuses the facial features in different dimensions and highlights key feature messaging nodes in feature maps. SemaRR module establishes the deep semantic representation of global facial regions on spatial and channel dimensions. Therefore, RRN has stronger feature extraction capability when the parameters and computational complexity are significantly reduced, and TST is proposed for contributing to the training effect of RRN at the condition of insufficient training data. We tested our method on the extended Cohn-Kanade dataset (CK+), Radboud faces database (RaFD), FERPLUS, and the real-world affective faces database (RAF-DB) datasets, and the recognition accuracy are 100%, 98.62%, 89.64%, and 88.72%, respectively. Furthermore, the preliminary application experiment, which demonstrates the feasibility of our method in practice during HMI.