The weight-adapted convolution neural network (WACNN) is proposed to extract discriminative expression representations for recognizing facial expression. It aims to make good use of the convolution neural network's (CNN's) potential performance in avoiding local optima and speeding up convergence by the hybrid genetic algorithm (HGA) with optimal initial population, in such a way that it realizes deep and global emotion understanding in human-robot interaction. Moreover, the idea of novelty search is introduced to solve the deception problem in the HGA, which can expend the search space to help genetic algorithm jump out of local optimum and optimize large-scale parameters. In the proposal, the facial expression image preprocessing is conducted first, then the low-level expression features are extracted by using a principal component analysis. Finally, the high-level expression semantic features are extracted and recognized by WACNN which is optimized by HGA. In order to evaluate the effectiveness of WACNN, experiments on JAFFE, CK+, and static facial expressions in the wild 2.0 databases are carried out by using k-fold cross validation, and experimental results show the recognition accuracies of the proposal are superior to that of the state-of-the-art, such as local directional ternary pattern and weighted mixture deep neural network (DNN), which aim to extract discriminative and are the DNN-based methods. Moreover, recognition accuracies of the proposal are also higher than the deep CNN without HGA, which indicates that the proposal has better global optimization ability. Meanwhile, preliminary application experiments are also carried out by using the proposed algorithm on the emotional social robot system, where seven volunteers and two-wheeled robots experience the scenario of emotion understanding. Application results indicate that the wheeled robots can recognize basic expressions, such as happy, surprise, and so on.

AdaBoost-KNN using adaptive feature selection with direct optimization is proposed for dynamic emotion recognition in human–robot interaction, where the real-time dynamic emotion is recognized based on facial expression. It can make robots capable of understanding human dynamic emotions, in such a way that human–robot interaction is realized in a smooth manner. Based on the facial key points extracted by Candide-3 model, adaptive feature selection is adopted, namely Plus-L Minus-R Selection is completed. It can determine the features that contribute the most to emotion recognition, thereby forming the basis of emotion classification. Emotion classification is based on AdaBoost-KNN, which builds a series of KNN classifiers. AdaBoost-KNN adjusts the weights of the data in an iterative manner. Moreover, global optimal parameters are approximated with direct optimization until the recognition rate reaches its maximal value. The experimental performance of the proposal is verified by a $k$ -fold cross-validation. Results show that the recognition rate of the proposed approach is higher than that of the AdaBoost-KNN, adaptive feature selection-based AdaBoost-KNN, and AdaBoost-KNN with direct optimization. It is also higher than the rate achieved by other traditional recognition methods, such as AdaBoost, KNN, and SVM. In addition, preliminary application experiments are developed in our emotional social robot system, composed of two mobile robots. The experiments demonstrate the dynamic emotion understanding ability of robots in human–robot interaction.

Deep learning has greatly promoted the development of multi-view stereo in recent years. However, how to measure the reliability of the estimated depth map for practical applications and make reasonable depth hypothesis sampling for the cost volume building in the coarse-to-fine architecture are still unresolved crucial problems. To this end, an Uncertainty-Guided multi-view Network (UGNet) is proposed in this paper. In order to enable the network to perceive the uncertainty, an uncertainty-aware loss function is introduced, which not only can infer uncertainty implicitly in an unsupervised manner but also can reduce the bad impact of high uncertainty regions and the erroneous labels in the training set during training. Moreover, an uncertainty-based depth hypothesis sampling strategy is further proposed to adaptively determine the depth search range of each pixel for finer stages, which helps to generate more rational depth intervals compared with other methods and build more compact cost volumes without redundancy. Experimental results on DTU dataset, BlendedMVS dataset, Tanks and Temples dataset and ETH3D high-res benchmark show that our method achieves promising reconstruction results compared with other state-of-the-art methods.

A fuzzy deep neural network with sparse autoencoder (FDNNSA) is proposed for intention understanding based on human emotions and identification information (i.e., age, gender, and region), in which the Fuzzy C-Means (FCM) is used to cluster the input data, and deep neural network with sparse autoencoder (DNNSA) is designed for emotional intention understanding in human-robot interaction. It aims to make robots capable of recognizing human emotions and understanding related emotional intention, the FCM is suitable for gathering similar information so that the calculating dimensionality of DNNSA will be reduced, and the sparse autoencoder of DNNSA can make the neuron of DNNSA sparse to reduce the complexity of the network in such a way human-robot interaction is running smoothly. To validate the proposal, simulation experiments based on benchmark databases such as facial expression database of CK+, and speech emotion corpus of CASIA were completed. The experimental results show that the proposal outperforms the baseline algorithms of Softmax regression (SR), DNNSA, FCM-based SR (FSR), Softplus, Gath Geva based DNNSA (GDNNSA), and ensemble DNNSA (EDNNSA). Preliminary application experiments are performed in the development of emotional social robot system (ESRS), where volunteers experience the scenario of “drinking at the bar”. The obtained results indicate that the proposed FDNNSA can promote robot understanding of emotional intention of human.

Over the years, learning-based multi-view stereo methods have achieved great success based on their coarse-to-fine depth estimation frameworks. However, 3D CNN-based cost volume regularization inevitably leads to over-smoothing problems at object boundaries due to its smooth properties. Moreover, discrete and sparse depth hypothesis sampling exacerbates the difficulty in recovering the depth of thin structures and object boundaries. To this end, we present an Efficient edge-Preserving multi-view stereo Network (EPNet) for practical depth estimation. To keep delicate estimation at details, a Hierarchical Edge-Preserving Residual learning (HEPR) module is proposed to progressively rectify the upsampling errors and help refine multi-scale depth estimation. After that, a Cross-view Photometric Consistency (CPC) is proposed to enhance the gradient flow for detailed structures, which further boosts the estimation accuracy. Last, we design a lightweight cascade framework and inject the above two strategies into it to achieve better efficiency and performance trade-offs. Extensive experiments show that our method achieves state-of-the-art performance with fast inference speed and low memory usage. Notably, our method tops the first place on challenging Tanks and Temples advanced dataset and ETH3D high-res benchmark among all published learning-based methods. Code will be available at https://github.com/susuwj/EPNet.