Scene images generally show the characteristics of large intra-class variety and high inter-class similarity because of complicated appearances, subtle differences, and ambiguous categorization. Hence, it is difficult to achieve a satisfactory accuracy by using a single representation. For solving this issue, we present a comprehensive representation for scene recognition by fusing deep features extracted from three discriminative views including the information of object semantics, global appearance and contextual appearance. These views show diversity and complementarity of features. The object semantics representation of the scene image, denoted by Spatial-layout-maintained Object Semantics Features (SOSF), is extracted from the output of a deep-learning-based multi-classes detector by using Spatial Fisher Vectors (SFV), which can simultaneously encode the category and layout information of objects. A multi-direction Long Short-Term Memory (LSTM)-based model is built to represent contextual information of the scene image, and, the activation of the fully connected layer of a Convolutional Neural Network (CNN) is used to represent the global appearance of scene image. These three kinds of deep features are then fused to draw a final conclusion for scene recognition. Extensive experiments are conducted to evaluate the proposed comprehensive representation on three benchmarks scene image database. The results show that the three deep features complement to each other strongly and are effective in improving recognition performance after fusion. The proposed method can achieve scene recognition accuracy of 89.51% on MIT67 database, 78.93% on SUN397 database, and 57.27% on Places365 databases, respectively, which are better percentages than the accuracies obtained by the latest reported deep-learning-based scene recognition methods.

Falls are the main cause of accidental injuries, and even death among elderly people, especially those who live alone in their homes. The absence of a reliable fall detection system has long been a serious problem for home health monitoring. A video surveillance system can be used to monitor elderly people at home to detect falls, but the traditional implementation of such intelligent detection falls short of personal privacy-related considerations; additionally, many people do not want to be watched in their homes. To solve this problem, we propose a fall detection system with visual shielding that can ensure the safety of elderly people in their homes while preserving their personal privacy. Multilayer compressed sensing is first used to achieve visually shielded video frames. By combining low-rank sparse decomposition theory with the improved local binary pattern on the three orthogonal planes, the object features are extracted from the shielded video frames. Finally, to compensate for the information lost in the compressed video to a certain extent, a private information-embedded classification model is proposed to identify fall-related behavior. The experimental results on two public fall datasets show that the proposed method delivers impressive accuracy and a low error rate while effectively distinguishing between fall- and nonfall-related behaviors in videos.

Facial expression recognition (FER) in the wild unavoidably sufferes from the effects of face posture, illumination and partial occlusion. In this paper, we attempt to alleviate the above negative effects and improve the performance of FER in the wild based on 3D face feature reconstruction and learning. 3D face reconstruction not only can effectively make up for the facial apparent information missing inform a 2D face images but can also extract accurate 3D facial geometric information in self-occlusion and extreme illumination scenarios. Therefore, we propose a novel end-to-end trainable 3D face feature reconstruction and learning network(3DF-RLN) is proposed for FER in the wild. In 3DF-RLN, the 2D implicitly frontalized face apparent data and 3D facial landmarks are reconstructed by a 3D face reconstruction module and input to two feature extraction pathways. The appearance pathway learns apparent features from the reconstructed 2D face apparent data using a convolutional neural network. The geometry pathway learns the geometric features from the reconstructed 3D facial landmarks using a graph convolutional network. Finally, FER is achieved via the fusion of the two pathways. Extensive experiments were conducted to evaluate the proposed method with three benchmark databases including Multi-PIE, RAF-DB and AffectNet. The results show that the proposed 3DF-RLN model has better FER performance, both in the lab and in the wild. In addition, the face graph from the geometry pathway reveals the correlations between facial landmarks in FER.

We propose an unsupervised cross-view facial expression adaptation network(UCFEAN) to simultaneously generate and recognize cross-view facial expressions in images in an unsupervised manner. The main idea of UCFEAN is to convert the unsupervised domain adaptation between two image spaces with different appearance into semi-supervised learning(SSL) in feature spaces with the same semantic content. The cyclic image generation of cross-view facial expressions based on the generative adversarial network(GAN) is carried out to project unlabeled target images and labelled source images to the corresponding feature spaces with the same semantic content. This helps realize the unsupervised feature learning of the target image. Labels of facial expressions represented in the projected target features can then be learned using the projected source features, because the distributions of the projected features in the two domains are close enough for knowledge transfer by using SSL. Three techniques are developed to train UCFEAN in an effective and stable manner. Extensive experiments are conducted to evaluate the UCFEAN on two multi-view facial expression image databases including RaFD and Multi-PIE. The results show that the proposed method can generate realistic target images of the facial expression and recognize cross-view facial expressions with high precision.