Deep learning methods, and especially convolutional neural networks (CNNs), have made a considerable breakthrough in various fields of machine vision, basically by employing large-scale labeled databases. However, deep learning methods applied in finger-vein area are basically implemented on small-scale datasets, which are probably faced with challenges such as overfitting, susceptible to finger position, unstable performance on various datasets and son on. In this study, we present a lightweight and fully convolutional Generative Adversarial Network (GAN) architecture, which is named FCGAN, using preliminary batch normalization, and tightly-constrained loss function for implementing finger-vein image augmentation. In addition, we present a novel scheme FCGAN-CNN for finger-vein classification, which reveals that synthetic finger-vein images using FCGAN are capable of improving the property of CNN for finger-vein image classification. The experiment of sample augmentation shows that the training accuracy using FCGAN-augmented samples could go beyond 99%, which is higher than 96.34% obtained using only classic sample augmentation. Furthermore, the well-trained CNN is further evaluated on a totally different dataset, which indicates that the proposed scheme FCGAN-CNN is capable of improving the ability of CNN to extract deep features. We consider that the proposed method for sample augmentation could be extended to other biometric systems.

In the past, GAN-based face reenactment methods concentrated mostly on transferring the facial expressions and positions of the source. However, the generated results were susceptible to blurring in some minute details of the face, such as teeth and hair, and their backgrounds were also not guaranteed to be consistent with the manipulated images in terms of light and shadow. Because of these issues, the generated results could be distinguishable as fakes. In this paper, we proposed a landmark based method named HR-Net, which can render source facial expressions and postures on any identity and simultaneously generate realistic face details. Firstly, a lightweight landmark identity conversion module (LIC) was designed to address the identity leakage problem, and it represented facial expressions and poses with only 68 2D landmarks. On this basis, a boundary-guided face reenactment module (BFR) was presented to only learn the background of the reference images; thus, the results generated by BFR can be consistent with the reference images’ light and shadow. Moreover, a novel local perceptual loss function was implemented to support the BFR module in generating more realistic details. Extensive experiments demonstrated that our method achieved the state of the art.

This paper addresses the problem of Facial Expression Recognition (FER), focusing on unobvious facial movements. Traditional methods often cause overfitting problems or incomplete information due to insufficient data and manual selection of features. Instead, our proposed network, which is called the Multi-features Cooperative Deep Convolutional Network (MC-DCN), maintains focus on the overall feature of the face and the trend of key parts. The processing of video data is the first stage. The method of ensemble of regression trees (ERT) is used to obtain the overall contour of the face. Then, the attention model is used to pick up the parts of face that are more susceptible to expressions. Under the combined effect of these two methods, the image which can be called a local feature map is obtained. After that, the video data are sent to MC-DCN, containing parallel sub-networks. While the overall spatiotemporal characteristics of facial expressions are obtained through the sequence of images, the selection of keys parts can better learn the changes in facial expressions brought about by subtle facial movements. By combining local features and global features, the proposed method can acquire more information, leading to better performance. The experimental results show that MC-DCN can achieve recognition rates of 95%, 78.6% and 78.3% on the three datasets SAVEE, MMI, and edited GEMEP, respectively.

Timely feedback of students’ listening status is crucial for teaching work. However, it is often difficult for teachers to pay attention to all students at the same time. By leveraging surveillance cameras in the classroom, we are able to assist the teaching work. However, the existing methods either lack the protection of students’ privacy, or they have to reduce the accuracy of success, because they are concerned about the leakage of students’ privacy. We propose federated semi-supervised class assistance system to evaluate the listening status of students in the classroom. Rather than training the semi-supervised model in a centralized manner, we train a semi-supervised model in a federated manner among various monitors while preserving students’ privacy. We also formulate a new loss function according to the difference between the pre-trained initial model and the expected model to restrict the training process of the unlabeled data. By applying the pseudo-label assignment method on the unlabeled data, the class monitors are able to recognize the student class behavior. In addition, simulation and real-world experimental results demonstrate that the performance of the proposed system outperforms that of the baseline models.

In the field of meteorology, radiosonde data and observation data are critical for analyzing regional meteorological characteristics. Because of the high false alarm rate, severe convection forecasting is still challenging. In addition, the existing methods are difficult to use to capture the interaction of meteorological factors at the same time. In this research, a cascade of extreme gradient boosting (XGBoost) for feature transformation and a factorization machine (FM) for second-order feature interaction to capture the nonlinear interaction—XGB+FM—is proposed. An attention-based bidirectional long short-term memory (Att-Bi-LSTM) network is proposed to impute the missing data of meteorological observation stations. The problem of class imbalance is resolved by the support vector machines–synthetic minority oversampling technique (SVM-SMOTE), in which two oversampling strategies based on the support vector discrimination mechanism are proposed. It is proven that the method is effective, and the threat score (TS) is 7.27~14.28% higher than other methods. Moreover, we propose the meteorological factor selection method based on XGB+FM and improve the forecast accuracy, which is one of our contributions, as well as the forecast system.