Heterogeneous face recognition (HFR), referring to matching face images across different domains, is a challenging problem due to the vast cross-domain discrepancy and insufficient pairwise cross-domain training data. This paper proposes a quadruplet framework for learning domain-invariant discriminative features (DIDF) for HFR, which integrates domain-level and class-level alignment in one unified network. The domain-level alignment reduces the cross-domain distribution discrepancy. The class-level alignment based on a special quadruplet loss is developed to further diminish the intra-class variations and enlarge the inter-class separability among instances, thus handling the misalignment and adversarial equilibrium problems confronted by the domain-level alignment. With a bidirectional cross-domain data selection strategy, the quadruplet loss-based method prominently enriches the training set and further eliminates the cross-modality shift. Benefiting from the joint supervision and mutual reinforcement of these two components, the domain invariance and class discrimination of identity features are guaranteed. Extensive experiments on the challenging CASIA NIR-VIS 2.0 database, the Oulu-CASIA NIR&VIS database, the BUAA-VisNir database, and the IIIT-D viewed sketch database demonstrate the effectiveness and preferable generalization capability of the proposed method.

Identity preservation is crucial for the practical application of face sketch and photo synthesis, such as law enforcement and entertainment. However, existing methods mainly focus on generating faces with good visual quality, leading the evaluation of identity preservation is partial and insufficient. Besides, it is difficult to simultaneously synthesize photos and sketches that have good visual quality and identity preservation. In this article, we propose to utilize auxiliary deep features extracted by an off-the-shelf face classifier to inject into the synthesis processes, so that the synthesized faces have more identity information. Furthermore, we propose a light interpolated convolutional neural network that has a shared encoder and two face-specialized decoders to simultaneously complete the transformations between sketches and photos. Evaluations on identity preservation and visual quality show our method is superior to existing methods in synthesizing both sketches and photos, and is qualified in practical application.

In recent years, Generative Adversarial Networks (GANs)-based illumination processing of facial images has made favorable achievements. However, some GANs-based illumination-processing methods only pay attention to the image quality and neglect the recognition accuracy, whereas others only crop partial face area and ignore the challenges to synthesize photographic face, background and hair when the original face image is under extreme illumination (Image under extreme illumination (extreme illumination conditions) means that we cannot see the texture and structure information clearly and most pixel values tend to 0 or 255.). Moreover, the recognition accuracy is low when the faces are under extreme illumination conditions. For these reasons, we present an elaborately designed architecture based on convolutional neural network and GANs for processing the illumination of facial image. We use ResBlock at the down-sampling stage in our encoder and adopt skip connections in our generator. This special design together with our loss can enhance the ability to preserve identity and generate high-quality images. Moreover, we use different convolutional layers of a pre-trained feature network to extract varisized feature maps, and then use these feature maps to compute loss, which is named multi-stage feature maps (MSFM) loss. For the sake of fairly evaluating our method against state-of-the-art models, we use four metrics to estimate the performance of illumination-processing algorithms. A variety of experimental data indicate that our method is superior to the previous models under various illumination challenges in illumination processing. We conduct qualitative and quantitative experiments on two datasets, and the experimental data indicate that our scheme obviously surpasses the state-of-the-art algorithms in image quality and identification accuracy.