Face recognition has been successfully deployed in real-time applications, including secure applications such as border control. The vulnerability of face recognition systems (FRSs) to various kinds of attacks (both direct and indirect attacks) and face morphing attacks has received great interest from the biometric community. The goal of a morphing attack is to subvert an FRS at an automatic border control (ABC) gate by presenting an electronic machine-readable travel document (eMRTD) or e-passport that is obtained based on a morphed face image. Since the application process for an e-passport in the majority of countries requires a passport photograph to be presented by the applicant, a malicious actor and an accomplice can generate a morphed face image to obtain the e-passport. An e-passport with a morphed face image can be used by both the malicious actor and the accomplice to cross a border, as the morphed face image can be verified against both of them. This can result in a significant threat, as a malicious actor can cross the border without revealing the trace of his/her criminal background, while the details of the accomplice are recorded in the log of the access control system. This survey aims to present a systematic overview of the progress made in the area of face morphing in terms of both morph generation and morph detection. In this article, we describe and illustrate various aspects of face morphing attacks, including different techniques for generating morphed face images and state-of-the-art morph attack detection (MAD) algorithms based on a stringent taxonomy as well as the availability of public databases, which allow us to benchmark new MAD algorithms in a reproducible manner. The outcomes of competitions and benchmarking, vulnerability assessments, and performance evaluation metrics are also provided in a comprehensive manner. Furthermore, we discuss the open challenges and potential future areas that need to be addressed in the evolving field of biometrics.

Face morphing attacks target to circumvent Face Recognition Systems (FRS) by employing face images derived from multiple data subjects (e.g., accomplices and malicious actors). Morphed images can be verified against contributing data subjects with a reasonable success rate, given they have a high degree of facial resemblance. The success of morphing attacks is directly dependent on the quality of the generated morph images. We present a new approach for generating strong attacks extending our earlier framework for generating face morphs. We present a new approach using an Identity Prior Driven Generative Adversarial Network, which we refer to as MIPGAN (Morphing through Identity Prior driven GAN). The proposed MIPGAN is derived from the StyleGAN with a newly formulated loss function exploiting perceptual quality and identity factor to generate a high quality morphed facial image with minimal artefacts and with high resolution. We demonstrate the proposed approach's applicability to generate strong morphing attacks by evaluating its vulnerability against both commercial and deep learning based Face Recognition System (FRS) and demonstrate the success rate of attacks. Extensive experiments are carried out to assess the FRS's vulnerability against the proposed morphed face generation technique on three types of data such as digital images, re-digitized (printed and scanned) images, and compressed images after re-digitization from newly generated MIPGAN Face Morph Dataset. The obtained results demonstrate that the proposed approach of morph generation poses a high threat to FRS.

Morphing attacks have posed a severe threat to Face Recognition System (FRS). Despite the number of advancements reported in recent works, we note serious open issues such as independent benchmarking, generalizability challenges and considerations to age, gender, ethnicity that are inadequately addressed. Morphing Attack Detection (MAD) algorithms often are prone to generalization challenges as they are database dependent. The existing databases, mostly of semi-public nature, lack in diversity in terms of ethnicity, various morphing process and post-processing pipelines. Further, they do not reflect a realistic operational scenario for Automated Border Control (ABC) and do not provide a basis to test MAD on unseen data, in order to benchmark the robustness of algorithms. In this work, we present a new sequestered dataset for facilitating the advancements of MAD where the algorithms can be tested on unseen data in an effort to better generalize. The newly constructed dataset consists of facial images from 150 subjects from various ethnicities, age-groups and both genders. In order to challenge the existing MAD algorithms, the morphed images are with careful subject pre-selection created from the contributing images, and further post-processed to remove morphing artifacts. The images are also printed and scanned to remove all digital cues and to simulate a realistic challenge for MAD algorithms. Further, we present a new online evaluation platform to test algorithms on sequestered data. With the platform we can benchmark the morph detection performance and study the generalization ability. This work also presents a detailed analysis on various subsets of sequestered data and outlines open challenges for future directions in MAD research.

Face verification is a popular way for verifying identities in access control systems. In this work, a partial face manipulation-based morphing attack (MA) is proposed to compromise the uniqueness of face templates. Different from existing research, this work changes MA from a holistic face level to component level, and only the most effective facial components (eyes and nose) are used. Therefore, a manipulated face is more similar to a bona fide one in terms of visual quality, texture, and noise characteristics. To validate the effectiveness of the proposed attack, a novel metric called actual mated morph presentation match rate (AMPMR) is proposed to evaluate MA performance under real-world conditions. With a collected dataset containing different attack types, image qualities, and manipulation parameters, the results indicate the proposed attack has better anti-detectability compared with the existing complete, splicing, and combined MAs. Moreover, it has low visual distortion and can reach a better tradeoff among facial biometrics verification, anti-detectability, and visual differences.

Automatic deception detection from the video has gained a paramount of interest because of their applicability in various real-life applications. The recorded videos contain various information such as temporal variations of the face, linguistics and acoustics, which can be used together, to detect deception automatically. In this work, we proposed a new approach based on multimodal information like audio, linguistic (or text) and non-verbal features. The proposed multimodal deception detection framework is based on combining the decision from the audio, text and non-verbal features using majority voting. The proposed multimodal deception system is banked on the audio system based on Cepstral Coefficients (CC) and Spectral Regression Kernel Discriminant Analysis (SRKDA) of fixed length audio sequences. The text system is based on bag-of-n-grams features and the linear Support Vector Machine (SVM) classifier while the non-verbal features are classified using the AdaBoost classifier. Extensive experiments are carried out on a publicly available real-life deception video dataset to evaluate the efficacy of the proposed scheme. The obtained results on a 25-cross-fold validation have indicated a deception detection accuracy of 97% out-performing both state-of-the-art techniques and human performance on the whole dataset.

The vulnerability of face recognition systems towards evolving presentation attacks has drawn significant interest in the last decade. In this paper, we present an empirical study on both vulnerability analysis and presentation attack detection for commercial face recognition systems (FRS) using custom 3D silicone face masks corresponding to real subjects. To this end, a new database is collected consisting of 8 custom 3D silicone masks together with bona fide presentations of the corresponding subjects using three different devices (smartphones). The vulnerability of FRS for 3D face silicone face masks is effectively evaluated using two well-known commercial-off-the-shelf (COTS) FRS (Verilook from Neurotechnology, and the Cognitec Face-VACS). Further, extensive experiments are carried out to evaluate the effectiveness of five state-of-the-art presentation attack detection (PAD) techniques for detecting such masks. Key insights on silicone mask PAD are provided along with a discussion on the accuracy achieved in our experiments.