Face recognition (FR) systems have a growing effect on critical decision-making processes. Recent works have shown that FR solutions show strong performance differences based on the user’s demographics. However, to enable a trustworthy FR technology, it is essential to know the influence of an extended range of facial attributes on FR beyond demographics. Therefore, in this work, we analyze FR bias over a wide range of attributes. We investigate the influence of 47 attributes on the verification performance of two popular FR models. The experiments were performed on the publicly available MAAD-Face attribute database with over 120M high-quality attribute annotations. To prevent misleading statements about biased performances, we introduced control group-based validity values to decide if unbalanced test data causes the performance differences. The results demonstrate that also many nondemographic attributes strongly affect recognition performance, such as accessories, hairstyles and colors, face shapes, or facial anomalies. The observations of this work show the strong need for further advances in making the FR system more robust, explainable, and fair. Moreover, our findings might help to a better understanding of how FR networks work, enhance the robustness of these networks, and develop more generalized bias-mitigating FR solutions.

Investigating new methods of creating face morphing attacks is essential to foresee novel attacks and help mitigate them. Creating morphing attacks is commonly either performed on the image-level or on the representation-level. The representation-level morphing has been performed so far based on generative adversarial networks (GAN) where the encoded images are interpolated in the latent space to produce a morphed image based on the interpolated vector. Such a process was constrained by the limited reconstruction fidelity of GAN architectures. Recent advances in the diffusion autoencoder models have overcome the GAN limitations, leading to high reconstruction fidelity. This theoretically makes them a perfect candidate to perform representation-level face morphing. This work investigates using diffusion autoencoders to create face morphing attacks by comparing them to a wide range of image-level and representation-level morphs. Our vulnerability analyses on four state-of-the-art face recognition models have shown that such models are highly vulnerable to the created attacks, the MorDIFF, especially when compared to existing representation-level morphs. Detailed detectability analyses are also performed on the MorDIFF, showing that they are as challenging to detect as other morphing attacks created on the image- or representation-level. Data and morphing script are made public ¹ .

In this work, we introduce the concept of pixel-level face image quality that determines the utility of single pixels in a face image for recognition. We propose a training-free approach to assess the pixel-level qualities of a face image given an arbitrary face recognition network. To achieve this, a model-specific quality value of the input image is estimated and used to build a sample-specific quality regression model. Based on this model, quality-based gradients are back-propagated and converted into pixel-level quality estimates. In the experiments, we qualitatively and quantitatively investigated the meaningfulness of our proposed pixel-level qualities based on real and artificial disturbances and by comparing the explanation maps on faces incompliant with the ICAO standards. In all scenarios, the results demonstrate that the proposed solution produces meaningful pixel-level qualities enhancing the interpretability of the face image and its quality. The code is publicly available.

The use of synthetic data for training neural networks has recently received increased attention, especially in the area of face recognition. This was mainly motivated by the increase of privacy, ethical, and legal concerns of using privacy-sensitive authentic data to train face recognition models. Many authentic datasets such as MS-Celeb-1M or VGGFace2 that have been widely used to train state-of-the-art deep face recognition models are retracted and officially no longer maintained or provided by official sources as they often have been collected without explicit consent. Toward this end, we first propose a synthetic face generation approach, SFace which utilizes a class-conditional generative adversarial network to generate class-labeled synthetic face images. To evaluate the privacy aspect of using such synthetic data in face recognition development, we provide an extensive evaluation of the identity relation between the generated synthetic dataset and the original authentic dataset used to train the generative model. The investigation proved that the associated identity of the authentic dataset to the one with the same class label in the synthetic dataset is hardly possible, strengthening the possibility for privacy-aware face recognition training. We then propose three different learning strategies to train the face recognition model on our privacy-friendly dataset, SFace, and report the results on five authentic benchmarks, demonstrating its high potential. Noticing the relatively low (in comparison to authentic data) identity discrimination in SFace, we started by analysing the w-space of the class-conditional generator, finding identity information that is highly correlated to that in the embedding space. Based on this finding, we proposed an approach that performs the sampling in the w-space driven to generate data with higher identity discrimination, the SFace2. Our experiments showed the disentanglement of the latent w-space and the benefit of training face recognition models on the more identity-discriminated synthetic dataset SFace2.

Morphing attacks are one of the many threats that are constantly affecting deep face recognition systems. It consists of selecting two faces from different individuals and fusing them into a final image that contains the identity information of both. In this work, we propose a novel regularisation term that takes into account the existent identity information in both and promotes the creation of two orthogonal latent vectors. We evaluate our proposed method (OrthoMAD) in five different types of morphing in the FRLL dataset and evaluate the performance of our model when trained on five distinct datasets. With a small ResNet-18 as the backbone, we achieve state-of-the-art results in the majority of the experiments, and competitive results in the others.