By characterizing each image set as a nonsingular covariance matrix on the symmetric positive definite (SPD) manifold, the approaches of visual content classification with image sets have made impressive progress. However, the key challenge of unhelpfully large intraclass variability and interclass similarity of representations remains open to date. Although, several recent studies have mitigated the two problems by jointly learning the embedding mapping and the similarity metric on the original SPD manifold, their inherent shallow and linear feature transformation mechanism are not powerful enough to capture useful geometric features, especially in complex scenarios. To this end, this article explores a novel approach, termed SPD manifold deep metric learning (SMDML), for image set classification. Specifically, SMDML first selects a prevailing SPD manifold neural network (SPDNet) as the backbone (encoder) to derive an SPD matrix nonlinear representation. To counteract the degradation of structural information during multistage feature embedding, we construct a Riemannian decoder at the end of the encoder, trained by a reconstruction error term (RT), to induce the generated low-dimensional feature manifold of the hidden layer to capture the pivotal information about the visual data describing the imaged scene. We demonstrate through theory and experiments that it is feasible to replace the Riemannian metric with Euclidean distance in RT. Then, the ReCov layer is introduced into the established Riemannian network to regularize the local statistical information within each input feature matrix, which enhances the effectiveness of the learning process. The theoretical analysis of the activation function used in the ReCov layer in terms of continuity and conditions for generating positive definite matrices is beneficial for network design. Inspired by the fact that the single cross-entropy loss used for training is unable to effectively parse the geometric distribution of the deep representations, we finally endow the suggested model with a novel metric learning regularization term. By explicitly incorporating the encoding and processing of the data variations into the network learning process, this term can not only derive a powerful Riemannian representation but also train an effective classifier. The experimental results show the superiority of the proposed approach on three typical visual classification tasks.

Modern face recognition systems widely use deep convolutional neural networks (DCNNs). However, DCNNs are susceptible to adversarial examples, posing security risks to these systems. Transferable adversarial examples that can be transferred from surrogate to target models greatly undermine the robustness of DCNNs. Numerous attempts have been made to generate transferable adversarial examples, but the existing methods often suffer from limited transferability or produce adversarial examples with poor image perceptual quality. Recently, diffusion models have shown remarkable success in image generation and have excelled in various downstream tasks. However, their potential in adversarial attacks remains largely unexplored. To bridge this gap, we propose a novel approach, namely Adversarial Diffusion Attack (ADA), in generation of transferable adversarial facial examples. ADA employs a dynamic game-like strategy between injection and denoising that progressively reinforces the robustness of adversarial perturbation in the reverse process of diffusion model. Additionally, both adversarial perturbation and residual image are embedded to drift benign distribution towards adversarial distribution, crafting adversarial examples with high image quality. Extensive experimental results obtained on two benchmarking datasets, LFW and CelebA-HQ, demonstrate that ADA achieves higher attack success rates and produces adversarial examples with superior image quality compared to the state-of-the-art methods.

Adversarial examples have raised great concerns about the security of deep learning models. Substitute training makes it possible to conduct black-box substitute attacks in real-world scenarios where the attacker does not need access to the structure, parameters, and training set of the target model. However, existing substitute training methods require a large number of queries on the target model and suffer from low attack success rates. To alleviate these problems, we propose a novel black-box adversarial attack method, named substitute meta-learning (SML), which combines meta-learning with the training of the substitute model. Different from existing substitute training methods that rely on data augmentation tactics or refined loss functions, we aim to boost the learning efficiency of the substitute model to improve training efficiency and attack performance. Specifically, we introduce meta-learning to enable the substitute model to learn the knowledge of the target model using a few queries. Extensive experiments are conducted on MNIST and CIFAR-10 datasets. The experimental results show that the proposed SML can improve the attack success rate from 46.1% to 61.3% while requiring fewer queries.

Improved generative adversarial network (Improved GAN) is a successful method by using generative adversarial model to solve the problem of semi-supervised learning. Improved GAN learns a generator with the technique of mean feature matching which penalizes the discrepancy of the first order moment of the latent features. To better describe common attributes of a distribution, the paper proposes a novel semi-supervised learning method which incorporates the first order moment and the second order moment of the features in an intermediate layer of the discriminator, called mean and variance feature matching GAN(MVFM-GAN). To capture more precisely the data manifold, not only mean but also variance is used in the latent feature learning. Compared with improved gan and other traditional methods, MVFM-GAN shows its superior performance of semi-supervised classification and the stability of GAN training, particularly in the cases when the number of labelled samples is low. It shows the comparable performance with the state-of-the-art methods on several benchmark datasets. As a byproduct of the novel approach, MVFM-GAN can generate realistic images with good visual quality.

To learn disentangled representations of facial images, we present a Dual Encoder-Decoder based Generative Adversarial Network (DED-GAN). In the proposed method, both the generator and discriminator are designed with deep encoder-decoder architectures as their backbones. To be more specific, the encoder-decoder structured generator is used to learn a pose disentangled face representation, and the encoder-decoder structured discriminator is tasked to perform real/fake classification, face reconstruction, determining identity and estimating face pose. We further improve the proposed network architecture by minimizing the additional pixel-wise loss defined by the Wasserstein distance at the output of the discriminator so that the adversarial framework can be better trained. Additionally, we consider face pose variation to be continuous, rather than discrete in existing literature, to inject richer pose information into our model. The pose estimation task is formulated as a regression problem, which helps to disentangle identity information from pose variations. The proposed network is evaluated on the tasks of pose-invariant face recognition (PIFR) and face synthesis across poses. An extensive quantitative and qualitative evaluation carried out on several controlled and in-the-wild benchmarking datasets demonstrates the superiority of the proposed DED-GAN method over the state-of-the-art approaches.