23 November 2021 in IEEE Transactions on Circuits and Systems for Video Technology
As face editing scenarios have become popular, the face inpainting technique has become a hot topic. Although some existing methods can inpaint faces with preserved identity information, they fail to solve a more flexible inpainting problem that fills the “holes” with identity-specific content from other faces. In this work, we propose a disentangle and subject-agnostic framework that affects both full and partial-face inpainting with the guidance of a reference face image. The framework consists of an identity encoding module, a content inference module and a generative module. The identity encoding module extracts the identity embedding from the reference image, the content inference module learns to predict the content image, and the generative module integrates the content image and the reference identity embedding to generate the identity-specific inpainted result. To minimize the structure and style gap between the incomplete image and inpainted image, we use a double attribute loss to the generative module and a postprocess of blending operation to the swapped result. We compare our method with state-of-the-art works and demonstrate that our method achieves higher identity similarity and better structural correctness.
13 July 2020 in IEEE Access
In this paper, we propose a novel deep generative model for image animation synthesis. Based on self-supervised learning and adversarial training, the model can find labeling rules and mark them without origin sample labels. In addition, our model can generate continuous changing images based on the automatically labels learning. The labels learning model can be implemented on a large number of out-of-order samples to generate two types of pseudo-labels, discrete labels and continuous labels. The discrete labels can generate different animation clips, and the continuous labels can generate different frames in the same clip. Embedding pseudo-labels with latent variables into latent space, our model discovers regularities and features from latent space. Animation features are fully characterized by the pseudo-labels learned from the self-supervised module. Using upgraded adversarial training steps, the model learns to map animation features to pseudo-labels from the latent space and then organizes pseudo-labels embedding into latent variables to generate animation features. By adapting dimensions of pseudo-labels, we match fine features with latent variables. Such as using the two types of pseudo-labels, our model can also generate different styles of videos from the same dataset. The specific implementation tricks depend on the different pseudo-label dimensions and the number of pseudo-label dimensions. Comparing the results of our model with other state-of-the-art approaches, the model does not use complicated components, such as 3D convolution layers and recurrent neural networks. Our experimental results show that an appropriate number of the pseudo-label dimensions can better characterize animation features. In this case, an animation which reached human-level perception can be synthesized. The performance of animation synthesis has reached relatively superior results on several challenging datasets.
28 November 2025 in CAAI Transactions on Intelligence Technology
Discovering meaningful face morphing is critical for applications in image synthesis. Traditional unsupervised methods rely on global or layer‐wise representations, neglecting finer local details and thus limiting the control over specific facial attributes. In this work, we introduce an improved unsupervised approach that leverages contrastive learning and K‐means clustering to learn both layer‐wise and local features (LLF) in the latent space of StyleGAN. Our method segments latent representations into multiple local components across different layers, enabling fine‐grained control over attributes such as hair, eyes, and mouth. Experimental results demonstrate that LLF outperforms existing methods by providing more interpretable facial transformations while preserving high image realism, offering a promising solution for enhanced unsupervised face morphing applications. The code is available at https://github.com/disanda/LLF .
31 August 2021 in TechRxiv
Finely tuned deep generative adversarial networks (GANs) can generate high-quality (HQ) images. However, the discriminator in GAN is only able to distinguish true or fake images. Moreover, numerous synthesized images from GANs are imperfect, and we can not reconstruct those images via GANs. In this paper, we revisit pre-trained GANs and offer a self-supervised method to quickly transform GAN’s discriminators into encoders. We reuse parameters of the GAN’s discriminator and replace its output layer, so that it can be transformed into an encoder and output reformed latent vectors. The transformation makes the pre-trained GAN a symmetrical architecture and allows for better performance. Based on the method, GANs can be made to reconstruct synthesized images via encoders. Compared to synthesized images, these reconstructions can maintain or even attain higher quality.
9 September 2021 in arXiv
Vehicle Re-identification (ReID) is an important yet challenging problem in computer vision. Compared to other visual objects like faces and persons, vehicles simultaneously exhibit much larger intraclass viewpoint variations and interclass visual similarities, making most exiting loss functions designed for face recognition and person ReID unsuitable for vehicle ReID. To obtain a high-performance vehicle ReID model, we present a novel Distance Shrinking with Angular Marginalizing (DSAM) loss function to perform hybrid learning in both the Original Feature Space (OFS) and the Feature Angular Space (FAS) using the local verification and the global identification information. Specifically, it shrinks the distance between samples of the same class locally in the Original Feature Space while keeps samples of different classes far away in the Feature Angular Space. The shrinking and marginalizing operations are performed during each iteration of the training process and are suitable for different SoftMax based loss functions. We evaluate the DSAM loss function on three large vehicle ReID datasets with detailed analyses and extensive comparisons with many competing vehicle ReID methods. Experimental results show that our DSAM loss enhances the SoftMax loss by a large margin on the PKU-VD1-Large dataset: 10.41% for mAP, 5.29% for cmc1, and 4.60% for cmc5. Moreover, the mAP is increased by 9.34% on the PKU-VehicleID dataset and 6.13% on the VeRi-776 dataset. Source code will be released to facilitate further studies in this research direction.