Wenxia Yang

Dr. Wenxia Yang


Research Focus & Metrics

Academic Career Length

7Years (2020-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
28 March 2024 in IEEE Transactions on Multimedia
Verified
Journal Article
Closed access
CZCong Zhang
Cong Zhang
WYWenxia Yang
Wenxia Yang
XLXin Li
Xin Li
HHHuan Han
Huan Han
Proper inference of semantics is necessary for realistic image inpainting. Most image inpainting methods use deep generative models, which require large image datasets to predict and generate content. However, predicting the missing regions and generating coherent content is difficult due to limited control. Existing approaches include image-guided or text-guided image inpainting, but none of them has taken both image and text as the guidance signals, as far as we know. To fill this gap, we propose a multi-modality guided (MMG) image inpainting approach based on the diffusion model. This MMGInpainting method uses both image and text as guidance for generating content within the target area for inpainting, effectively integrating the semantic information conveyed by the guiding image or text into the content of the inpainted region. To construct MMGInpainting, we start by enhancing the U-Net backbone with a customized Nonlinear Activation Free Network (NAFNet). This adapted NAFNet incorporates an Anchored Stripe Attention mechanism, which utilizes anchor points to effectively model global contextual dependencies. To regulate inpainting, we use a Semantic Fusion Encoder to guide the inverse process of the diffusion model. The process is iteratively executed to denoise and generate the desired inpainting result. Additionally, we explore how different modes of meaning interact and coordinate to offer users useful guidance for a more manageable inpainting procedure. Experimental results demonstrate that our approach produces faithful results adhering to the guiding information, while significantly improving computational efficiency. Github Repository: https://github.com/skipper-zc/MMGInpainting/

1 May 2020 in Environmental Technology & Innovation
Verified
Journal Article
Closed access
WYWenxia Yang
Wenxia Yang
Tony HadibarataTony Hadibarata
Tony Hadibarata
AMAhmed Hossam Mahmoud
Ahmed Hossam Mahmoud
AYAdhi Yuniarto
Adhi Yuniarto

30 January 2023 in Applied Sciences
Verified
Journal Article
Open access
PDF file
ZCZongze Chen
Zongze Chen
WYWenxia Yang
Wenxia Yang
XLXin Li
Xin Li
Chinese characters carry a wealth of morphological and semantic information; therefore, zero-shot Chinese character recognition with the morphology of Chinese characters has drawn significant attention. The previous methods are mainly based on radical-level decomposition or stroke-level decomposition, which usually cannot capture adequately the structural and spatial information of Chinese characters. In this paper, we develop a stroke-based autoencoder (SAE), to model the sophisticated morphology of Chinese characters with a self-supervised method. Following its canonical writing order, we first represent a Chinese character as a series of stroke images with a fixed writing order, and then our SAE model is trained to reconstruct this stroke image sequence. This pre-trained SAE model can predict the stroke image series for unseen characters, as long as their strokes or radicals are in the training set. We have designed two contrasting SAE architectures on different forms of stroke images. One is fine-tuned on existing stroke-based method for zero-shot recognition of handwritten Chinese characters, and the other is applied to enrich the Chinese word embeddings from their morphological features. The experimental results validate that after pre-training, our SAE architecture outperforms other existing methods in zero-shot recognition and enhances the representation of Chinese characters with their abundant morphological and semantic information.

28 June 2023 in Applied Sciences
Verified
Journal Article
Open access
PDF file
YCYueqiao Chen
Yueqiao Chen
WYWenxia Yang
Wenxia Yang
FXFang Xi
Fang Xi
HHHuan Han
Huan Han
Image completion methods based on deep learning, such as Variational Autoencoders (VAEs) and Generative Adversarial Networks (GANs), have succeeded in producing semantically plausible results. However, existing facial image completion methods can either produce only one result or, although they can provide multiple results, cannot attribute particular emotions to the results. We propose EC-GAN, a novel facial Emotion-Controllable GAN-based image completion model that can infer and customize generative facial emotions. We propose an emotion inference module that infers the emotions of faces based on the unmasked regions of the faces. The emotion inference module is trained in a supervised manner and enforces the encoder to disentangle the emotion semantics from the native latent space. We also developed an emotion control module to modify the latent codes of emotions, moving the latent codes of the initial emotion toward the desired one while maintaining the remaining facial features. Extensive experiments were conducted on two facial datasets, CelebA-HQ and CFEED. Quantitative and qualitative results indicate that EC-GAN produces images with diverse desired expressions even when the main features of the faces are masked. On the other hand, EC-GAN promotes semantic inference capability with irregularly masked holes, resulting in more natural facial expressions.

15 August 2024 in Highlights in Science, Engineering and Technology
Verified
Journal Article
Open access
PDF file
YSYizuo Shi
Yizuo Shi
WYWenxia Yang
Wenxia Yang
MYMengxu Yuan
Mengxu Yuan
ZYZeping Yi
Zeping Yi
Generating Chinese characters via few-shot font generation is an intriguing and important challenge in recent years, primarily due to the intricate and unique nature of Chinese fonts. However, the conventional GAN-based model for font generation has encountered issues such as unpredictable training and inaccurate generation. Simultaneously, in the realm of image generation, diffusion models have demonstrated remarkable success, even garnering application in AI painting commercials. Some studies have endeavored to integrate diffusion models into Few-shot Font Generation (FFG). In this paper, we present a straightforward, few-shot font generation framework utilizing a conditional diffusion model. We generate conditional embedding tokens using three encoders, which extract essential character information such as content and style. By combining these conditions into the diffusion process, we can effectively model these three pieces of information. Our model possesses three key features: i) Our model attains disentanglement of all encoders and the diffusion model. The content encoder focuses solely on extracting the content or the relative position of strokes, the style-coding provides only style features, and the diffusion model is limited to generating the target image without obscuring any content or style information. This enhances the model’s interpretability and makes the addition of new functionalities a simpler process. ii) For different fonts, our model requires fewer training steps due to the use of pre-training. We only train the style-coding on a small scale, bypassing the need for extensive training of the large-scale diffusion model. iii) Our model achieves two types of "Few-shot" training. The first type involves the same style but different characters, requiring only a few characters for training. The second type pertains to different styles, needing only a few style fonts for training. Experimental results reveal that our model outperforms previous few-font generation models in terms of quality, generation speed, and the scale of well-trained training datasets.