Xueting Liu

Dr. Xueting Liu


Research Focus & Metrics

Academic Career Length

5Years (2022-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
23 January 2024 in IEEE Transactions on Visualization and Computer Graphics
Verified
Journal Article
Closed access
YCYu Cao
Yu Cao
XMXiangqiao Meng
Xiangqiao Meng
PMP. Y. Mok
P. Y. Mok
TLTong-Yee Lee
Tong-Yee Lee
XLXueting Liu
Xueting Liu
PLPing Li
Ping Li
Being essential in animation creation, colorizing anime line drawings is usually a tedious and time-consuming manual task. Reference-based line drawing colorization provides an intuitive way to automatically colorize target line drawings using reference images. The prevailing approaches are based on generative adversarial networks (GANs), yet these methods still cannot generate high-quality results comparable to manually-colored ones. In this paper, a new AnimeDiffusion approach is proposed via hybrid diffusions for the automatic colorization of anime face line drawings. This is the first attempt to utilize the diffusion model for reference-based colorization, which demands a high level of control over the image synthesis process. To do so, a hybrid end-to-end training strategy is designed, including phase 1 for training diffusion model with classifier-free guidance and phase 2 for efficiently updating color tone with a target reference colored image. The model learns denoising and structure-capturing ability in phase 1, and in phase 2, the model learns more accurate color information. Utilizing our hybrid training strategy, the network convergence speed is accelerated, and the colorization performance is improved. Our AnimeDiffusion generates colorization results with semantic correspondence and color consistency. In addition, the model has a certain generalization performance for line drawings of different line styles. To train and evaluate colorization methods, an anime face line drawing colorization benchmark dataset, containing 31,696 training data and 579 testing data, is introduced and shared. Extensive experiments and user studies have demonstrated that our proposed AnimeDiffusion outperforms state-of-the-art GAN-based methods and another diffusion-based model, both quantitatively and qualitatively.

1 March 2022 in Computational Visual Media
Verified
Journal Article
Open access
PDF file
XLXueting Liu
Xueting Liu
WWWenliang Wu
Wenliang Wu
CLChengze Li
Chengze Li
YLYifan Li
Yifan Li
HWHuisi Wu
Huisi Wu
Digital cartoon production requires extensive manual labor to colorize sketches with visually pleasant color composition and color shading. During colorization, the artist usually takes an existing cartoon image as color guidance, particularly when colorizing related characters or an animation sequence. Reference-guided colorization is more intuitive than colorization with other hints, such as color points or scribbles, or text-based hints. Unfortunately, reference-guided colorization is challenging since the style of the colorized image should match the style of the reference image in terms of both global color composition and local color shading. In this paper, we propose a novel learning-based framework which colorizes a sketch based on a color style feature extracted from a reference color image. Our framework contains a color style extractor to extract the color feature from a color image, a colorization network to generate multi-scale output images by combining a sketch and a color feature, and a multi-scale discriminator to improve the reality of the output image. Extensive qualitative and quantitative evaluations show that our method outperforms existing methods, providing both superior visual quality and style reference consistency in the task of reference-based colorization.

12 May 2022 in IEEE Transactions on Visualization and Computer Graphics
Verified
Journal Article
Closed access
CXCheng Xu
Cheng Xu
WQWei Qu
Wei Qu
XXXuemiao Xu
Xuemiao Xu
XLXueting Liu
Xueting Liu
Occluding effects have been frequently used to present weather conditions and environments in cartoon animations, such as raining, snowing, moving leaves, and moving petals. While these effects greatly enrich the visual appeal of the cartoon animations, they may also cause undesired occlusions on the content area, which significantly complicate the analysis and processing of the cartoon animations. In this article, we make the first attempt to separate the occluding effects and content for cartoon animations. The major challenge of this problem is that, unlike natural effects that are realistic and small-sized, the effects of cartoons are usually stylistic and large-sized. Besides, effects in cartoons are manually drawn, so their motions are more unpredictable than realistic effects. To separate occluding effects and content for cartoon animations, we propose to leverage the difference in the motion patterns of the effects and the content, and capture the locations of the effects based on a multi-scale flow-based effect prediction (MFEP) module. A dual-task learning system is designed to extract the effect video and reconstruct the effect-removed content video at the same time. We apply our method on a large number of cartoon videos of different content and effects. Experiments show that our method significantly outperforms the existing methods. We further demonstrate how the separated effects and content facilitate the analysis and processing of cartoon videos through different applications, including segmentation, inpainting, and effect migration.

12 December 2022 in IEEE Transactions on Visualization and Computer Graphics
Verified
Journal Article
Closed access
WXWenpeng Xiao
Wenpeng Xiao
CXCheng Xu
Cheng Xu
JMJiajie Mai
Jiajie Mai
XXXuemiao Xu
Xuemiao Xu
YLYue Li
Yue Li
SHShengfeng He
Shengfeng He
Converting a human portrait to anime style is a desirable but challenging problem. Existing methods fail to resolve this problem due to the large inherent gap between two domains that cannot be overcome by a simple direct mapping. For this reason, these methods struggle to preserve the appearance features in the original photo. In this paper, we discover an intermediate domain, the coser portrait (portraits of humans costuming as anime characters), that helps bridge this gap. It alleviates the learning ambiguity and loosens the mapping difficulty in a progressive manner. Specifically, we start from learning the mapping between coser and anime portraits, and present a proxy-guided domain adaptation learning scheme with three progressive adaptation stages to shift the initial model to the human portrait domain. In this way, our model can generate visually pleasant anime portraits with well-preserved appearances given the human portrait. Our model adopts a disentangled design by breaking down the translation problem into two specific subtasks of face deformation and portrait stylization. This further elevates the generation quality. Extensive experimental results show that our model can achieve visually compelling translation with better appearance preservation and perform favorably against the existing methods both qualitatively and quantitatively. Our code and datasets are available at https://github.com/NeverGiveU/PDA-Translation .

28 April 2023 in IEEE Transactions on Image Processing
Verified
Journal Article
Closed access
CXChenshu Xu
Chenshu Xu
XXXuemiao Xu
Xuemiao Xu
NZNanxuan Zhao
Nanxuan Zhao
Maxwell TsaiMaxwell Tsai
Maxwell Tsai
Huaidong ZhangHuaidong Zhang
Huaidong Zhang
XLXueting Liu
Xueting Liu
Using a sequence of discrete still images to tell a story or introduce a process has become a tradition in the field of digital visual media. With the surge in these media and the requirements in downstream tasks, acquiring their main topic or genre in a very short time is urgently needed. As a representative form of the media, comic enjoys a huge boom as it has gone digital. However, different from natural images, comic images are divided by panels, and the images are not visually consistent from page to page. Therefore, existing works tailored for natural images perform poorly in analyzing comics. Considering the identification of comic genres is tied to the overall story plotting, a long-term understanding that makes full use of the semantic interactions between multi-level comic fragments needs to be fully exploited. In this paper, we propose P²Comic, a Panel-Page-aware Comic genre classification model, which takes page sequences of comics as the input and produces class-wise probabilities. P²Comic utilizes detected panel boxes to extract panel representations and deploys self-attention to construct panel-page understanding, assisted with inter-dependent classifiers to model label correlation. We develop the first comic dataset for the task of comic genre classification with multi-genre labels. Our approach is proved by experiments to outperform state-of-the-art methods on related tasks. We also validate the extensibility of our network to perform on the multi-modal scenario. Finally, we show the practicability of our approach by giving effective genre prediction results for whole comic books.

24 May 2022 in Computer Graphics Forum
Verified
Journal Article
Closed access
ZZZibo Zhang
Zibo Zhang
XLXueting Liu
Xueting Liu
CLChengze Li
Chengze Li
HWHuisi Wu
Huisi Wu
ZWZhenkun Wen
Zhenkun Wen