Jinsong Zhang

Dr. Jinsong Zhang


Research Focus & Metrics

Academic Career Length

7Years (2020-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
13 October 2020 in IEEE Transactions on Image Processing
Verified
Journal Article
Closed access
KLKun Li
Kun Li
JZJinsong Zhang
Jinsong Zhang
YLYebin Liu
Yebin Liu
YLYu-Kun Lai
Yu-Kun Lai
QDQionghai Dai
Qionghai Dai
Human pose transfer, which aims at transferring the appearance of a given person to a target pose, is very challenging and important in many applications. Previous work ignores the guidance of pose features or only uses local attention mechanism, leading to implausible and blurry results. We propose a new human pose transfer method using a generative adversarial network (GAN) with simplified cascaded blocks. In each block, we propose a pose-guided non-local attention (PoNA) mechanism with a long-range dependency scheme to select more important regions of image features to transfer. We also design pre-posed image-guided pose feature update and post-posed pose-guided image feature update to better utilize the pose and image features. Our network is simple, stable, and easy to train. Quantitative and qualitative results on Market-1501 and DeepFashion datasets show the efficacy and efficiency of our model. Compared with state-of-the-art methods, our model generates sharper and more realistic images with rich details, while having fewer parameters and faster speed. Furthermore, our generated images can help to alleviate data insufficiency for person re-identification.

29 August 2022 in IEEE Transactions on Visualization and Computer Graphics
Verified
Journal Article
Closed access
XLXiongzheng Li
Xiongzheng Li
JHJing Huang
Jing Huang
JZJinsong Zhang
Jinsong Zhang
XSXiaokun Sun
Xiaokun Sun
HXHaibiao Xuan
Haibiao Xuan
KLKun Li
Kun Li
Accurately estimating the human inner-body under clothing is very important for body measurement, virtual try-on and VR/AR applications. In this article, we propose the first method to allow everyone to easily reconstruct their own 3D inner-body under daily clothing from a self-captured video with the mean reconstruction error of 0.73cm within 15s. This avoids privacy concerns arising from nudity or minimal clothing. Specifically, we propose a novel two-stage framework with a Semantic-guided Undressing Network (SUNet) and an Intra-Inter Transformer Network (IITNet). SUNet learns semantically related body features to alleviate the complexity and uncertainty of directly estimating 3D inner-bodies under clothing. IITNet reconstructs the 3D inner-body model by making full use of intra-frame and inter-frame information, which addresses the misalignment of inconsistent poses in different frames. Experimental results on both public datasets and our collected dataset demonstrate the effectiveness of the proposed method. The code and dataset is available for research purposes at http://cic.tju.edu.cn/faculty/likun/projects/Inner-Body

3 November 2023 in IEEE Transactions on Circuits and Systems for Video Technology
Verified
Journal Article
Closed access
XLXiongzheng Li
Xiongzheng Li
JZJinsong Zhang
Jinsong Zhang
YLYu-Kun Lai
Yu-Kun Lai
JYJingyu Yang
Jingyu Yang
KLKun Li
Kun Li
Much progress has been made in reconstructing garments from an image or a video. However, none of existing works meet the expectations of digitizing high-quality animatable dynamic garments that can be adjusted to various unseen poses. In this paper, we propose the first method to recover high-quality animatable dynamic garments from monocular videos without depending on scanned data. To generate reasonable deformations for various unseen poses, we propose a learnable garment deformation network that formulates the garment reconstruction task as a pose-driven deformation problem. To alleviate the ambiguity estimating 3D garments from monocular videos, we design a multi-hypothesis deformation module that learns spatial representations of multiple plausible deformations. Experimental results on several public datasets demonstrate that our method can reconstruct high-quality dynamic garments with coherent surface details, which can be easily animated under unseen poses. The code will be provided for research purposes.

13 January 2025 in IEEE Transactions on Visualization and Computer Graphics
Verified
Journal Article
Closed access
JZJinsong Zhang
Jinsong Zhang
MZMinjie Zhu
Minjie Zhu
YZYuxiang Zhang
Yuxiang Zhang
ZZZerong Zheng
Zerong Zheng
YLYebin Liu
Yebin Liu
KLKun Li
Kun Li
Whole-body motion generation from speech audio is crucial for computer graphics and immersive VR/AR. Prior methods struggle to produce natural and diverse whole-body motions from speech. In this paper, we introduce a novel method, named SpeechAct, based on a hybrid point representation and contrastive motion learning to boost realism and diversity in motion generation. Our hybrid point representation leverages the advantages of keypoint representation and surface points of 3D body model, which is easy to learn and helps to achieve smooth and natural motion generation from speech audio. We design a VQ-VAE to learn a motion codebook using our hybrid presentation, and then regress the motion from the input audio using a translation model. To boost diversity in motion generation, we propose a contrastive motion learning method according to the intuitive idea that the generated motion should be different from the motions of other audios and other speakers. We collect negative samples from other audio inputs and other speakers using our translation model. With these negative samples, we pull the current motion away from them using a contrastive loss to produce more distinctive representations. In addition, we compose a face generator to generate deterministic face motion due to the strong connection between the face movements and the speech audio. Experimental results validate the superior performance of our model. The code is available at http://cic.tju.edu.cn/faculty/likun/projects/SpeechAct/index.html.

16 October 2023 in IEEE Transactions on Circuits and Systems for Video Technology
Verified
Journal Article
Closed access
JZJinsong Zhang
Jinsong Zhang
LGLingfeng Gu
Lingfeng Gu
YLYu-Kun Lai
Yu-Kun Lai
XWXueyang Wang
Xueyang Wang
KLKun Li
Kun Li
Group detection, especially for large-scale scenes, has many potential applications for public safety and smart cities. Existing methods fail to cope with frequent occlusions in large-scale scenes with multiple people, and are difficult to effectively utilize spatio-temporal information. In this paper, we propose an end-to-end framework, GroupTransformer , for group detection in large-scale scenes. To deal with the frequent occlusions caused by multiple people, we design an occlusion encoder to detect and suppress severely occluded person crops. To explore the potential spatio-temporal relationship, we propose spatio-temporal transformers to simultaneously extract trajectory information and fuse inter-person features in a hierarchical manner. Experimental results on both large-scale and small-scale scenes demonstrate that our method achieves better performance compared with state-of-the-art methods. On large-scale scenes, our method significantly boosts the performance in terms of precision and F1 score by more than 10%. On small-scale scenes, our method still improves the performance of F1 score by more than 5%. We will release the code for research purposes .

25 June 2024 in Displays
Verified
Journal Article
Closed access
JZJinsong Zhang
Jinsong Zhang
YLYu-Kun Lai
Yu-Kun Lai
JMJian Ma
Jian Ma
KLKun Li
Kun Li