Shi Zheng

Shi Zheng

Auto-generated profile, based on publicly available information

Shi Zheng

Computer School, Beijing Information Science and Technology University, Beijing, China

Research Focus & Metrics

Academic Career Length

3Years (2024-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
11 June 2024 in IEEE Transactions on Visualization and Computer Graphics
Verified
Journal Article
Closed access
Wenfeng SongWenfeng Song
Wenfeng Song
XWXuan Wang
Xuan Wang
SZShi Zheng
Shi Zheng
SLShuai Li
Shuai Li
AHAimin Hao
Aimin Hao
Xia HouXia Hou
Xia Hou
It is a challenging task to create realistic 3D avatars that accurately replicate individuals’ speech and unique talking styles for speech-driven facial animation. Existing techniques have made remarkable progress but still struggle to achieve lifelike mimicry. This article proposes “TalkingStyle”, a novel method to generate personalized talking avatars while retaining the talking style of the person. Our approach uses a set of audio and animation samples from an individual to create new facial animations that closely resemble their specific talking style, synchronized with speech. We disentangle the style codes from the motion patterns, allowing our method to associate a distinct identifier with each person. To manage each aspect effectively, we employ three separate encoders for style, speech, and motion, ensuring the preservation of the original style while maintaining consistent motion in our stylized talking avatars. Additionally, we propose a new style-conditioned transformer decoder, offering greater flexibility and control over the facial avatar styles. We comprehensively evaluate TalkingStyle through qualitative and quantitative assessments, as well as user studies demonstrating its superior realism and lip synchronization accuracy compared to current state-of-the-art methods.