Bowen Ma

Bowen Ma

Auto-generated profile, based on publicly available information

Bowen Ma

Fuxi AI Lab, Netease, Hangzhou, China

Research Focus & Metrics

Academic Career Length

3Years (2024-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
20 June 2025 in IEEE Transactions on Multimedia
Verified
Journal Article
Closed access
YMYifeng Ma
Yifeng Ma
SWSuzhen Wang
Suzhen Wang
YDYu Ding
Yu Ding
BMBowen Ma
Bowen Ma
TLTangjie Lv
Tangjie Lv
XYXin Yu
Xin Yu
Audio-driven talking head generation has drawn growing attention. To produce talking head videos with desired facial expressions, previous methods rely on extra reference videos to provide expression information, which may be difficult to find and hence limits their usage. In this work, we propose TalkCLIP, a framework that can generate talking heads where the expressions are specified by natural language, hence allowing for specifying expressions more conveniently. To model the mapping from text to expressions, we first construct a text-video paired talking head dataset where each video has diverse text descriptions that depict both coarse-grained emotions and fine-grained facial movements. Leveraging the proposed dataset, we introduce a CLIP-based style encoder that projects natural language-based descriptions to the representations of expressions. TalkCLIP can even infer expressions for descriptions unseen during training. TalkCLIP can also use text to modulate expression intensity and edit expressions. Extensive experiments demonstrate that TalkCLIP achieves the advanced capability of generating photo-realistic talking heads with vivid facial expressions guided by text descriptions.

19 February 2024 in IEEE Transactions on Affective Computing
Verified
Journal Article
Closed access
BMBowen Ma
Bowen Ma
RARudong An
Rudong An
WZWei Zhang
Wei Zhang
YDYu Ding
Yu Ding
ZZZeng Zhao
Zeng Zhao
ZHZhipeng Hu
Zhipeng Hu
As a fine-grained and local expression behavior measurement, facial action unit (FAU) analysis (e.g., detection and intensity estimation) has been documented for its time-consuming, labor-intensive, and error-prone annotation. Thus a long-standing challenge of FAU analysis arises from the data scarcity of manual annotations, limiting the generalization ability of trained models to a large extent. Amounts of previous works have made efforts to alleviate this issue via semi/weakly supervised methods and extra auxiliary information. However, these methods still require domain knowledge and have not yet avoided the high dependency on data annotation. This paper introduces a robust facial representation model MAE-Face for AU analysis. Using masked autoencoding as the self-supervised pre-training approach, MAE-Face first learns a high-capacity model from a feasible collection of face images without additional data annotations. Then after being fine-tuned on AU datasets, MAE-Face exhibits convincing performance for both AU detection and AU intensity estimation, achieving a new state-of-the-art on nearly all the evaluation results. Further investigation shows that MAE-Face achieves decent performance even when fine-tuned on only 1% of the AU training set, strongly proving its robustness and generalization performance. The pre-trained model is available at GitHub repository.

27 November 2024
Verified
Book Chapter
Closed access
PDF file
HLHanwei Liu
Hanwei Liu
RARudong An
Rudong An
ZZZhimeng Zhang
Zhimeng Zhang
BMBowen Ma
Bowen Ma
WZWei Zhang
Wei Zhang
YDYu Ding
Yu Ding