Wenbo Zhou

Dr. Wenbo Zhou


Research Focus & Metrics

Academic Career Length

9Years (2018-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
19 June 2023 in Journal of Imaging
Verified
Journal Article
Open access
PDF file
DSDavide Salvi
Davide Salvi
HLHonggu Liu
Honggu Liu
SMSara Mandelli
Sara Mandelli
PBPaolo Bestagini
Paolo Bestagini
WZWenbo Zhou
Wenbo Zhou
STStefano Tubaro
Stefano Tubaro
The widespread use of deep learning techniques for creating realistic synthetic media, commonly known as deepfakes, poses a significant threat to individuals, organizations, and society. As the malicious use of these data could lead to unpleasant situations, it is becoming crucial to distinguish between authentic and fake media. Nonetheless, though deepfake generation systems can create convincing images and audio, they may struggle to maintain consistency across different data modalities, such as producing a realistic video sequence where both visual frames and speech are fake and consistent one with the other. Moreover, these systems may not accurately reproduce semantic and timely accurate aspects. All these elements can be exploited to perform a robust detection of fake content. In this paper, we propose a novel approach for detecting deepfake video sequences by leveraging data multimodality. Our method extracts audio-visual features from the input video over time and analyzes them using time-aware neural networks. We exploit both the video and audio modalities to leverage the inconsistencies between and within them, enhancing the final detection performance. The peculiarity of the proposed method is that we never train on multimodal deepfake data, but on disjoint monomodal datasets which contain visual-only or audio-only deepfakes. This frees us from leveraging multimodal datasets during training, which is desirable given their lack in the literature. Moreover, at test time, it allows to evaluate the robustness of our proposed detector on unseen multimodal deepfakes. We test different fusion techniques between data modalities and investigate which one leads to more robust predictions by the developed detectors. Our results indicate that a multimodal approach is more effective than a monomodal one, even if trained on disjoint monomodal datasets.

26 December 2025 in International Journal of Computer Vision
Verified
Journal Article
Closed access
PDF file
ZYZonghao Ying
Zonghao Ying
Aishan LiuAishan Liu
Aishan Liu
SLSiyuan Liang
Siyuan Liang
LHLei Huang
Lei Huang
JGJinyang Guo
Jinyang Guo
DTDacheng Tao
Dacheng Tao

27 March 2025 in IEEE Transactions on Information Forensics and Security
Verified
Journal Article
Closed access
Aishan LiuAishan Liu
Aishan Liu
YZYuguang Zhou
Yuguang Zhou
Xianglong LiuXianglong Liu
Xianglong Liu
TZTianyuan Zhang
Tianyuan Zhang
SLSiyuan Liang
Siyuan Liang
DTDacheng Tao
Dacheng Tao
Large language models (LLMs) have transformed the development of embodied intelligence. By providing a few contextual demonstrations (such as rationales and solution examples) developers can utilize the extensive internal knowledge of LLMs to effortlessly translate complex tasks described in abstract language into sequences of code snippets, which will serve as the execution logic for embodied agents. However, this paper uncovers a significant backdoor security threat within this process and introduces a novel method called Contextual Backdoor Attack. By poisoning just a few contextual demonstrations, attackers can covertly compromise the contextual environment of a black-box LLM, prompting it to generate programs with context-dependent defects. These programs appear logically sound but contain defects that can activate and induce unintended behaviors when the operational agent encounters specific triggers in its interactive environment. To compromise the LLM’s contextual environment, we employ adversarial in-context generation to optimize poisoned demonstrations, where an LLM judge evaluates these poisoned prompts, reporting to an additional LLM that iteratively optimizes the demonstration in a two-player adversarial game using chain-of-thought reasoning. To enable context-dependent behaviors in downstream agents, we implement a dual-modality activation strategy that controls both the generation and execution of program defects through textual and visual triggers. We expand the scope of our attack by developing five program defect modes that compromise key aspects of confidentiality, integrity, and availability in embodied agents. To validate the effectiveness of our approach, we conducted extensive experiments across various tasks, including robot planning, robot manipulation, and compositional visual reasoning. Additionally, we demonstrate the potential impact of our approach by successfully attacking real-world autonomous driving systems. The contextual backdoor threat introduced in this study poses serious risks for millions of downstream embodied agents, given that most publicly available LLMs are third-party-provided. This paper aims to raise awareness of this critical threat. Our code and demos are available at https://contextual-backdoor.github.io/.

19 January 2024 in IEEE Transactions on Multimedia
Verified
Journal Article
Closed access
ZYZixin Yin
Zixin Yin
JWJiakai Wang
Jiakai Wang
YXYisong Xiao
Yisong Xiao
HZHanqing Zhao
Hanqing Zhao
TLTianlin Li
Tianlin Li
Xianglong LiuXianglong Liu
Xianglong Liu
The abuse of deepfake techniques has raised serious concerns about social security and ethical problems, which motivates the development of deepfake detection. However, without fully addressing the domain gap issue, existing deepfake detection methods still show weak generalization ability among datasets belonging to different domains with domain-specific characteristics like identities and generation methods, limiting their practical applications. In this paper, we propose the Invariant Domain-oriented Deepfake Detection method (ID$_{3}$) , which improves the generalization of deepfake detection on multiple domains through invariant risk minimization, a novel learning paradigm that addresses the domain gap problem by jointly training a purified invariant predictor and learning an aligned invariant representation. To train a purified invariant predictor, we design the Domain Refinement Data Augmentation strategy with self-face-swapping and region-erasing approaches, which suppresses domain-specific features and encourages the models to focus on critical domain-invariant characteristics. To learn an aligned invariant representation, we propose the Domain Calibration Batch Normalization approach with multiple BN branches, which normalizes input features from different domains into aligned representations during both training and testing. Extensive experiments on multiple datasets demonstrate that our framework can boost the deepfake detection generalization ability and outperform other baselines by large margins. Our codes can be found here

1 October 2023
Verified
Conference Paper
Open access
TWTianyi Wei
Tianyi Wei
DCDongdong Chen
Dongdong Chen
WZWenbo Zhou
Wenbo Zhou
JLJing Liao
Jing Liao
WZWeiming Zhang
Weiming Zhang
NYNenghai Yu
Nenghai Yu
Hair editing has made tremendous progress in recent years. Early hair editing methods use well-drawn sketches or masks to specify the editing conditions. Even though they can enable very fine-grained local control, such interaction modes are inefficient for the editing conditions that can be easily specified by language descriptions or reference images. Thanks to the recent breakthrough of cross-modal models (e.g., CLIP), HairCLIP is the first work that enables hair editing based on text descriptions or reference images. However, such text-driven and reference-driven interaction modes make HairCLIP unable to support fine-grained controls specified by sketch or mask. In this paper, we propose HairCLIPv2, aiming to support all the aforementioned interactions with one unified framework. Simultaneously, it improves upon HairCLIP with better irrelevant attributes (e.g., identity, background) preservation and unseen text descriptions support. The key idea is to convert all the hair editing tasks into hair transfer tasks, with editing conditions converted into different proxies accordingly. The editing effects are added upon the input image by blending the corresponding proxy features within the hairstyle or hair color feature spaces. Besides the unprecedented user interaction mode support, quantitative and qualitative experiments demonstrate the superiority of HairCLIPv2 in terms of editing effects, irrelevant attribute preservation and visual naturalness. Our code is available at https://github.com/wty-ustc/HairCLIPv2.

2 December 2024
Verified
Conference Paper
Open access
KTKunsheng Tang
Kunsheng Tang
WZWenbo Zhou
Wenbo Zhou
JZJie Zhang
Jie Zhang
Aishan LiuAishan Liu
Aishan Liu
GDGelei Deng
Gelei Deng
NYNengHai Yu
NengHai Yu
Large language models (LLMs) have exhibited remarkable capabilities in natural language generation, but they have also been observed to magnify societal biases, particularly those related to gender. In response to this issue, several benchmarks have been proposed to assess gender bias in LLMs. However, these benchmarks often lack practical flexibility or inadvertently introduce biases. To address these shortcomings, we introduce GenderCARE, a comprehensive framework that encompasses innovative Criteria, bias Assessment, Reduction techniques, and Evaluation metrics for quantifying and mitigating gender bias in LLMs. To begin, we establish pioneering criteria for gender equality benchmarks, spanning dimensions such as inclusivity, diversity, explainability, objectivity, robustness, and realisticity. Guided by these criteria, we construct GenderPair, a novel pair-based benchmark designed to assess gender bias in LLMs comprehensively. Our benchmark provides standardized and realistic evaluations, including previously overlooked gender groups such as transgender and non-binary individuals. Furthermore, we develop effective debiasing techniques that incorporate counterfactual data augmentation and specialized fine-tuning strategies to reduce gender bias in LLMs without compromising their overall performance. Extensive experiments demonstrate a significant reduction in various gender bias benchmarks, with reductions peaking at over 90% and averaging above 35% across 17 different LLMs. Importantly, these reductions come with minimal variability in mainstream language tasks, remaining below 2%. By offering a realistic assessment and tailored reduction of gender biases, we hope that our GenderCARE can represent a significant step towards achieving fairness and equity in LLMs. More details are available at https://github.com/kstanghere/GenderCARE-ccs24.