Yizhe Zhu

Yizhe Zhu

Auto-generated profile, based on publicly available information

Yizhe Zhu

Shanghai Jiao Tong University

Research Focus & Metrics

Academic Career Length

5Years (2022-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
28 February 2023 in IEEE Transactions on Circuits and Systems for Video Technology
Verified
Journal Article
Closed access
XSXin Sun
Xin Sun
JGJialin Gao
Jialin Gao
YZYizhe Zhu
Yizhe Zhu
XWXuan Wang
Xuan Wang
XZXi Zhou
Xi Zhou
Video moment retrieval aims to retrieve a target moment from an untrimmed video that semantically corresponds to the given language query. Existing methods commonly treat it as a regression task or a ranking task from the perspective of computer vision. Most of these works neglect comprehensive relations between video content and language context at a multi-granularity level and fail to efficiently model temporal relations among different video moments. In this paper, we formulate video moment retrieval into video reading comprehension by treating the input video as a text passage and language query as a question. To tackle the above impediments, we propose a Comprehensive Relation-aware Network (CRNet) to perceive comprehensive relations from extensive aspects. Specifically, we unite visual and textual features simultaneously at both clip-level and moment-level to thoroughly exploit inter-modality information, leading to a coarse-and-fine cross-modal interaction. Moreover, a background suppression module is introduced to restrain irrelevant background clips, meanwhile, a novel IoU attention mechanism and graph attention layer are efficiently devised to focus on the dependencies among highly-correlated video moments for the best choice selection. In-depth experiments on three public datasets TACoS, ActivityNet Captions, and Charades-STA demonstrate the superiority of our solution.

10 October 2023 in Neurocomputing
Verified
Journal Article
Closed access
YZYizhe Zhu
Yizhe Zhu
CZChunhui Zhang
Chunhui Zhang
JGJialin Gao
Jialin Gao
XSXin Sun
Xin Sun
ZRZihan Rui
Zihan Rui
XZXi Zhou
Xi Zhou

12 June 2023
Verified
Conference Paper
Open access
YZYizhe Zhu
Yizhe Zhu
JGJialin Gao
Jialin Gao
XZXi Zhou
Xi Zhou
Existing cross-dataset deepfake detection approaches exploit mouth-related mismatches between the auditory and visual modalities in fake videos to enhance generalisation to unseen forgeries. However, such methods inevitably suffer performance degradation with limited or unaltered mouth motions, we argue that face forgery detection consistently benefits from using high-level cues across the whole face region. In this paper, we propose a two-phase audio-driven multi-modal transformer-based framework, termed AVForensics, to perform deepfake video content detection from an audio-visual matching view related to full face. In the first pre-training phase, we apply the novel uniform masking strategy to model global facial features and learn temporally dense video representations in a self-supervised cross-modal manner, by capturing the natural correspondence between the visual and auditory modalities regardless of large-scaled labelled data and heavy memory usage. Then we use these learned representations to fine-tune for the down-stream deepfake detection task in the second phase, which encourages the model to offer accurate predictions based on captured global facial movement features. Extensive experiments and visualizations on various public datasets demonstrate the superiority of our self-supervised pre-trained method for achieving generalisable and robust deepfake video detection.

31 December 2022 in Pattern Recognition Letters
Verified
Journal Article
Closed access
YZYizhe Zhu
Yizhe Zhu
JGJialin Gao
Jialin Gao
TWTianshu Wu
Tianshu Wu
QLQiong Liu
Qiong Liu
XZXi Zhou
Xi Zhou