28 August 2024 in IEEE Transactions on Information Forensics and Security
Deepfake detection plays a key role in preventing the misuses of artificial intelligence in video editing. Current deep learning-based deepfake detection methods often perform quite well in intra-dataset testing, but they may lose good performance in cross-dataset testing. In other words, generalization capability is still a crucial problem to be resolved. In this paper, we address deepfake detection by treating an image as non-Euclidean data and representing it as a graph so as to infer the informative connections between image patches/nodes to improve the detector’s generalization capability. Specifically, we propose a graph neural network-based paradigm that casts deepfake detection as a graph binary classification problem. First, we propose a dual-branch network to extract node features from both RGB images and their color difference images (CDIs) via the Transformer-based trainable node encoder module (TNEM). Second, we adopt the adjacency matrix to establish the connections of the nodes and further optimize the graph representation by applying the adaptive threshold to the adjacency matrix. Third, multi-head graph convolutional neural networks are carried out for node feature extraction. RGB node features and CDI node features are concatenated and separately fed into the graph classifier and node classifier for forgery detection and forgery localization. Experimental results demonstrate that our method can overall outperform other state-of-the-art methods on 7 popular benchmark datasets. Notably, our model achieves the highest AUC values of 96.19%, 80.99% and 87.68% on Celeb-DF-V2, DFDC and DFDCP in turn when trained on FF++ (C23). The visualization of node classification results also provides good interpretability of our proposed approach.
2 April 2025 in Pattern Recognition
6 November 2023 in IEEE Signal Processing Letters
Resizing input face images of arbitrary sizes to a uniform size is an essential preprocessing to satisfy the architectural requirements of face forgery detectors. In this letter, we reveal an important observation that traditional resizing methods degrade the performance of face forgery detectors due to the loss of high-frequency information. To address this issue, we propose a simple yet effective learnable information-preserving resizer to replace its lossy traditional counterparts. Specifically, we use Haar transform to separate low-and high-frequency components, and then perform learnable resizing on the high-frequency subbands. We conduct experiments to compare our learnable resizer with other methods and evaluate three existing detectors with and without incorporating our resizer. Experimental results show that our resizer outperforms other resizers and consistently enhances the detection performance of tested detectors, confirming the effectiveness of our proposed resizer
1 January 2024 in IET Biometrics
Identity‐based Deepfake detection methods have the potential to improve the generalization, robustness, and interpretability of the model. However, current identity‐based methods either require a reference or can only be used to detect face replacement but not face reenactment. In this paper, we propose a novel Deepfake video detection approach based on identity anomalies. We observe two types of identity anomalies: the inconsistency between clip‐level static ID (facial appearance) and clip‐level dynamic ID (facial behavior) and the temporal inconsistency of image‐level static IDs. Since these two types of anomalies can be detected through self‐consistency and do not depend on the manipulation type, our method is a reference‐free and manipulation‐independent approach. Specifically, our detection network consists of two branches: the static–dynamic ID discrepancy detection branch for the inconsistency between dynamic and static ID and the temporal static ID anomaly detection branch for the temporal anomaly of static ID. We combine the outputs of the two branches by weighted averaging to obtain the final detection result. We also designed two loss functions: the static–dynamic ID matching loss and the dynamic ID constraint loss, to enhance the representation and discriminability of dynamic ID. We conduct experiments on four benchmark datasets and compare our method with the state‐of‐the‐art methods. Results show that our method can detect not only face replacement but also face reenactment, and also has better detection performance over the state‐of‐the‐art methods on unknown datasets. It also has superior robustness against compression. Identity‐based features provide a good explanation of the detection results.