Due to the development of facial manipulation technologies, the generated deepfake videos cause a severe trust crisis in society. Existing methods prove that effective extraction of the artifacts introduced during the forgery process is essential for deepfake detection. However, since the features extracted by supervised binary classification contain a lot of artifact-irrelevant information, existing algorithms suffer severe performance degradation in the case of the mismatch between training and testing datasets. To overcome this issue, we propose an Artifacts-Disentangled Adversarial Learning (ADAL) framework to achieve accurate deepfake detection by disentangling the artifacts from irrelevant information. Furthermore, the proposed algorithm provides visual evidence by effectively estimating artifacts. Specifically, Multi-scale Feature Separator (MFS) in the disentanglement generator is designed to precisely transmit the artifact features and optimize the connection between the encoder and decoder. In addition, we design an Artifacts Cycle Consistency Loss (ACCL) which uses the disentangled artifacts to construct new samples and enables pixel-level supervised training for the generator to estimate more accurate artifacts. The symmetric discriminators are paralleled to differentiate the constructed samples from the original images in both fake and real domains, making the adversarial training process more stable. Extensive experiments on existing benchmarks demonstrate that the proposed method outperforms the state-of-the-art approaches.

Image source forensics is widely considered as one of the most effective ways to verify in a blind way digital image authenticity and integrity. In the last few years, many researchers have applied data-driven approaches to this task, inspired by the excellent performance obtained by those techniques on computer vision problems. In this survey, we present the most important data-driven algorithms that deal with the problem of image source forensics. To make order in this vast field, we have divided the area in five sub-topics: source camera identification, recaptured image forensic, computer graphics (CG) image forensic, GAN-generated image detection, and source social network identification. Moreover, we have included the works on anti-forensics and counter anti-forensics. For each of these tasks, we have highlighted advantages and limitations of the methods currently proposed in this promising and rich research field.

A key challenge of image splicing detection is how to localize integral tampered regions without false alarm. Although current forgery detection approaches have achieved promising performance, the integrality and false alarm are overlooked. In this paper, we argue that the insufficient use of splicing boundary is a main reason for poor accuracy. To tackle this problem, we propose an Edge-enhanced Transformer (ET) for tampered region localization. Specifically, to capture rich tampering traces, a two-branch edge-aware transformer is built to integrate the splicing edge clues into the forgery localization network, generating forgery features and edge features. Furthermore, we design a feature enhancement module to highlight the artifacts of the edge area in forgery features and assign weight values to the resulting tensor in spatial domain for vital signal strengthening and noise suppression. Extensive experimental results on CASIA v1.0, CASIA v2.0 and NC2016 demonstrate that the proposed method can accurately localize tampered regions in both pixel and edge levels and outperforms state-of-the-art methods.

In recent years, high-dimensional histogram modification based reversible data hiding for images has drawn much attention among researchers. Pairwise prediction-error expansion (pairwise PEE) achieves great performance by exploiting the correlations among prediction-errors, in which a two-dimensional (2D) prediction-error histogram (PEH) is generated, and then modified based on a specific 2D modification mapping for reversible embedding. However, the used 2D mapping is fixed and heuristically designed without considering the image content, so the embedding performance of pairwise PEE can be further improved. To this end, the adaptive pairwise PEE (APPEE) is proposed in this paper to adaptively design the 2D mapping according to the distribution of 2D PEH, such that a better embedding performance is derived. To further improve the embedding performance, the proposed APPEE is extended from one single 2D PEH to multiple 2D PEHs generated based on different local complexities, in which multiple content-based 2D mappings are designed adaptively and individually for each 2D PEH. Experimental results show that the APPEE for either one single 2D PEH or multiple 2D PEHs significantly outperforms the conventional pairwise PEE. Simultaneously, the superiority of the proposed method is also experimentally verified compared with some other state-of-the-art works.

Deepfake techniques can forge the visual or audio signals in the video, which leads to inconsistencies between visual and audio (VA) signals. Therefore, multimodal detection methods expose deepfake videos by extracting VA inconsistencies. Recently, deepfake technology has started VA collaborative forgery to obtain more realistic deepfake videos, which poses new challenges for extracting VA inconsistencies. Recent multimodal detection methods propose to first extract natural VA correspondences in real videos in a self-supervised manner, and then use the learned real correspondences as targets to guide the extraction of VA inconsistencies in the subsequent deepfake detection stage. However, the inherent VA relations are difficult to extract due to the modality gap, which leads to the limited auxiliary performance of the aforementioned self-supervised methods. In this paper, we propose Predictive Visual-audio Alignment Self-supervision for Multimodal Deepfake Detection (PVASS-MDD), which consists of PVASS auxiliary and MDD stages. In the PVASS auxiliary stage in real videos, we first devise a three-stream network to associate two augmented visual views with corresponding audio clues, leading to explore common VA correspondences based on cross-view learning. Secondly, we introduce a novel cross-modal predictive align module for eliminating VA gaps to provide inherent VA correspondences. In the MDD stage, we propose to the auxiliary loss to utilize the frozen PVASS network to align VA features of real videos, to better assist multimodal deepfake detector for capturing subtle VA inconsistencies. We conduct extensive experiments on existing widely used and latest multimodal deepfake datasets. Our method obtains a significant performance improvement compared to state-of-the-art methods.

Recently, realistic DeepFake videos have raised severe security concerns in society. Existing video-based detection methods observe local spatial regions with the coarse temporal view, thus it is difficult to obtain subtle spatiotemporal information, resulting in limited generalization ability. In this paper, we propose a novel Augmented Multi-scale Spatiotemporal Inconsistency Magnifier (AMSIM) with a Global Inconsistency View (GIV) and a more meticulous Multi-timescale Local Inconsistency View (MLIV), focusing on mining comprehensive and more subtle spatiotemporal cues. Firstly, the GIV that includs the global spatial and long-term temporal views is established to ensure comprehensive spatiotemporal clues are captured. Then, the MLIV with the critical local spatial and multi-timescale local temporal views is designed for magnifying the indetectable spatiotemporal abnormality. Subsequently, GIV is utilized to guide MLIV to dynamically find local spatiotemporal anomalies that are highly relevant to the overall video. Finally, to further obtain a generalized framework, the adversarial data augmentation is specially designed to expand source domains and simulate unseen forgery domains. Extensive experiments on six large-scale datasets show that our AMSIM outperforms state-of-the-art detection methods and remains effective when applied to unseen forgery techniques and datasets.