In the existing deep learning based watermarking models, extracted image features for fusing with watermark are not abundant enough and more critically, essential features are not highlighted to be learned with the purpose of robust watermarking, both of which limit the watermarking performance. To solve those two drawbacks, this paper proposes an attention-guided robust image watermarking model based on generative adversarial network (ARWGAN). To acquire a great deal of representational image features, a feature fusion module (FFM) is devised to learn shallow and deep features effectively for multi-layer fusion with watermark, and meanwhile, reuse of those features by the dense connection enhances robustness. To alleviate image distortion caused by embedding watermark, an attention module (AM) is deployed to compute the attention mask by mining the global features of the original image. Specifically, with the guidance of the attention mask, image features representing inconspicuous regions and texture regions are enhanced for embedding the high strength of watermark, and simultaneously other features are suppressed to improve the watermarking performance. Furthermore, the noise sub-network is adopted for robustness enhancement by simulating various image attacks in iterative training. The discriminator is used to distinguish the encoded image from the original image for improving watermarking invisibility continuously. Experimental results demonstrate that ARWGAN is superior to the existing state-of-the-art watermarking models, and ablation experiments prove the effectiveness of the FFM and the AM. The code is avaliable in https://github.com/river-huang/ARWGAN.
16 October 2023 in IEEE Transactions on Information Forensics and Security
The serious concerns over the negative impacts of Deepfakes have attracted wide attentions in the community of multimedia forensics. The existing detection works achieve deepfake detection by improving the traditional backbone networks to capture subtle manipulation traces. However, there is no attempt to construct new backbone networks with different structures for Deepfake detection by improving the internal feature representation of convolution. In this work, we propose a novel Space-Frequency Interactive Convolution (SFIConv) to efficiently model the manipulation clues left by Deepfake. To obtain high-frequency features from tampering traces, a Multichannel Constrained Separable Convolution (MCSConv) is designed as the component of the proposed SFIConv, which learns space-frequency features via three stages, namely generation, interaction and fusion. In addition, SFIConv can replace the vanilla convolution in any backbone networks without changing the network structure. Extensive experimental results show that seamlessly equipping SFIConv into the backbone network greatly improves the accuracy for Deepfake detection. In addition, the space-frequency interaction mechanism does benefit to capturing common artifact features, thus achieving better results in cross-dataset evaluation. Our code will be available at https://github.com/EricGzq/SFIConv .
21 April 2021 in IEEE Transactions on Broadcasting
Versatile video coding (VVC) further improves video coding gain by introducing a new Quad-tree (QT) with nested multi-type tree (QTMT) partition structure, which includes QT partition, horizontal and vertical binary-tree (BT) partition, and horizontal and vertical ternary-tree (TT) partition. However, the coding gain is achieved at the cost of drastically increased coding complexity. We propose an early intra coding unit (CU) size partition method for VVC based on a tunable decision model. The distortion of the CU is firstly obtained by the difference between the original luminance pixel and the predicted one. By exploiting the distortion, two decision models are established for the early skipping of BT and TT partition for each CU, respectively. Then, a tunable decision model is presented to achieve different trade-off between encoding loss and complexity reduction. Extensive experimental results show that the proposed approach reduces the encoding time ranging from 22.6% to 67.6% with the BD-Rate increase ranging from 0.56% to 2.61%. It is also suitable for parallel implementation by exploiting the pipeline structure of the VVC encoder due to its independency with neighboring CU blocks.
26 November 2022 in Expert Systems with Applications
With the rapid growth of Internet technology, security concerns have risen, particularly with the prevalence of Deepfakes, a popular visual forgery technique. Therefore, there is necessary to research more powerful methods to detect Deepfakes. However, many Convolutional Neural Networks-based detection methods struggle with cross-database performance, often overfitting to specific color textures. We observe that image noises can weaken the influence of color textures and expose the forgery traces in the noise domain. This is because tampering techniques, when altering face images, disrupt the consistency of feature distribution in the noise space. And the forgery traces in the noise space are complementary to the tampering artifacts present in the image space information. Therefore, we propose a novel face forgery detection network that combines spatial domain and noise domain. Our Dual Feature Fusion Module and Local Enhancement Attention Module contribute to more comprehensive feature representations, enhancing our method’s discriminative ability. Experimental results demonstrate superior performance compared to existing methods on mainstream datasets. https://github.com/jhchen1998/DeepfakeDetection.