The serious concerns over the negative impacts of Deepfakes have attracted wide attentions in the community of multimedia forensics. The existing detection works achieve deepfake detection by improving the traditional backbone networks to capture subtle manipulation traces. However, there is no attempt to construct new backbone networks with different structures for Deepfake detection by improving the internal feature representation of convolution. In this work, we propose a novel Space-Frequency Interactive Convolution (SFIConv) to efficiently model the manipulation clues left by Deepfake. To obtain high-frequency features from tampering traces, a Multichannel Constrained Separable Convolution (MCSConv) is designed as the component of the proposed SFIConv, which learns space-frequency features via three stages, namely generation, interaction and fusion. In addition, SFIConv can replace the vanilla convolution in any backbone networks without changing the network structure. Extensive experimental results show that seamlessly equipping SFIConv into the backbone network greatly improves the accuracy for Deepfake detection. In addition, the space-frequency interaction mechanism does benefit to capturing common artifact features, thus achieving better results in cross-dataset evaluation. Our code will be available at https://github.com/EricGzq/SFIConv .
21 February 2024 in Expert Systems with Applications
Full-reference image dehazing quality assessment (FR-IDQA) evaluates the visual quality of a dehazed image by measuring its differences with a clear reference. The existing FR-IDQA methods are not convincing due to the lack of well-aligned datasets of hazy and clear image pairs and the limited hand-crafted features make it difficult to simulate the complicated perception by the human visual system (HVS). In this work, we build a real-world image dataset, namely RW-Haze, which comprises natural hazy images and their well-aligned clear references. Each clear image is paired with several hazy images with diverse haze levels from slight to heavy. Meanwhile, the existing FR-IDQA works evaluate the dehazed image quality in a global manner, without considering local haze distributions in the original hazy image. Actually, the perceived haze in a natural hazy image is not uniformly distributed, and the haze density varies with scene depth. Based on this priori observation, we design a haze density-aware convolutional neural network (CNN), namely DehIQA, for FR-IDQA. It adopts transfer learning to alleviate the issue of lacking sufficient labeled data. Specifically, we divide image dehazing assessment into two tasks. The source task is to classify unpaired clear and hazy images, which enforces the deep network to learn haze-related features. The target task is image quality assessment, which is achieved by transferring the trained model for the source task to the target task. Considering the fact that the perceived distortion in a dehazed image is also not uniform, we present a haze density-aware mechanism into DehIQA, which assigns different weights for different local regions in a dehazed image in terms of the dark channel of the original hazy image. Extensive experimental results show that DehIQA outperforms the state-of-the-art (SOTA) works on the benchmark dataset and achieves better consistency with human perceptions.
18 September 2023 in ACM Transactions on Multimedia Computing, Communications, and Applications
Image steganography is a technique used to conceal secret information within cover images without being detected. However, the advent of convolutional neural networks (CNNs) has threatened the security of image steganography. Due to the inherent properties of adversarial examples, adding perturbations to stego images can mislead the CNN-based image steganalysis, but it also easily leads to some errors when extracting secret information. Recently, some adversarial embedding methods have been proposed for improving image steganography security. In this work, we aim at furthering enhance the security of adversarial embedding-based image steganography by exploiting the strong correlation between adjacent pixels. Specifically, we divide the cover image into four non-overlapping parts for four-stage information embedding. During the adversarial embedding process, we cluster the modification directions of adjacent pixels and select only those with relatively larger amplitudes of gradients and smaller embedding costs to update their original embedding costs. Experimental results demonstrate that our proposed method can effectively fool targeted steganalyzers and outperform state-of-the-art techniques under different scenarios.
In this paper, a reversible data hiding method in encrypted image (RDHEI) is proposed. Prior to image encryption, the embeddable pixels are selected from an original image according to prediction errors due to adjacent pixels with strong correlation. Then the embeddable pixels and the other pixels are both rearranged and encrypted to generate an encrypted image. Secret bits are directly embedded into the multiple MSBs (most significant bit) of the embeddable pixels in the encrypted image to generate a marked encrypted image during the encoding phase. In the decoding phase, secret bits can be extracted from the multiple MSBs of the embeddable pixels in the marked encrypted image. Moreover, the original embeddable pixels are restored losslessly by using correlation of the adjacent pixels. Thus, a reconstructed image with high visual quality can be obtained only when the encryption key is available. Since exploiting multiple MSBs of the embeddable pixels, the proposed method can obtain a very large embedding capacity. Experimental results show that the proposed method is able to achieve an average embedding rate as large as 1.7215 bpp (bits per pixel) for the BOW-2 database.
3 March 2026 in IEEE Transactions on Circuits and Systems for Video Technology
Existing anti-screen-shooting watermarking methods embed watermarks on either the server or client side. Server-side embedding incurs high computational and communication overhead under concurrent requests, while client-side methods risk watermark interception during transmission and require additional encrypted channels. To address these limitations, we propose an end-to-end separable watermarking framework (SepWater) that exploits latent space representations. By decoupling server-side watermark embedding from client-side image generation, SepWater enhances security by preventing leaks of both the original content and the watermark, while also reducing transmission costs. On the server side, a dedicated encoder processes the watermark information, while a frozen pre-trained encoder handles the original image. We then fuse their outputs into a compact latent vector for transmission to the client. On the client side, a frozen VQGAN decoder reconstructs the watermarked image directly. In addition, we propose a local residual attention loss, combined with other image quality constraints, to produce watermarked images with high capture resistance and visual fidelity. Furthermore, to improve the robustness of the SepWater, two noise modes are simulated that contain eye protection noise and lightweight edge grayscale deviation noise. Experiments show that SepWater outperforms state-of-the-art methods in withstanding screen-shooting distortions, optimizing communication efficiency, and scaling under high concurrency, making it suitable for practical deployment. The source code is released at https://github.com/CVhnu/SepWater.