SAR images have become the latest high-resolution imaging equipment, which can monitor the Earth 24 hours a day. More and more deep learning technologies are applied to ship target detection, however, in complex environments, due to the small target of the ship, problems such as false detection and miss detection often occur. For this reason, SSPNet is proposed with several small-target-augmentation strategies to complete the detection of small ships on the sea. This network is an improvement of FPN. The model uses Context Attention Module (CAM), Scale Enhancement Module (SEM), and Scale Selection Module (SSM). CAM introduces the attention heat map, SEM uses the residual module to make the network pay more attention to specific scale targets, SSM introduces deep semantic features into shallow features. Weighted Negative Sampling (WNS) strategy is proposed to enable the network to select more representative samples. These modules make the network more suitable for small target detection. The results on SSDD dataset show that the model is superior to the existing object detection network, and the AP50 reaches 91.57%.
After several years of development, deep synthesis technology has made significant progress in image and video synthesis. Deep forgery represented by Deepfakes has become a research hotspot, which is used as a tool for disinformation attacks. The current strongly discriminative models can have good performance on specific datasets, even close to $100\%$ accuracy. Unfortunately, since a specific discriminative method only fits a specific data distribution, and different forgery methods or datasets have different data distributions. These methods fail to achieve high performance in cross-dataset detection. In response to this problem and focusing on the actual situation, we adjust the strong generalization detection across the dataset to the generalization detection of unseen fake video. We propose Multi-Crise-Cross Attention and StyleGANv2 Generative Adversarial Network (MCS-GAN). Firstly, we built a Generative Adversarial Network (GAN) framework to learn the distribution of real face data and generate corresponding face images. Secondly, to break the high stitch between the fake region and the background, the model needs to have strong enough feature analysis and pixel restoration capabilities. Therefore, we propose a generator consisting of a Multi-Crise-Cross-Attention (MC) encoder and a StyleGANv2 (SG2) decoder. Finally, to avoid the situation where as long as a face is normal or different faces are abnormal, we set a latent space encoding discriminator and increase the ratio of latent space vector, so as to detect anomaly generated by the forgery operation acting on latent space. We conduct some model generalization experiments on videos on the Internet and some popular deepfake databases. The results show that the accuracy of our method is better compared with the best methods.
11 September 2023 in IEEE Transactions on Multimedia
Face-swapping technology has been widely used in people's life, and people also put forward higher requirements for it. Most of the current face-swapping methods are difficult to generate a high-definition face image. Through StyleGAN, we can generate high-definition face images. However, face-swapping with StyleGAN is still challenging. Firstly, we need to map the target image to the latent space of StyleGAN. Many tasks need to map the input image to a new latent space for face-swapping, because identity features are complex and challenging to map to specific latent space layers directly. So face-swapping is completed in the remapping process, which consumes excess computing resources for reconstruction. And the generated image is difficult to maintain the original image color, face attributes, background and other attributes. We propose a new method, which only edits the code of w+ latent space of StyleGAN to complete the face-swapping and generate high-definition face images. We propose the GAN inversion method to improve the effect of face swapping, which combines convolution networks' advantages in extracting texture features and the benefits of transformers in extracting structure features. In the latent space of StyleGAN, the low-level feature layer is dominated by structure information, and the high-level feature layer is overwhelmed by texture information. Furthermore, we propose latent space selection, through which the neural network can learn disentangled representations of identity information in the latent space. Finally, we improved the post-processing process of face swapping to keep the image's background. Our method can complete face-swapping by editing the w+ space. Thus, high-quality face image can be generated and a lot of computing resource is saved on image reconstruction. At the same time, our method can keep other attributes better in the face-swapping process.
11 September 2023 in IEEE Transactions on Multimedia
Research in psychology demonstrates that visual features and semantic content can convey various emotions. Furthermore, studies have proved that image emotion and aesthetics are inextricably linked. During the image aesthetic assessment process (IAA), images elicit emotional responses from individuals, leading to emotional resonance and influencing the evaluation of images. This paper proposes an image aesthetics assessment method based on hypernetwork of emotion fusion (HNEF). Our method incorporates the emotions depicted in images into the process of IAA. To accomplish this, we extract both aesthetic and emotional features from the images. Additionally, we employed the self-attention mechanism of the transformer to comprehensively investigate the intimate connection between aesthetics and emotion. Additionally, the hypernetwork is designed to establish perception rules governing the high-level semantic information in images. The experimental results validate the strong correlation between emotion and aesthetics. Furthermore, the proposed method exhibits a significantly competitive advantage when compared to existing methods on the Aesthetic Visual Analysis (AVA) dataset. The code is available on https://github.com/LanGuipeng/Image-Aesthetics-Assessment-Based-on-Hypernetwork-of-Emotion-Fusion .
30 October 2023 in IEEE Journal of Biomedical and Health Informatics
In the decade, artificial intelligence has achieved great popularity and applications in medicine and healthcare. Various AI-based algorithms have shown astonishing performance. However, in various data-driven smart healthcare algorithms, the problem of incomplete dataset remains a huge challenge. In this paper, we propose a data completeness enhancement algorithm based on generative AI (i.e., GenAI-DAA) to solve the problems of the in-sufficient data for model training, the data imbalance, and the biases of the training samples. We first construct the cognitive field of the generative models and effectively understand the state of incomplete cognition in generative models. Secondly, on this basis, we propose a quest algorithm for abnormal samples in the cognitive field based on local outlier factor. By fine-grained value evaluation, abnormal samples are given more refined attention. Finally, integrating the above process through multiple cognitive adjustments, GenAI-DAA gradually improves the cognitive ability. GenAI-DAA can be summarized as “Quest−→Estimate−→Tune-up”. We have conducted extensive experiments to demonstrate the effectiveness of our proposed algorithm, and shown widely applications to some typical data-driven smart healthcare algorithms
25 October 2022 in IEEE Transactions on Computational Social Systems
With the rapid development of information technology, social network has become an indispensable part of daily life. People have been able to get news from all over the world through social networks for a long time. People spend more time online than they do in real life. However, the information we get in the world of social network is not purely benign. Due to the development of artificial intelligence technology, more and more tampered media information appears in social networks, some for entertainment, while others become the dark side of social networks, of which the most harmful is to people in the media tamper. For fake news and misinformation caused by media tampering, we need to trace the source and clearly distinguish the truth from the manipulated. This article proposes an image media forgery classification method of multiorder attribute nodes. First, we use different methods to extract the edge, texture, grayscale, and color attributes of the image. Second, according to the characteristics of different attributes, we calculate the first-order entropy of edge attributes, the second-order entropy of texture attributes, local entropy of grayscale, and color properties. Finally, we represent each image with some nodes and build a graph convolutional network (GCN) to classify real and fake images. Experimental results on mainstream media manipulation datasets show that our method is the state-of-the-art compared with similar methods.