Generative AI technologies, including text-to-speech (TTS) and voice conversion (VC), frequently become indistinguishable from genuine samples, posing challenges for individuals in discerning between real and synthetic content. This indistinguishability undermines trust in media, and the arbitrary cloning of personal voice signals presents significant challenges to privacy and security. In the field of deepfake audio detection, the majority of models achieving higher detection accuracy currently employ self-supervised pre-trained models. However, with the ongoing development of deepfake audio generation algorithms, maintaining high discrimination accuracy against new algorithms grows more challenging. To enhance the sensitivity of deepfake audio features, we propose a deepfake audio detection model that incorporates an SLS (Sensitive Layer Selection) module. Specifically, utilizing the pre-trained XLS-R enables our model to extract diverse audio features from its various layers, each providing distinct discriminative information. Utilizing the SLS classifier, our model captures sensitive contextual information across different layer levels of audio features, effectively employing this information for fake audio detection. Experimental results show that our method achieves state-of-the-art (SOTA) performance on both the ASVspoof 2021 DF and In-the-Wild datasets, with a specific Equal Error Rate (EER) of 1.92% on the ASVspoof 2021 DF dataset and 7.46% on the In-the-Wild dataset. Codes and data can be found at https://github.com/QiShanZhang/SLSforADD.
5 March 2021 in IEEE Transactions on Image Processing
Automatic image visual recognition can make full use of largely available images with text descriptions on social media platforms to build large-scale image labeled datasets. In this paper, we propose a novel visual text representation, named DG-VRT (Diverse GAN-Visual Representation on Text), which extracts visual features from synthetic images generated by a diverse conditional Generative Adversarial Network (DCGAN) on the text, for visual recognition. The DCGAN incorporates the current state-of-the-art text-to-image GANs and generates multiple synthetic images with various prior noises conditioned on a text. Then we extract deep visual features from the generated synthetic images to explore the underlying visual concepts and provide a visual transformation on text in feature space. Finally, we combine image-level visual features, text-level features and visual features based on synthetic images together to recognize the images, and we also extend the proposed work to semantic segmentation. We conduct extensive experiments on two benchmark datasets and the experimental results demonstrate the efficacy of our proposed representation on text for visual recognition.
18 November 2024 in IEEE Signal Processing Letters
Artificial intelligence technology has been developing rapidly, and speech synthesis models have become increasingly mature, capable of generating highly realistic synthetic audio used to disseminate misinformation, which poses a serious security risk problem. Digital watermarking technology can effectively protect digital content. Deep learning is currently achieving significant research success in digital watermarking. However, the current robustness against audio manipulation remains understudied. Based on this, we propose a robust audio watermarking method based on deep learning against manipulation attacks. Specifically, the embedding of watermarking information is performed in the encoder and the extraction of watermarking information is performed in the decoder; In addition, various audio attacks are simulated during iterative training, a sampling noise layer is used to increase robustness, and a discriminator is used to distinguish between encoded audio and original audio to improve the invisibility of the watermark. We comprehensively evaluate the performance of our model against various manipulation attacks. Experimental results demonstrate that the framework effectively embeds and extracts watermarked signals, exhibiting strong robustness.
23 July 2020 in Journal of Visual Communication and Image Representation
Robot joints are the main structure for controlling the motion of the machine body, where the motion state of them directly affects the performance of the industrial robot. Due to the difficulty of obtaining the joint torque information of industrial robots, it is very hard to monitor the motion state of them. Based on the velocity and force driven by current of motors, we propose a novel Bayesian optimization framework to predict the joint motion state of industrial robot in this paper. Based on the temporal correlation of joint current and the correlation between the current and motion state of joint, we use the LSTM and BiLSTM to regressing prediction of the current and state of joint motor first. Then, the Bayesian optimization method is used to adjust the hyperparameters of our network, which realize the analysis of the joint motor current under different motion states and improve the accuracy of the prediction of joint motion states. Finally, we design the joint current acquisition platform of industrial robot based on Hall current sensors, which can collect joint currents without contact and generate experimental dataset. Comparing with the popular intelligent methods, the results show that our Bayesian optimization framework realizes a more accurate prediction of motion state for the four-axis industrial robot on the basis of contact-less current acquisition.
Traffic flow prediction plays a crucial role in Intelligent Transportation Systems (ITS), as it substantially enhances traffic management efficiency, alleviates congestion, and improves road safety. Traditional models often face challenges in addressing the dynamic complexity of modern highway traffic, whereas multivariate machine learning models demonstrate superior predictive accuracy by leveraging diverse data sources. To address these limitations, this study proposes a traffic flow prediction framework based on sensor networks and multivariate machine learning techniques. Real-time vehicle data are collected using cameras deployed along highways, and key traffic parameters such as flow, density, and speed are precisely extracted using the YOLOv8 object detection model. Subsequently, five machine learning algorithms and three deep learning algorithms are employed to predict traffic flow. Building on this, we propose a hybrid model combining Gradient Boosting Regression (GBR) and Support Vector Regression (SVR), where the GBR component captures complex nonlinear patterns in traffic flow data, while the SVR component enhances the model’s generalization ability by optimizing predictions. Experimental results demonstrate that the proposed GBR-SVR model outperforms comparative methods in short-term traffic flow prediction, particularly under dynamic and complex traffic conditions. In conclusion, by integrating real-time data collection from cameras with the predictive advantages of machine learning models, this algorithm offers a more efficient and sustainable solution for urban traffic management.