Deepfake technology uses artificial intelligence to create realistic but false audio, images, and videos. Deepfake technology poses a significant threat to the authenticity of visual content, particularly in live-stream scenarios where the immediacy of detection is crucial. Existing Deepfake detection approaches have limitations and challenges, prompting the need for more robust and accurate solutions. This research proposes an innovative approach: combining eye movement analysis with a hybrid deep learning model to address the need for real-time Deepfake detection. The proposed hybrid deep learning model integrates two deep neural network architectures, MesoNet4 and ResNet101, to leverage their respective architectures’ strengths for effective Deepfake classification. MesoNet4 is a lightweight CNN model designed explicitly to detect subtle manipulations in facial images. At the same time, ResNet101 handles complex visual data and robust feature extraction. Combining the localized feature learning of MesoNet4 with the deeper, more comprehensive feature representations of ResNet101, our robust hybrid model achieves enhanced performance in distinguishing between manipulated and authentic videos, which cannot be conducted with the naked eye or traditional methods. The model is evaluated on diverse datasets, including FaceForensics++, CelebV1, and CelebV2, demonstrating compelling accuracy results, with the hybrid model attaining an accuracy of 0.9873 on FaceForensics++, 0.9689 on CelebV1, and 0.9790 on CelebV2, showcasing its robustness and potential for real-world deployment in content integrity verification and video forensics applications.
11 March 2025 in Signal, Image and Video Processing
Deepfake detection has become a critical challenge nowadays with the rise of sophisticated generative techniques that manipulate audio-visual data. Existing methods primarily focus on lip movement synchronization using audio and visual features, often relying on local feature extraction with Convolutional Neural Networks (CNNs). In this work, we propose an enhanced multimodal framework that integrates with local and global features for advanced deepfake detection. Our approach extends traditional pipelines by introducing additional visual features such as eye movement and facial regions, combined with audio features to model cross-modal dependencies. While CNNs capture local features, Vision Transformers (ViTs) extract global contextual relationships from both visual and audio modalities. The diffusion models are incorporated as pre-processors to refine noisy data and generate realistic augmentations, ensuring high-quality feature representation. The proposed framework achieves state-of-the-art performance, with accuracy scores of 0.9987, 0.9825, 0.9915, and 0.9812 on the FakeAVCeleb, AV-Deepfake1M, TVIL, and LAV-DF datasets, respectively. These results demonstrate significant improvements over existing methods, highlighting the framework’s superior generalization and robustness in detecting subtle inconsistencies across manipulated audio-visual data.
27 December 2024 in ICCK Transactions on Intelligent Systematics
Indoor scene recognition poses considerable hurdles, especially in cluttered and visually analogous settings. Although several current recognition systems perform well in outside settings, there is a distinct necessity for enhanced precision in inside scene detection, particularly for robotics and automation applications. This research presents a revolutionary deep Convolutional Neural Network (CNN) model tailored with bespoke parameters to improve indoor image comprehension. Our proprietary dataset consists of seven unique interior scene types, and our deep CNN model is trained to attain excellent accuracy in classification tasks. The model exhibited exceptional performance, achieving a training accuracy of 99%, a testing accuracy of 89.73%, a precision of 90.11%, a recall of 89.73%, and an F1-score of 89.79%. These findings underscore the efficacy of our methodology in tackling the intricacies of indoor scene recognition. This research substantially advances the domain of robotics and automation by establishing a more resilient and dependable framework for autonomous navigation and scene comprehension in GPS-denied settings, facilitating the development of more efficient and intelligent robotic systems.
Recently, renewable energy sources integrated with microgrid (MG) networks have provided safe, secure, and reliable power supply to both utility and industrial purposes. Power quality disturbances (PQDs) seriously afect the performance of MG networks and reduce the lifecycle of numerous sensitive devices in MG networks. Hence, this paper presents a new approach to detect and classify the PQDs using discrete wavelet transform, multiresolution analysis, and optimized-kernel support vector machine. Te obtained unique features from DWT-MRA are fed to train the well-known intelligent classifers. In the optimizedkernel SVM model, computing power is enhanced for classifying multiple PQ events based on the local density and leave-one-out (LOO) algorithm. To get higher separation in feature space, the kernel width of each sample is estimated based on the local density. By using the LOO method, an improved grid search strategy is implemented to get the penalty parameter to achieve satisfactory results. Moreover, a typical MG network is simulated in MATLAB software considering the validation of the proposed technique to address the power quality issues in MG networks, and the results of the proposed method are compared with other conventional ML classifers. Te simulation results confrm that the proposed method is more efective and accurate than other intelligent classifers.
20 November 2024 in ICCK Transactions on Emerging Topics in Artificial Intelligence
Recently, the primary reason for blindness in adults has been diabetic retinopathy (DR) disease. Therefore, there is an increasing demand for a real-time efficient classification and detection system for diabetic retinopathy (DR) to overcome fast-growing disease (DR). We introduced a novel deep hybrid model for automated diabetic retinopathy (DR) disease recognition and classification. Our model leverages the power of CNN architectures: Inception V3 and VGG16 models by combining their strengths to cater to exact requirements. VGG16 model efficiently captures fine features and wide-ranging features such as textures and edges, crucial for classifying initial signs of DR. Similarly, Inception V3’s architecture is proficient at detecting multiscale patterns, providing an extensive setting for shaping the occurrence of more complex DR severity stages. Our deep hybrid model allows the extraction of various appearance features in retinal images, which can better assist the classification and detection of DR. Our proposed model was evaluated on diverse datasets, including EyePACS1 and APTOS2019, demonstrating confident performance of 99.63% accuracy in classifying the DR severity levels on EyePACS1 dataset, while 98.70% accuracy on the APTOS2019 dataset, indicating that our proposed deep hybrid model well distinguished different stages and highly efficient in DR detection. This model helps clinicians and medical experts to classify and identify diabetic retinopathy DR stages and severity levels early. This automatic system helps to manage and treat the patient more effectively and introduces timely treatment.
About Me
Master of Science in Computer Science and Technology & Researcher at Southeast University (China)
Master of Science in Computer Science and Technology & Researcher at Southeast University (China)