Laboratoire Biomécanique et Bioingénierie UMR 7338, Université de Technologie de Compiègne, Centre de Recherches de Royallieu, CS- 20529 - 60205 Compiègne Cedex, France
Dr. Imad Rida
Laboratoire Biomécanique et Bioingénierie UMR 7338, Université de Technologie de Compiègne, Centre de Recherches de Royallieu, CS- 20529 - 60205 Compiègne Cedex, France
Multimodal sentiment analysis faces significant challenges in real-world applications due to the frequent absence of modalities caused by privacy concerns, device limitations, or security policies. This article introduces a random modality dropout based on generative approach (RMDG), designed to enhance the robustness and performance of multimodal models under various modality absence scenarios. The RMDG method employs a generative approach during the training phase, where random modality dropout is applied to simulate missing modalities. By leveraging the remaining modalities to predict and regenerate the key features of the missing ones, the model effectively adapts to dynamic and unpredictable modality absences. This strategy not only eliminates the need for separate training or adjustments for each modality combination but also significantly improves the efficiency and accuracy of sentiment analysis in incomplete multimodal data scenarios. Extensive experiments demonstrate that RMDG outperforms existing methods, achieving superior performance in both complete and missing modality conditions.
12 September 2024 in ACM Transactions on Multimedia Computing, Communications, and Applications
Recent advances in artificial intelligence have led to deepfake images, enabling users to replace a real face with a genuine one. deepfake images have recently been used to malign public figures, politicians, and even average citizens. deepfake but realistic images have been used to stir political dissatisfaction, blackmail, propagate false news, and even carry out bogus terrorist attacks. Thus, identifying real images from fakes has got more challenging. To avoid these issues, this study employs transfer learning and data augmentation technique to classify deepfake images. For experimentation, 190,335 RGB-resolution deepfake and real images and image augmentation methods are used to prepare the dataset. The experiments use the deep learning models: convolutional neural network (CNN), Inception V3, visual geometry group (VGG19), and VGG16 with a transfer learning approach. Essential evaluation metrics (accuracy, precision, recall, F1-score, confusion matrix, and AUC-ROC curve score) are used to test the efficacy of the proposed approach. Results revealed that the proposed approach achieves an accuracy, recall, F1-score and AUC-ROC score of 90% and 91% precision, with our fine-tuned VGG16 model outperforming other DL models in recognizing real and deepfakes.
Among various palmprint identification methods proposed in the literature, sparse representation for classification (SRC) is very attractive offering high accuracy. Although SRC has good discriminative ability, its performance strongly depends on the quality of the training data. In particular, SRC suffers from two major problems: lack of training samples per class and large intra-class variations. In fact, palmprint images not only contain identity information but they also have other information, such as illumination and geometrical distortions due to the unconstrained conditions and the movement of the hand. In this case, the sparse representation assumption may not hold well in the original space since samples from different classes may be considered from the same class. This paper aims to enhance palmprint identification performance through SRC by proposing a simple yet efficient method based on an ensemble of sparse representations through an ensemble of discriminative dictionaries satisfying SRC assumption. The ensemble learning has the advantage to reduce the sensitivity due to the limited size of the training data and is performed based on random subspace sampling over 2D-PCA space while keeping the image inherent structure and information. In order to obtain discriminative dictionaries satisfying SRC assumption, a new space is learned by minimizing and maximizing the intra-class and inter-class variations using 2D-LDA. Extensive experiments are conducted on two publicly available palmprint data sets: multispectral and PolyU. Obtained results showed very promising results compared with both state-of-the-art holistic and coding methods. Besides these findings, we provide an empirical analysis of the parameters involved in the proposed technique to guide the neophyte.
21 March 2015 in Signal, Image and Video Processing
With ever-increasing dataset size and data storage capacity, there is a strong need to build systems that can effectively utilize these vast datasets to extract valuable information. Large datasets often exhibit sparsity and pose cold start problems, necessitating the development of responsible recommender systems. Knowledge graphs have utility in responsibly representing information related to recommendation scenarios. However, many studies overlook explicitly encoding contextual information, which is crucial for reducing the bias of multi-layer propagation. Additionally, existing methods stack multiple layers to encode high-order neighbor information while disregarding the relational information between items and entities. This oversight hampers their ability to capture the collaborative signal latent in user-item interactions. This is particularly important in health informatics, where knowledge graphs consist of various entities connected to items through different relations. Ignoring the relational information renders them insufficient for modeling user preferences. This work presents an end-to-end recommendation framework named KGCAN (Knowledge Graph Enhanced Contextualized Attention-Based Network), which explicitly encodes both relational and contextual information of entities to preserve the original entity information. Furthermore, a user-specific attention mechanism is employed to capture personalized recommendations. The proposed model is validated on three benchmark datasets through extensive experiments. The experimental results demonstrate that KGCAN outperforms existing knowledge graph based recommendation models. Additionally, a case study from the healthcare domain is discussed, highlighting the importance of attention mechanisms and high-order connectivity in the responsible recommendation system for health informatics.