In recent years, multi-modal analysis has gained significant prominence across domains such as audio/speech processing, natural language processing, and affective computing, with a particular focus on speech emotion recognition (SER). The integration of data from diverse sources, encompassing text, audio, and images, in conjunction with classifier algorithms has led to the realization of enhanced performance in SER tasks. Traditionally, the cross-entropy loss function has been employed for the classification problem. However, it is challenging to discriminate the feature representations among classes for multi-modal classification tasks. In this study, we focus on the impact of the loss functions on multi-modal SER rather than designing the model architecture. Mainly, we evaluate the performance of multi-modal SER with different loss functions, such as cross-entropy loss, center loss, contrastive-center loss, and their combinations. Based on extensive comparative analysis, it is proven that the combination of cross-entropy loss and contrastive-center loss achieves the best performance for multi-modal SER. This combination reaches the highest accuracy of 80.27% and the highest balanced accuracy of 81.44% on the IEMOCAP dataset.
In the field of image and video colorization, the existing research employs a CNN to extract information from each video frame. However, due to the local nature of a kernel, it is challenging for CNN to capture the relationships between each pixel and others in an image, leading to inaccurate colorization. To solve this issue, we introduce an end-to-end network called Vitexco for colorizing videos. Vitexco utilizes the power of the Vision Transformer (ViT) to capture the relationships among all pixels in a frame with each other, providing a more effective method for colorizing video frames. We evaluate our approach on DAVIS datasets and demonstrate that it outperforms the state-of-the-art methods regarding color accuracy and visual quality. Our findings suggest that using a ViT can significantly enhance the performance of video colorization.
In the era of Vietnam’s Industrial Revolution 4.0, technological advancements are crucial, especially in computing and telecommunications. Digital images and videos have become indispensable tools in the modern information age. Human faces hold significant value as biometric identifiers in image and video databases used for surveillance systems. The foundation of model-automated attendance systems utilizing facial recognition is based on taking a face image captured by the camera as input and producing a text recording of the person’s presence. Our study focuses on developing an automatic attendance system based on face recognition using deep learning techniques for real-time recognition. The system includes image capture from a webcam, face detection, dataset management, face recognition, and attendance recording. We investigated and compared three different face recognition models: the VGG-Face model, the Siamese network model, and the Inception-ResNet-v2 model pre-trained. The Siamese Network model achieved the highest accuracy of 94.1% on the LFW dataset. On the other hand, we also trained this enhanced Siamese Network model on the VN-Celeb dataset specifically for testing on our system to be used for FPT University students - Vietnamese students. This enhanced Siamese Network model is integrated into the system, achieving 94.4% accuracy on the FPT University student dataset. This system streamlines attendance recording by eliminating the need for manual attendance taking, thereby minimizing human error and saving time.