Speech is one of the most natural communication channels for expressing human emotions. Therefore, speech emotion recognition (SER) has been an active area of research with an extensive range of applications that can be found in several domains, such as biomedical diagnostics in healthcare and human–machine interactions. Recent works in SER have been focused on end-to-end deep neural networks (DNNs). However, the scarcity of emotion-labeled speech datasets inhibits the full potential of training a deep network from scratch. In this paper, we propose new approaches for classifying emotions from speech by combining conventional mel-frequency cepstral coefficients (MFCCs) with image features extracted from spectrograms by a pretrained convolutional neural network (CNN). Unlike prior studies that employ end-to-end DNNs, our methods eliminate the resource-intensive network training process. By using the best prediction model obtained, we also build an SER application that predicts emotions in real time. Among the proposed methods, the hybrid feature set fed into a support vector machine (SVM) achieves an accuracy of 0.713 in a 6-class prediction problem evaluated on the Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS) dataset, which is higher than the previously published results. Interestingly, MFCCs taken as unique input into a long short-term memory (LSTM) network achieve a slightly higher accuracy of 0.735. Our results reveal that the proposed approaches lead to an improvement in prediction accuracy. The empirical findings also demonstrate the effectiveness of using a pretrained CNN as an automatic feature extractor for the task of emotion prediction. Moreover, the success of the MFCC-LSTM model is evidence that, despite being conventional features, MFCCs can still outperform more sophisticated deep-learning feature sets.
Fault diagnosis plays an essential role in reducing the maintenance costs of rotating machinery manufacturing systems. In many real applications of fault detection and diagnosis, data tend to be imbalanced, meaning that the number of samples for some fault classes is much less than the normal data samples. At the same time, in an industrial condition, accelerometers encounter high levels of disruptive signals and the collected samples turn out to be heavily noisy. As a consequence, many traditional Fault Detection and Diagnosis (FDD) frameworks get poor classification performances when dealing with real-world circumstances. Three main solutions have been proposed in the literature to cope with this problem: (1) the implementation of generative algorithms to increase the amount of under-represented input samples, (2) the employment of a classifier being powerful to learn from imbalanced and noisy data, (3) the development of an efficient data preprocessing including feature extraction and data augmentation. This paper proposes a hybrid framework which uses the three aforementioned components to achieve an effective signal based FDD system for imbalanced conditions. Specifically, it first extracts the fault features, using Fourier and wavelet transforms to make full use of the signals. Then, it employs Wasserstein Generative Adversarial with Gradient Penalty Networks (WGAN-GP) to generate synthetic samples to populate the rare fault class and enrich the training set. Moreover, to achieve a higher performance a novel combination of Convolutional Long Short-term Memory (CLSTM) and Weighted Extreme Learning Machine (WELM) is also proposed. To verify the effectiveness of the developed framework, different bearing datasets settings on different imbalance severities and noise degrees were used. The comparative results demonstrate that in different scenarios GAN-CLSTM-ELM significantly outperforms the other state-of-the-art FDD frameworks.
19 May 2007 in Computational Optimization and Applications
The article proposes the use of a smartwatch-based system for measuring the emotions of individuals in a classroom setting with respect to five mood variables: Activation, Tiredness, Pleasance, Quality of Presentation and Understanding. Internal (body) and external (environment) data such as movement, heart rate, noise, temperature and humidity were collected through the built-in sensors of the smartwatch. The system was verified by means of a longitudinal study that has been carried out in a series of workshops and lectures. Through experience-based sampling, participants were polled at periodic time intervals asking them to enter a self-assessment of the aforementioned mood states directly on the smartwatch. The goal was to demonstrate whether sensor data can be used to effectively predict the five moods. By resorting to a machine learning approach our system was able to predict the moods with an accuracy ranging between 89-95 percent for single-output classification, 92-99 percent for the chain classification task and of approximately 93 percent for the multi-output analysis. Our results showed also that body signals are better predictors compared to the external environmental variables. These results demonstrate and verify the potential of smartwatches in collecting and predicting human emotions, enabling dynamic feedback loops to enhance user experience.