1 July 2019 in IEEE/ACM Transactions on Audio, Speech, and Language Processing
Automatic speech emotion recognition has been a research hotspot in the field of human-computer interaction over the past decade. However, due to the lack of research on the inherent temporal relationship of the speech waveform, the current recognition accuracy needs improvement. To make full use of the difference of emotional saturation between time frames, a novel method is proposed for speech recognition using frame-level speech features combined with attention-based long short-term memory (LSTM) recurrent neural networks. Frame-level speech features were extracted from waveform to replace traditional statistical features, which could preserve the timing relations in the original speech through the sequence of frames. To distinguish emotional saturation in different frames, two improvement strategies are proposed for LSTM based on the attention mechanism: first, the algorithm reduces the computational complexity by modifying the forgetting gate of traditional LSTM without sacrificing performance and second, in the final output of the LSTM, an attention mechanism is applied to both the time and the feature dimension to obtain the information related to the task, rather than using the output from the last iteration of the traditional algorithm. Extensive experiments on the CASIA, eNTERFACE, and GEMEP emotion corpora demonstrate that the performance of the proposed approach is able to outperform the state-of-the-art algorithms reported to date.
9 March 2020 in IEEE Access
Because traditional single-channel speech enhancement algorithms are sensitive to the environment and perform poorly, a speech enhancement algorithm based on attention-gated long short-term memory (LSTM) is proposed. To simulate human auditory perceptual characteristics, the algorithm divides the frequency band according to the Bark scale. Based on these bands, bark frequency cepstral coefficients (BFCCs), their derivative features and pitch-based features are extracted. Furthermore, considering that different noises have different influence on the clean speech, the attention mechanism is applied to screen out the information less polluted by noise, which is helpful to reconstruct the clean speech. To adaptively reallocate the power ratio of the speech and noise during the construction of the ratio mask, the ideal ratio mask (IRM) with the inter-channel correlation (ICC) is adopted as the learning target. In addition, to improve the performance of the network, the algorithm introduces a multiobjective learning strategy to jointly optimize the networks by using a voice activity detector (VAD). Subjective and objective experiments show that the proposed algorithm outperforms other baseline algorithms. In real-time experiment, the proposed algorithm maintains high real-time performance and fast convergence speed.
22 November 2018 in IEEE Access
Despite the widespread use of multi-physiological parameters for deception detection, they have been severely restricted due to the high degree of cooperation in contacting-detection. Therefore, a non-contacting method is proposed for deception detection using acoustic features as an input and Convolutional Bidirectional Long Short-Term Memory (CovBiLSTM) as a classifier. The algorithm extracts frame-level acoustic features whose dimension dynamically varies with the length of speech, in order to preserve the temporal information in the original speech. Bidirectional Long Short-Term Memory (BiLSTM) was applied to match temporal features with variable dimension in order to learn the context dependencies in speech. Furthermore, the convolution operation replaces multiplication in traditional LSTM, which is used to excavate time-frequency mixed data. The average accuracy of the experiment on Columbia-SRI-Colorado corpus reaches 70.3%, which is better than the previous works with non-contacting modes.
15 January 2020 in IEEE Access