In one study on vocal emotion recognition using noise-vocoded speech (NVS), the high similarities between modulation spectral features (MSFs) and the results of vocal-emotion-recognition experiments indicated that MSFs contribute to vocal emotion recognition in a clean environment (with no noise and no reverberation). Other studies also clarified that vocal emotion recognition using NVS is not affected by noisy reverberant environments (signal-to-noise ratio is greater than 10 dB and reverberation time is less than 1.0 s). However, the contribution of MSFs to vocal emotion recognition in noisy reverberant environments is still unclear. We aimed to clarify whether MSFs can be used to explain the vocal-emotion-recognition results in noisy reverberant environments. We analyzed the results of vocal-emotion-recognition experiments and used an auditory-based modulation filterbank to calculate the modulation spectrograms of NVS. We then extracted ten MSFs as higher-order statistics of modulation spectrograms. As shown from the relationship between MSFs and vocal-emotion-recognition results, except for extremely high noisy reverberant environments, there were high similarities between MSFs and the vocal emotion recognition results in noisy reverberant environments, which indicates that MSFs can be used to explain such results in noisy reverberant environments. We also found that there are two common MSFs (MSKT${}_k$ (modulation spectral kurtosis) and MSTL${}_k$ (modulation spectral tilt)) that contribute to vocal emotion recognition in all daily environments.

Purpose: Previous studies using noise-vocoded speech (NVS) have demonstrated the significance of the temporal amplitude envelope (TAE) of speech signals, such as modulation perception, in vocal emotion perception. In addition, due to the importance of modulation processing for TAE in speech perception, researchers began to focus on the role of TAE modulation components. A previous study suggested the contributions of modulation frequency components in vocal emotion perception. However, the important components remain unclear. This study aims to clarify the important components in vocal emotion perception. Method: Two experiments on vocal emotion perception using NVS were conducted with 10 native Japanese speakers (two women and eight men). In NVS generation, a modulation filterbank (MFB) is used to simulate modulation perception in the auditory system. The modulation frequency components of TAE are bandpass and bandstop filtered using the filterbank. The contributions of the individual modulation frequency components are evaluated by comparing the emotion recognition rates of NVS. Results: The results indicate that the use of an MFB does not affect emotion perception in NVS. The modulation frequency components within the 0- to 16-Hz band are important for each emotion, as well as for all emotions collectively. The important modulation frequency components for vocal emotion perception may differ slightly between positive and negative emotions. However, this observation should be interpreted cautiously and needs more verification due to the imbalance in the number of emotional categories in this study. Conclusion: This study investigated the important modulation frequency components of TAE that contribute to vocal emotion perception and suggested that modulation frequency components in the 0- to 16-Hz band are important components in vocal emotion perception.