Regularization approaches, such as multi-task learning and dropout, prevent overfitting and improve generalization ability. Speech emotion recognition suffers from insufficiently transcribed databases, where labels are subjectively annotated. Because emotions are a more universally recognized language, the paralinguistic feature space of emotional speech can be better generalized, even across substantially heterogeneous languages. We investigate the effect of regularization and normalization frameworks on two emotional speech databases, the IEMOCAP for English and the JTES for Japanese. We obtain absolute gains of unweighted average recall over ten runs (1.48% for the IEMOCAP and 1.03% for the JTES) and achieve a maximum of 59.49% on the IEMOCAP. From comparative experiments, we confirm that dropout and multi-task learning strategies are effective for multilingual speech emotion recognition, and common normalization over two languages leads to further improvement under all conditions, which suggests that better generalization is available even when two highly heterogeneous languages are merged.
The paper describes subword-based approaches for open-vocabulary spoken document retrieval. First, the feasibility of subword units in spoken document retrieval is investigated, and our previously proposed sub-phonetic segment units are compared to typical subword units, such as syllables, phonemes, and triphones. Next, we explore the linear combination of retrieval score from multiple subword representations to improve retrieval performance. Experimental evaluation of open-vocabulary spoken document retrieval tasks demonstrates that our proposed sub-phonetic segment units are more effective than typical subword units, and the linear combination of multiple subword representations resulted in a consistent improvement in the F-measure.
Much research has been conducted on spoken term detection. Query-by-example has also been an important topic in spoken term detection, which covers spoken queries. A previous study examined posteriorgrams, which are sequences of output probabilities generated by a deep neural network from speech queries and speech data. Although posteriorgram matching between a spoken query and speech data exhibits improved retrieval accuracy, the time required to search with a spoken query is long, even for a relatively small quantity of speech data. Reducing retrieval time is thus a crucial problem. In this paper, we propose two methods for reducing retrieval time in posteriorgram matching. One is to accelerate posteriorgram matching by transforming a posteriorgram to a bit matrix, and the other is to use a sparse vector method. The first method, "posteriorgram bit operation," transforms the posteriorgrams of both spoken queries and speech data to bit matrices. The second method retains a small number of elements, of which only a few are of high probability, in a posteriorgram. Because most of the elements in a sparse vector are 0, the thousands of output probabilities of the posteriorgram are reduced to only a small number of output probabilities. Evaluation experiments have been carried out using open test collections (Spoken-Doc tasks of NTCIR-10 workshops) [1,2], and the results have demonstrated the effectiveness of the proposed method.