Tomoki Toda

Tomoki Toda

Auto-generated profile, based on publicly available information

Tomoki Toda

Information Technology Center, Nagoya University, Nagoya, Japan

Research Focus & Metrics

Academic Career Length

20Years (2007-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
5 January 2021 in IEEE/ACM Transactions on Audio, Speech, and Language Processing
Verified
Journal Article
Open access
WHWen-Chin Huang
Wen-Chin Huang
THTomoki Hayashi
Tomoki Hayashi
YWYi-Chiao Wu
Yi-Chiao Wu
HKHirokazu Kameoka
Hirokazu Kameoka
TTTomoki Toda
Tomoki Toda
Sequence-to-sequence (seq2seq) voice conversion (VC) models are attractive owing to their ability to convert prosody. Nonetheless, without sufficient data, seq2seq VC models can suffer from unstable training and mispronunciation problems in the converted speech, thus far from practical. To tackle these shortcomings, we propose to transfer knowledge from other speech processing tasks where large-scale corpora are easily available, typically text-to-speech (TTS) and automatic speech recognition (ASR). We argue that VC models initialized with such pretrained ASR or TTS model parameters can generate effective hidden representations for high-fidelity, highly intelligible converted speech. In this work, we examine our proposed method in a parallel, one-to-one setting. We employed recurrent neural network (RNN)-based and Transformer based models, and through systematical experiments, we demonstrate the effectiveness of the pretraining scheme and the superiority of Transformer based models over RNN-based models in terms of intelligibility, naturalness, and similarity.


24 December 2020 in IEEE/ACM Transactions on Audio, Speech, and Language Processing
Verified
Journal Article
Closed access
HKHirokazu Kameoka
Hirokazu Kameoka
WHWen-Chin Huang
Wen-Chin Huang
KTKou Tanaka
Kou Tanaka
TKTakuhiro Kaneko
Takuhiro Kaneko
NHNobukatsu Hojo
Nobukatsu Hojo
TTTomoki Toda
Tomoki Toda
This paper proposes a voice conversion (VC) method based on a sequence-to-sequence (S2S) learning framework, which enables simultaneous conversion of the voice characteristics, pitch contour, and duration of input speech. We previously proposed an S2S-based VC method using a transformer network architecture called the voice transformer network (VTN). The original VTN was designed to learn only a mapping of speech feature sequences from one speaker to another. The main idea we propose is an extension of the original VTN that can simultaneously learn mappings among multiple speakers. This extension called the many-to-many VTN makes it able to fully use available training data collected from multiple speakers by capturing common latent features that can be shared across different speakers. It also allows us to introduce a training loss called the identity mapping loss to ensure that the input feature sequence will remain unchanged when the source and target speaker indices are the same. Using this particular loss for model training has been found to be extremely effective in improving the performance of the model at test time. We conducted speaker identity conversion experiments and found that our model obtained higher sound quality and speaker similarity than baseline methods. We also found that our model, with a slight modification to its architecture, could handle any-to-many conversion tasks reasonably well.

25 July 2022 in IEEE Journal of Selected Topics in Signal Processing
Verified
Journal Article
Closed access
WHWen-Chin Huang
Wen-Chin Huang
SYShu-Wen Yang
Shu-Wen Yang
THTomoki Hayashi
Tomoki Hayashi
TTTomoki Toda
Tomoki Toda
We present a large-scale comparative study of self-supervised speech representation (S3R)-based voice conversion (VC). In the context of recognition-synthesis VC, S3Rs are attractive owing to their potential to replace expensive supervised representations such as phonetic posteriorgrams (PPGs), which are commonly adopted by state-of-the-art VC systems. Using S3PRL-VC, an open-source VC software we previously developed, we provide a series of in-depth objective and subjective analyses under three VC settings: intra-/cross-lingual any-to-one (A2O) and any-to-any (A2A) VC, using the voice conversion challenge 2020 (VCC2020) dataset. We investigated S3R-based VC in various aspects, including model type, multilinguality, and supervision. We also studied the effect of a post-discretization process with k-means clustering and showed how it improves in the A2A setting. Finally, the comparison with state-of-the-art VC systems demonstrates the competitiveness of S3R-based VC and also sheds light on the possible improving directions.

15 January 2020 in IEEE/ACM Transactions on Audio, Speech, and Language Processing
Verified
Journal Article
Closed access
AAAtsushi Ando
Atsushi Ando
RMRyo Masumura
Ryo Masumura
HKHosana Kamiyama
Hosana Kamiyama
SKSatoshi Kobashikawa
Satoshi Kobashikawa
YAYushi Aono
Yushi Aono
TTTomoki Toda
Tomoki Toda
This paper presents a novel customer satisfaction (CS) estimation method that outputs both turn-level and call-level estimations simultaneously. Our key idea is to directly apply turn-level estimation results to call-level estimation and optimize them jointly; previous works treat both as being independent. Our proposal applies long short-term memory recurrent neural networks (LSTM-RNNs) to turn-level and call-level CS estimation to capture long-range sequential context in contact center calls. In addition, both networks are hierarchically stacked so as to use turn-level estimation results for call-level estimation directly. In order to learn the relationship between the two tasks, we also introduce joint optimization training to the stacked model. Several analyses of turn-level and call-level CS are provided on acted and real calls to support the proposed method. Experiments show that the proposed framework outperforms the conventional methods in both turn-level and call-level estimations.

14 January 2021 in IEEE/ACM Transactions on Audio, Speech, and Language Processing
Verified
Journal Article
Open access
YWYi-Chiao Wu
Yi-Chiao Wu
THTomoki Hayashi
Tomoki Hayashi
TOTakuma Okamoto
Takuma Okamoto
HKHisashi Kawai
Hisashi Kawai
TTTomoki Toda
Tomoki Toda
In this paper, we propose a quasi-periodic parallel WaveGAN (QPPWG) waveform generative model, which applies a quasi-periodic (QP) structure to a parallel WaveGAN (PWG) model using pitch-dependent dilated convolution networks (PDCNNs). PWG is a small-footprint GAN-based raw waveform generative model, whose generation time is much faster than real time because of its compact model and non-autoregressive (non-AR) and non-causal mechanisms. Although PWG achieves high-fidelity speech generation, the generic and simple network architecture lacks pitch controllability for an unseen auxiliary fundamental frequency (F 0 ) feature such as a scaled F 0 . To improve the pitch controllability and speech modeling capability, we apply a QP structure with PDCNNs to PWG, which introduces pitch information to the network by dynamically changing the network architecture corresponding to the auxiliary F 0 feature. Both objective and subjective experimental results show that QPPWG outperforms PWG when the auxiliary F 0 feature is scaled. Moreover, analyses of the intermediate outputs of QPPWG also show better tractability and interpretability of QPPWG, which respectively models spectral and excitation-like signals using the cascaded fixed and adaptive blocks of the QP structure.