Ryandhimas E. Zezario

Ryandhimas E. Zezario

Auto-generated profile, based on publicly available information

Ryandhimas E. Zezario

Research Centre for Information Technology Innovation, Academia Sinica, Taipei, Taiwan

Research Focus & Metrics

Academic Career Length

11Years (2016-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
29 September 2022 in IEEE/ACM Transactions on Audio, Speech, and Language Processing
Verified
Journal Article
Open access
RZRyandhimas E. Zezario
Ryandhimas E. Zezario
SFSzu-Wei Fu
Szu-Wei Fu
FCFei Chen
Fei Chen
CFChiou-Shann Fuh
Chiou-Shann Fuh
HWHsin-Min Wang
Hsin-Min Wang
YTYu Tsao
Yu Tsao
This study proposes a cross-domain multi-objective speech assessment model, called MOSA-Net, which can simultaneously estimate the speech quality, intelligibility, and distortion assessment scores of an input speech signal. MOSA-Net comprises a convolutional neural network and bidirectional long short-term memory architecture for representation extraction, and a multiplicative attention layer and a fully connected layer for each assessment metric prediction. Additionally, cross-domain features (spectral and time-domain features) and latent representations from self-supervised learned (SSL) models are used as inputs to combine rich acoustic information to obtain more accurate assessments. Experimental results show that in both seen and unseen noise environments, MOSA-Net can improve the linear correlation coefficient (LCC) scores in perceptual evaluation of speech quality (PESQ) prediction, compared to Quality-Net, an existing single-task model for PESQ prediction, and improve LCC scores in short-time objective intelligibility (STOI) prediction, compared to STOI-Net, an existing single-task model for STOI prediction. Moreover, MOSA-Net can be used as a pre-trained model to be effectively adapted to an assessment model for predicting subjective quality and intelligibility scores with a limited amount of training data. Experimental results show that MOSA-Net can improve LCC scores in mean opinion score (MOS) predictions, compared to MOS-SSL, a strong single-task model for MOS prediction. We further adopt the latent representations of MOSA-Net to guide the speech enhancement (SE) process and derive a quality-intelligibility (QI)-aware SE (QIA-SE) approach. Experimental results show that QIA-SE outperforms the baseline SE system with improved PESQ scores in both seen and unseen noise environments over a baseline SE model.

5 October 2020 in IEEE/ACM Transactions on Audio, Speech, and Language Processing
Verified
Journal Article
Open access
CYCheng Yu
Cheng Yu
RZRyandhimas E. Zezario
Ryandhimas E. Zezario
SWSyu-Siang Wang
Syu-Siang Wang
JSJonathan Sherman
Jonathan Sherman
YHYi-Yen Hsieh
Yi-Yen Hsieh
YTYu Tsao
Yu Tsao
Deep learning-based models have greatly advanced the performance of speech enhancement (SE) systems. However, two problems remain unsolved, which are closely related to model generalizability to noisy conditions: (1) mismatched noisy condition during testing, i.e., the performance is generally sub-optimal when models are tested with unseen noise types that are not involved in the training data; (2) local focus on specific noisy conditions, i.e., models trained using multiple types of noises cannot optimally remove a specific noise type even though the noise type has been involved in the training data. These problems are common in real applications. In this paper, we propose a novel denoising autoencoder with a multi-branched encoder (termed DAEME) model to deal with these two problems. In the DAEME model, two stages are involved: training and testing. In the training stage, we build multiple component models to form a multi-branched encoder based on a decision tree (DSDT). The DSDT is built based on prior knowledge of speech and noisy conditions (the speaker, environment, and signal factors are considered in this paper), where each component of the multi-branched encoder performs a particular mapping from noisy to clean speech along the branch in the DSDT. Finally, a decoder is trained on top of the multi-branched encoder. In the testing stage, noisy speech is first processed by the tree and fed to each component model. The multiple outputs from these models are then integrated into the decoder to determine the final enhanced speech. Experimental results show that DAEME is superior to several baseline models in terms of objective evaluation metrics, automatic speech recognition results, and quality in subjective human listening tests.

1 May 2020
Verified
Conference Paper
Closed access
RZRyandhimas E. Zezario
Ryandhimas E. Zezario
THTassadaq Hussain
Tassadaq Hussain
XLXugang Lu
Xugang Lu
HWHsin-Min Wang
Hsin-Min Wang
YTYu Tsao
Yu Tsao
Nonlinear spectral mapping-based models based on supervised learning have successfully applied for speech enhancement. However, as supervised learning approaches, a large amount of labelled data (noisy-clean speech pairs) should be provided to train those models. In addition, their performances for unseen noisy conditions are not guaranteed, which is a common weak point of supervised learning approaches. In this study, we proposed an unsupervised learning approach for speech enhancement, i.e., denoising autoencoder with linear regression decoder (DAELD) model for speech enhancement. The DAELD is trained with noisy speech as both input and target output in a self-supervised learning manner. In addition, with properly setting a shrinkage threshold for internal hidden representations, noise could be removed during the reconstruction from the hidden representations via the linear regression decoder. Speech enhancement experiments were carried out to test the proposed model. Results confirmed that the proposed DAELD could achieve comparable and sometimes even better enhancement performance as compared to the conventional supervised speech enhancement approaches, in both seen and unseen noise environments. Moreover, we observe that higher performances tend to achieve by DAELD when the training data cover more diverse noise types and signal-tonoise-ratio (SNR) levels.

15 September 2019
Verified
Conference Paper
Closed access
RZRyandhimas E. Zezario
Ryandhimas E. Zezario
SFSzu-Wei Fu
Szu-Wei Fu
XLXugang Lu
Xugang Lu
HWHsin-Min Wang
Hsin-Min Wang
YTYu Tsao
Yu Tsao

3 February 2025 in IEEE Transactions on Audio, Speech and Language Processing
Verified
Journal Article
Open access
DWDyah A. M. G. Wisnu
Dyah A. M. G. Wisnu
SRStefano Rini
Stefano Rini
RZRyandhimas E. Zezario
Ryandhimas E. Zezario
HWHsin-Min Wang
Hsin-Min Wang
YTYu Tsao
Yu Tsao
This paper introduces HAAQI-Net, a non-intrusive music audio quality assessment model for hearing aid users. Unlike traditional methods such as Hearing Aid Audio Quality Index (HAAQI), which requires intrusive reference signal comparisons, HAAQI-Net offers a more accessible and computationally efficient alternative. Leveraging a bidirectional long short-term memory architecture with attention mechanisms and features extracted from a pre-trained BEATs model, it can predict HAAQI scores directly from music audio clips and hearing loss patterns. The experimental results demonstrate that, compared to the traditional HAAQI as the reference, HAAQI-Net achieves a linear correlation coefficient (LCC) of 0.9368, a Spearman's rank correlation coefficient (SRCC) of 0.9486, and a mean squared error (MSE) of 0.0064, while significantly reducing the inference time from 62.52 seconds to 2.54 seconds. Furthermore, a knowledge distillation strategy was applied, reducing the parameters by 75.85% and inference time by 96.46%, while maintaining strong performance (LCC: 0.9071, SRCC: 0.9307, MSE: 0.0091). To expand its capabilities, HAAQI-Net was adapted to predict subjective human scores, mean opinion score (MOS), by fine-tuning. This adaptation significantly improved the prediction accuracy. Furthermore, the robustness of HAAQI-Net was evaluated under varying sound pressure level (SPL) conditions, revealing optimal performance at a reference SPL of 65 dB, with the accuracy gradually decreasing as SPL deviated from this point. The advancements in subjective score prediction, SPL robustness, and computational efficiency position HAAQI-Net as a reliable solution for music audio quality assessment, significantly contributing to the development of efficient and accurate models in audio signal processing and hearing aid technology.

1 November 2018
Verified
Conference Paper
Closed access
RZRyandhimas E. Zezario
Ryandhimas E. Zezario
JHJen-Wei Huang
Jen-Wei Huang
XLXugang Lu
Xugang Lu
YTYu Tsao
Yu Tsao
HHHsin-Te Hwang
Hsin-Te Hwang
HWHsin-Min Wang
Hsin-Min Wang