1 April 2025 in IEICE Transactions on Information and Systems
Skillfully fabricated artificial replicas of authentic media using advanced AI-based generators are known as “deepfakes.” Deepfakes have become a growing concern due to their increased distribution in cyber-physical spaces. In particular, deepfake speech, which is fabricated by using advanced AI-based speech analysis/synthesis techniques, can be abused for spoofing and tampering with authentic speech signals. This can enable attackers to commit serious offenses such as fraud by voice impersonation and unauthorized speaker verification. Our research project aims to construct the basis of auditory-media signal processing for defending against deepfake speech attacks. To this end, we introduce current challenges and state-of-the-art techniques for deepfake speech detection and examine current trends and remaining issues. We then introduce the basis of the acoustical features related to auditory perception and propose methods for detecting deepfake speech based on auditory-media signal processing consisting of these features and deep neural networks (DNNs).
16 May 2025 in IEEE Transactions on Audio, Speech and Language Processing
Early detection of factory machinery malfunctions is crucial in industrial applications. In machine anomalous sound detection (ASD), different machines exhibit unique vibration-frequency ranges based on their physical properties. Meanwhile, the human auditory system is adept at tracking both temporal and spectral dynamics of machine sounds. Consequently, integrating the computational auditory models of the human auditory system with machine-specific properties can be an effective approach to machine ASD. We first quantified the frequency importances of four types of machines using the Fisher ratio (F-ratio). The quantified frequency importances were then used to design machine-specific non-uniform filterbanks (NUFBs), which extract the log non-uniform spectrum (LNS) feature. The designed NUFBs have a narrower bandwidth and higher filter distribution density in frequency regions with relatively high F-ratios. Finally, spectral and temporal modulation representations derived from the LNS feature were proposed. These proposed LNS feature and modulation representations are input into an autoencoder neural-network-based detector for ASD. The quantification results from the training set of the Malfunctioning Industrial Machine Investigation and Inspection dataset with a signal-to-noise (SNR) of 6 dB reveal that the distinguishing information between normal and anomalous sounds of different machines is encoded non-uniformly in the frequency domain. By highlighting these important frequency regions using NUFBs, the LNS feature can significantly enhance performance using the metric of AUC (area under the receiver operating characteristic curve) under various SNR conditions. Furthermore, modulation representations can further improve performance. Specifically, temporal modulation is effective for fans, pumps, and sliders, while spectral modulation is particularly effective for valves.
29 May 2025 in IEEE Transactions on Audio, Speech and Language Processing
Room acoustical parameters (RAPs), room geometrical parameters (RGPs) and instantaneous occupancy levels are essential metrics for parameterizing the room acoustical characteristics (RACs) of a sound field around a listener's local environment, offering comprehensive indications for various applications. Current blind estimation methods either fail to cover a broad range of real-world acoustic environments in the context of real background noise or estimate only a few RAPs and RGPs from noisy single-channel speech signals. In addition, they are limited in their ability to estimate the instantaneous occupancy level. In this paper, we propose BERP, a new universal approach to blindly estimate RAPs, RGPs, and occupancy levels. It consists of two modules: one for RAPs and RGPs and another for occupancy levels. For the former task, we use a shared room feature encoder that combines attention mechanisms with convolutional layers to learn common features across room parameters, and multiple separate parametric predictors for continuous estimation of each parameter in parallel. The combination of attention and convolutions enables the model to capture acoustic features both locally and globally from speech, yielding more robust and multitask generalizable common features. Separate predictors allow the model to independently optimize for each room parameter to reduce task learning conflict and improve per-task performance. This architecture enables universal and efficient estimation of room parameters while maintaining satisfactory performance. For occupancy level estimation, we reuse the identical encoder with a classification head, exploiting the encoder's strong sequencefeature extraction. To evaluate the effectiveness of the proposed approach, we compile a task-specific dataset from several publicly available datasets, including synthetic and real reverberant recordings. The results reveal that BERP achieves state-of-the-art (SOTA) performance and excellent adaptability to real-world scenarios. The code, datasets, and weights are available on GitHub.
1 January 2026 in IEEE Access
1 November 2025 in Journal of Signal Processing
The rapid advancements in artificial intelligence (AI) technologies for singing voice synthesis have revolutionized music production but introduced significant challenges, including the misuse of AI-generated voices for deepfake purposes. This study proposes a framework for detecting deepfake singing voices using data augmentation techniques, the Synthetic Minority Over- sampling Technique (SMOTE), time stretching, and time shifting, combined with RawNet-based waveform feature extraction integrated into the Audio Anti-Spoofing Using Integrated Spectro-Temporal Graph Attention Networks (AASIST) model. The method effectively captures subtle temporal and spectral artifacts, achieving an equal error rate (EER) of 10.39% on the SVDD SingFake 2024 dataset, outperforming baseline models. This work provides a robust solution to safeguard the authenticity of AI-generated singing voices in digital media.