30 April 2018 in IEEE/ACM Transactions on Audio, Speech, and Language Processing
Speech enhancement has been widely investigated for several decades, but by modifying only the amplitude spectrum of a speech signal, ignoring the phase spectrum, which has been regarded as an unimportant feature. However, it was recently reported that the phase spectrum plays an important role in speech quality and intelligibility. In this paper, we propose a phase reconstruction method based on harmonic enhancement using the fundamental frequency and phase distortion feature. This feature is known to show fluctuations in the phase spectrum with respect to time and frequency. We estimate the speech phase spectrum by considering the relationship between harmonic phase spectra. Experimental evaluations indicate that the proposed phase reconstruction method improves speech quality in various noisy environments.
20 April 2023 in IEEE/ACM Transactions on Audio, Speech, and Language Processing
This paper presents improvements to two-stage algorithms for estimating the short-time Fourier transform (STFT) phase from only the amplitude by using deep neural networks (DNNs). The phase is difficult to reconstruct due to its sensitivity to the waveform shift and wrapping issue. To mitigate these problems, two-stage approaches indirectly estimate the phase through phase derivatives, i.e., instantaneous frequency (IF) and group delay (GD). In the first stage, the IF and GD are estimated from the amplitude using DNNs, and then in the second stage, the phase is reconstructed by maintaining the IF/GD information. Conventional methods for the second stage do not consider the importance of high-amplitude time–frequency bins, e.g., the least squares-based method, or lack a solid model, e.g., the average-based method. To address these problems, we propose improvements to the second stage of two-stage algorithms by using von Mises distribution-based maximum likelihood and weighted least squares. We also provide theoretical discussions for the phase reconstruction, including the investigations of the properties of the GD and roles of the IF/GD information in the inverse STFT. On the basis of the analysis, we propose a new phase-based feature, i.e., inter-frequency phase difference (IFPD), and demonstrate its application in two-stage phase reconstruction algorithms. We conducted subjective and objective experiments to compare the performances of our proposed and conventional methods. The results confirm that the proposed method using the IFPD performs better than other methods for all metrics.
1 June 2023 in IEEE/ACM Transactions on Audio, Speech, and Language Processing
In this paper, we present a sound field interpolation for array signal processing (ASP) that is robust to rotation of a circular microphone array (CMA), and we evaluate beamforming as one of its applications. Most ASP methods assume a time-invariant acoustic transfer system (ATS) from sources to the microphone array. This assumption makes it challenging to perform ASP in real situations where sources and the microphone array can move. Therefore, considering a time-variant ATS is an essential task for the use of ASP. In this study, we focus on one such movement, the rotation of the CMA. Our method interpolates the sound field on the circumference of a circle, where microphones are equally spaced, based on the sampling theorem on the circle. The interpolation enables us to estimate the signals at the microphone positions before the rotation. Hence, conventional ASP, which assumes a time-invariant ATS, is applicable after interpolation without modification. We developed two beamforming schemes, one for batch and one for online processing, that combine the minimum power distortionless response beamformer and sound field interpolation. We evaluated the dependences of the interpolation on frequency and rotation angle using the signal-to-error ratio. Additionally, simulation results demonstrated that the two proposed schemes improve the beamformer's performance when the CMA rotates.
24 October 2023 in IEEE Access
In this paper, we present a novel approach to estimating multiple time delays (TDs) in sensor arrays that is robust to outliers of TD measurements. These measurements are typically obtained from the peak of the cross correlation of two sensor signals but may contain outliers due to noise, significantly degrading the performance of downstream applications. To address this issue, we propose an approach to leverage only the best minimum-necessary TD measurements. First, we describe the general properties of TDs and show that the degree of freedom of TDs is the number of sensors minus one for a signal source, which indicates that the full set of TDs is redundant in this case. We then consider selecting nonredundant TDs from all measurements given the necessary and sufficient condition to reconstruct all TDs uniquely. We represent this condition by utilizing the graph theory, and then, formulate an optimization problem to select the optimal nonredundant TDs while satisfying the condition above. We reduce this problem to the problem of finding a minimum spanning tree and propose an efficient algorithm for TD estimation. Experimental evaluation shows that our method successfully eliminates outliers while ensuring that all TDs can be restored.
6 June 2024 in IEEE/ACM Transactions on Audio, Speech, and Language Processing
In this paper, we present an enhanced method designed to facilitate sound field interpolation (SFI) for rotationrobust beamforming using unequally spaced circular microphone arrays (unes-CMAs). Unlike the previous approach that necessitated an equally spaced circular microphone array (es- CMA), our method addresses the challenge of handling nonuniformly spaced microphones, making it suitable for real-world applications where unes-CMAs are more prevalent. Our proposed method enables the estimation of a virtual signal of an unes- CMA before rotation, derived from the observed signal after rotation. A modified SFI technique is utilized to compensate for the positional errors of microphones on an unes-CMA and to estimate a virtual signal at equally spaced positions after rotation. As an intermediate step, the previous SFI method is utilized to obtain equally spaced signals before rotation. Subsequently, the target signal of the unes-CMA before rotation is reconstructed, effectively achieving rotation-robust beamforming on the unes-CMA. Moreover, we provide an in-depth analysis of our proposed method's properties. We conducted simulated experiments, including online beamforming applications, to evaluate its performance. The experimental results demonstrated that our method effectively mitigates the adverse effects of unequal microphone placement, yielding significant improvements in estimating the signal before rotation under various conditions. Moreover, our proposed method consistently outperformed the previous approach, significantly enhancing the performance of beamforming.