With the development of deep neural networks, speech enhancement technology has been vastly improved. However, commonly used speech enhancement approaches cannot fully leverage contextual information from different scales, which limits performance improvement. To address this problem, we propose a nested U-Net with self-attention and dense connectivity (SADNUNet) for monaural speech enhancement in the time domain. SADNUNet is an encoder-decoder structure with skip connections. In SADNUNet, the multi-scale aggregation block is proposed to explore more contextual information from different scales. By this means, the advantage of global and local speech features can be fully utilized to improve speech reconstruction ability. Furthermore, dense connectivity and self-attention are incorporated in the network for better feature extraction and utterance level context aggregation. The experimental results demonstrate that the proposed approach achieves on-par or better performance than other models in objective speech intelligibility and quality scores.

One of the leading speech enhancement technologies is the deep neural network-based approach, which dominates the recent development in single-channel speech enhancement. In this paper, we propose a convolutional network with multi-scale and attention mechanisms for end-to-end single-channel speech enhancement (MASENet). More specifically, the MASENet network consists of five modules, namely multi-scale speech encoder, frequency-dilated module, temporal convolutional attention module, post-processing module, and single-scale speech decoder. The frequency-dilated module and temporal convolutional attention module are leveraged to extract local and global information. The dense connections are used to avoid the vanishing gradient problem. Furthermore, we design the attention block to improve the discriminative learning ability of the network. The experimental results show that the proposed network achieves significantly better enhancement performance than other baselines in terms of objective speech intelligibility and quality metrics.

In order to achieve remote voice acquisition, a 9.75-GHz microwave Doppler radar with a high-sensitivity coherent homodyne demodulator is used to detect voice signals. However, the combined noise sources, including background noise and shock noise, are included in the Doppler audio radar-captured voice, which seriously damages the quality of speech. In the current work, a multi-stage enhancement method is used to enhance Doppler audio radar-captured voice. The first stage is dictionary learning, which uses the K-singular value decomposition algorithm to train speech and noise dictionaries. The noise training data come from the Doppler audio radar-captured shock noise signals under the influence of different relative motion modulations. The second stage generates a position mask of the shock noise. The third stage is the removal of background noise and shock noise. The removal of background noise adopts our proposed constrained low-rank sparse matrix decomposition algorithm based on power spectral density. The final speech spectrum with shock noise removed is obtained through sparse restoration using the LARC algorithm. The experimental results show that the 9.75-GHz Doppler audio radar can effectively acquire human voice signals, and the proposed multi-stage enhancement method can effectively remove the background noise and shock noise in the radar speech, thereby improving the quality of the audio radar voice. The obtained results imply the potential to implement practical Doppler audio radar systems for human voice signal acquisition.

The neural network has become a new and powerful paradigm in speech enhancement, triggering the surge of research. However, most existing methods directly predict speech and ignore the noise information. In this paper, we propose a two-stage learning and fusion network with noise awareness for time-domain monaural speech enhancement, which can be regarded as a progressive learning process. More specifically, in the first network, speech and noise are estimated simultaneously. The estimated speech and the signal by subtracting estimated noise from the noisy speech are stacked with noisy speech as input to obtain further refined speech in the second network. Both networks are mainly built on encoders and decoders with skip connections. To better control the information flow in the network, we introduce the gated linear unit in the encoder and the decoder, which also can help model complex interactions. Dilated dense blocks are added after each layer of the encoder and decoder to improve the model efficiency and enlarge the receptive field. Our experiments confirm that the proposed two-stage learning network with noise awareness achieves better performance than several advanced systems under various conditions.

With the development of deep neural networks, multi-channel speech separation techniques with fixed array geometries have achieved remarkable performance. However, distributed microphone array processing remains a challenging problem because it requires the network to be able to process inputs with varying dimensions. To address this problem, we propose a triple-path recurrent neural network (TPRNN) with multi-scale aggregation blocks for distributed microphone array multi-channel speech separation. First, we extend the single-channel dual-path recurrent neural network by additionally adding multi-scale aggregation blocks and adaptive feature fusion blocks. Next, a third path along the spatial dimension is introduced to model spatial information. By this means, TPRNN can alternately and iteratively perform inter-channel, intra-chunk, and inter-chunk modeling. The experimental results show that the proposed approach outperforms other advanced baselines for multi-channel speech separation and enhancement tasks using spatially distributed microphones.

Thanks to the use of deep neural networks (DNNs), microphone array speech separation methods have achieved impressive performance. However, most existing neural beamforming methods explicitly follow traditional beamformer formulas, which possibly causes sub-optimal performance. In this study, a pre-separation and all-neural beamformer framework is proposed for multi-channel speech separation without following the solutions of the conventional beamformers, such as the minimum variance distortionless response (MVDR) beamformer. More specifically, the proposed framework includes two modules, namely the pre-separation module and the all-neural beamforming module. The pre-separation module is used to obtain pre-separated speech and interference, which are further utilized by the all-neural beamforming module to obtain frame-level beamforming weights without computing the spatial covariance matrices. The evaluation results of the multi-channel speech separation tasks, including speech enhancement subtasks and speaker separation subtasks, demonstrate that the proposed method is more effective than several advanced baselines. Furthermore, this method can be used for symmetrical stereo speech.