We introduce TF-CrossNet, a complex spectral mapping approach to speaker separation and enhancement in reverberant and noisy conditions. The proposed architecture comprises an encoder layer, a global multi-head self-attention module, a cross-band module, a narrow-band module, and an output layer. TF-CrossNet captures global, cross-band, and narrow-band correlations in the time-frequency domain. To address performance degradation in long utterances, we introduce a random chunk positional encoding. Experimental results on multiple datasets demonstrate the effectiveness and robustness of TF-CrossNet, achieving state-of-the-art performance in tasks including reverberant and noisy-reverberant speaker separation. Furthermore, TF-CrossNet exhibits faster and more stable training in comparison to recent baselines. Additionally, TF-CrossNet's high performance extends to multi-microphone conditions, demonstrating its versatility in various acoustic scenarios.
Adding visual cues to audio-based speech separation can improve separation performance. This paper introduces AV-CrossNet, an audiovisual (AV) system for speech enhancement, target speaker extraction, and multi-talker speaker separation. AV-CrossNet is extended from the TF-CrossNet architecture, which is a recently proposed network that performs complex spectral mapping for speech separation by leveraging global attention and positional encoding. To effectively utilize visual cues, the proposed system incorporates pre-extracted visual embeddings and employs a visual encoder comprising temporal convolutional layers. Audio and visual features are fused in an early fusion layer before feeding to AV-CrossNet blocks. We evaluate AV-CrossNet on multiple datasets, including LRS, VoxCeleb, TCD-TIMIT, and COG-MHEAR challenge, in terms of the performance metrics of PESQ, STOI, SNR and SDR. Evaluation results demonstrate that AV-CrossNet advances the state-of-the-art performance in all audiovisual tasks, even on untrained and mismatched datasets.
In this paper, a new numerical method is proposed for solving nonlinear boundary value problems (BVPs) based on Bézier curves. The well-known Falkner-Skan equation which is a third-order nonlinear BVP is considered to examine the efficiency of the presented method in terms of accuracy, convergence and stability. The presented hybrid technique takes advantage of the Method of Adjoint together with a novel predictor-collector multi-step integration procedure based on the Bernstein Polynomials and Bézier curves. In this method, first the governing equation is rewritten as a set of first-order differential equations, and the semi-infinite domain is mapped into a unit interval using appropriate coordinate transformation. The missing initial value is then determined by employing an appropriate iterative algorithm using the Method of Adjoint. Finally, the resulting initial value problem is solved using the new multi-step predictor-corrector scheme. Results of the presented method are compared with those appeared in the literature which show excellent agreement while having higher efficiency. Moreover, the accuracy and efficiency of the method are further examined by comparing predictions for the Falkner-Skan equation with five well-known multistep methods in the literature. Finally, from the stability point of view, the study also revealed that the new technique is more stable in comparison with other prominent methods.