Deep learning approaches for multi-source sound localization face significant challenges, particularly the need for extensive training datasets encompassing diverse spatial configurations to achieve robust generalization. This requirement leads to substantial computational demands, which are further exacerbated when localizing overlapping sources in complex acoustic environments with reverberation and noise. In this paper, a new methodology is proposed for simultaneous localization of two overlapping sound sources in the time–frequency domain in a closed, reverberant environment with a spatial resolution of using a small-sized microphone array. The proposed methodology is based on the integration of the sound source separation method with a single-source sound localization model. A hybrid model was proposed to separate the sound source signals received by each microphone in the array. The model was built using a bidirectional long short-term memory (BLSTM) network and trained on a dataset using the ideal binary mask (IBM) as the training target. The modeling results show that the proposed localization methodology is efficient in determining the directions for two overlapping sources simultaneously, with an average localization accuracy of 86.1% for the test dataset containing short-term signals of 500 ms duration with different signal-to-signal ratio values.
1 January 2024 in Vestnik Tomskogo Gosudarstvennogo Universiteta. Upravlenie, Vychislitel'naya Tekhnika i Informatika
This paper presents a deep learning model for sound source localization, which considers localization as a source direction classification problem. An approach is proposed based on the integration of sound intensity features and GCC-PHAT (Generalized cross-correlation - phase transform) features as input data for convolutional neural networks. Taking into account the modeling conditions, datasets were created for the purpose of training, validating and testing the model with spatial resolutions of 10º and 2 º. Simulation results demonstrated the effectiveness of the proposed model in localizing the source with high accuracy in a closed environment and in the presence of reverberation. The proposed model with a resolution of 10º outperformed the model fed with only sound intensity features as input features, achieving improvement in accuracy by 6,57% and in prediction accuracy by 2,86%, while the model with a resolution of 2º achieved an improvement in accuracy by 15,57% and in prediction accuracy by 2,04%.
This article proposes a methodology for constructing a software-hardware system for solving the problem of sound source localization in real acoustic environments based on the synthesis of several models, including intelligent ones. To achieve this goal, a methodology is proposed that includes the selection of technologies and equipment for collecting and processing acoustic signals, training models on experimental data, and conducting an additional series of experiments to evaluate the effectiveness of the models. Two models were considered: SI-GCC-CNN (Sound Intensity – Generalized Cross-Correlation Convolutional Neural Network), which combines sound intensity features and a generalized cross-correlation phase transform as input to convolutional neural networks, and SI-CNN (Sound Intensity – Convolutional Neural Network), which feeds sound intensity features into a convolutional neural network. To evaluate the effectiveness of the deep learning models used to solve this problem at a spatial resolution of 10º, a series of experiments were conducted in closed reverberant rooms. The generalization ability of these models was assessed by varying configuration settings. The experimental results demonstrated the effectiveness and generalization ability of the SI-GCC-CNN model when working in real-world acoustic environments. The SI-GCC-CNN model outperformed the SI-CNN model, achieving a 2.9x improvement in localization accuracy when changing the room size, 2.5x when changing the distance between the source and the center of the microphone array, and 2x when changing the location of the microphone array.