9 June 2023 in IEEE/ACM Transactions on Audio, Speech, and Language Processing
The auditory brain circuits are biologically constructed to recognize and localize sounds by encoding a combination of cues that help individuals interpret sounds. The development of computational methods inspired by human capacities has established opportunities for improving machine hearing. Recent studies based on deep learning show that using convolutional recurrent neural networks (CRNNs) is a promising approach for sound event detection and localization in spatial sound. Nevertheless, depending on the sound environment, the performance of these systems is still far from reaching perfect metrics. Therefore, this work intends to boost the performance of state-of-the-art (SOTA) systems by using bio-inspired gammatone auditory filters and intensity vectors (IVs) for the acoustic feature extraction stage, along with the implementation of a temporal convolutional network (TCN) block into a CRNN model, to capture long term dependencies. Three data augmentation techniques are applied to increase the small number of samples in spatial audio datasets. The mentioned stages constitute our proposed Gammatone-based Sound Events Localization and Detection (G-SELD) system, which exceeded the SOTA results on four spatial audio datasets with different levels of acoustical complexity and with up to three sound sources overlapping in time.
2 December 2024 in IEEE Access
Sound Event Localization and Detection (SELD) is a critical challenge in various industrial applications, such as autonomous systems, smart cities, and audio surveillance, which require accurate identification and localization of sound events in complex environments. Traditional supervised approaches heavily rely on large, annotated multichannel audio datasets, which are expensive and time-consuming to produce. This paper addresses this limitation by introducing the w2v-SELD architecture, a self-supervised model adapted from the wav2vec 2.0 framework to learn effective sound event representations directly from raw, unlabeled 3D audio data. The proposed model follows a two-stage process: pre-training on large, unlabeled 3D audio datasets to capture high-level features, followed by fine-tuning with a smaller, labeled SELD dataset. Experimental results show that our w2v-SELD method outperforms baseline models on Detection and Classification of Acoustic Scenes and Events (DCASE) challenges, achieving a 66% improvement for DCASE TAU-2019 and a 57% improvement on DCASE TAU-2020 with respect to baseline systems. The w2v-SELD model performs competitively with state-of-the-art supervised methods, highlighting its potential to significantly reduce the dependency on labeled data in industrial SELD applications. The code and pre-trained parameters of our w2v-SELD model are available in this repository.
22 January 2025 in The Cleft Palate Craniofacial Journal
Objective: Post-surgical lip symmetry assessment is a key indicator of cleft repair success. Traditional methods rely on distances between anatomical landmarks, which are impractical for video analysis and overlook texture and appearance. We propose an artificial intelligence (AI) approach to automate this process, analyzing lateral lip morphology for a quantitative symmetry evaluation. Design: We utilize contrastive learning to quantify lip symmetry by measuring the similarity between the representations of the sides, which is subsequently used to classify the severity of asymmetry. Our model does not require patient images for training. Instead, we introduce dissimilarities in face images from open datasets using two methods: temporal misalignment for video frames and face transformations to simulate lip asymmetry observed in the target population. The model differentiates the left and right image representations to assess asymmetry. We evaluated our model on 146 images of patients with repaired cleft lip. Results: The deep learning model trained with face transformations categorized patient images into five asymmetry levels, achieving a weighted accuracy of 75% and a Pearson correlation of 0.31 with medical expert human evaluations. The model utilizing temporal misalignment achieved a weighted accuracy of 69% and a Pearson correlation of 0.27 for the same classification task. Conclusions: We propose an automated approach for assessing lip asymmetry in patients with repaired cleft lip by transforming facial images of control subjects to train a deep learning model, eliminating manual anatomical landmarks. Our promising results provide a more efficient and objective tool for evaluating surgical outcomes.
12 December 2023 in IEEE Access
Continuous monitoring of active volcanoes is essential for understanding their behavior and providing accurate forecasts and warnings. Automating the detection of volcano microseismic events plays a crucial role in large-scale analysis, early warning systems, and efficient handling of large-scale data. This paper presents a novel approach for the automated detection of microseismic events associated with volcanic activity, focusing on the Cotopaxi volcano in Ecuador. Inspired by voice activity detection (VAD) systems used in speech processing, we propose an Adaptive-Microseismic Activity Detector (A-MAD) that modifies VAD techniques to identify frames within seismic signals containing volcanic activity. The A-MAD system incorporates a spectral subtraction stage to mitigate the impact of environmental noise and employs Gaussian Mixture Models (GMMs) to model the probabilistic distribution of microseismic events. Mel Frequency Cepstral Coefficients (MFCCs) are used as features to describe the seismic signals, enabling adaptability for varying noise levels. Experimental results on signals from Cotopaxi Volcano in Ecuador demonstrate the effectiveness of the A-MAD system, achieving high accuracy (96.39% for discrete events and 98.45% for continuous signals) while meeting the efficiency requirements of volcano monitoring institutions. This work contributes to the advancement of early warning systems for volcanic eruptions, providing a robust and adaptive approach for microseismic activity detection.
30 December 2022 in Revista Eletrônica de Iniciação Científica em Computação
When songs are composed or performed, there is often an intent by the singer/songwriter of expressing feelings or emotions through it. For humans, matching the emotiveness in a musical composition or performance with the subjective perceptiveness of an audience can be quite challenging. Fortunately, the machine learning approach for this problem is simpler. Usually, it takes a data-set, from which audio features are extracted to present this information to a data-driven model, that will, in turn, train predicting the highest probability of an input song matching a target emotion. In this paper†, we studied the most common features and models used in recent publications to tackle this problem, revealing which ones are best suited for songs a cappella.