Convolutional networks have achieved the state-of-the-art performance on Acoustic Scene Classification (ASC). Given the Log Mel-Spectrogram of an audio sample, the network can extract useful semantic contents in a certain range receptive field by stacking local convolutional operations. However, the temporal relations between different receptive fields are not captured explicitly. In this letter, we propose an end-to-end 3D Convolutional Neural Network (CNN) for ASC, named SeNoT-Net, which can generate effective audio representations by capturing temporal relations from semantic neighbors of different receptive fields over time. The SeNoT-Net treats the Log-Mel spectrogram as an ordered segment-level sequence. For each segment, the residual block can produce the semantic feature maps, then the semantic neighbors over time (SeNoT) module is applied to capture the relations between each feature point in the feature maps and its top-k semantic neighbors. The proposed SeNoT-Net outperforms most of the state-of-the-art CNN models on both DCASE 2018 and 2019 ASC datasets.
15 January 2020 in IEEE/ACM Transactions on Audio, Speech, and Language Processing
Audio signals are temporally-structured data, and learning their discriminative representations containing temporal information is crucial for the audio classification. In this work, we propose an audio representation learning method with a hierarchical pyra-mid structure called pyramidal temporal pooling (PTP) which aims to capture the temporal information of an entire audio sample. By stacking a global temporal pooling layer on multiple local temporal pooling layers, the PTP can capture the high-level temporal dynamics of the input feature sequence in an unsuper-vised way. Furthermore, in the top global temporal pooling layer, we jointly optimize a learnable discriminative mapping (DM) and a softmax classifier. Such that, a joint learning method for the discriminative audio representations and the classifier called DM-PTP is also presented. By treating the temporal encoding as a low-level constraint of a bi-level optimization problem, the DM-PTP can produce the discriminative representation while maintaining the temporal information of the whole sequence. For an audio sample with an arbitrary time duration, both our PTP and DM-PTP can encode the input feature sequence with arbi-trary length into a fixed-length representation. Without using any data augmentation and ensemble learning methods, both PTP and DM-PTP outperform the state-of-the-art CNNs on the audio event recognition (AER) dataset, and can achieve comparable performance on the DCASE 2018 acoustic scene classification (ASC) dataset compared with other best models in the challenge.
Liwen Zhang, Ge Wang, Wenjuan Han, Kewei Tu. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.
Human activity recognition based on wireless signals emerges as a research hotspot recently. Though tremendous efforts have been devoted and significant progresses have been achieved, one fundamental issue still remains open, i.e., theoretical modeling between signal dynamics and human activities. This paper fills in the blank by addressing several theoretical issues and providing insightful mathematical analysis including a signal-activity model. To validate such analysis, a prototype system has been built, where a series of real-world experiments has been conducted. Empirical results have justified our theoretical findings. Moreover, important hands-on experiences on the system implementation and parameter settings have been offered.
29 November 2025 in Expert Systems with Applications
This paper presents exploratory work on the use of Testing with Concept Activation Vectors (TCAV) within a concept-based explanation framework to provide the explainability of millimeter-wave (MMW) radar target recognition. Given that the radar spectrum is difficult for non-domain experts to understand visually, defining concepts for radar remains a significant challenge. In response, drawing from the visual analytical experience of experts, some basic concepts based on brightness, striping, size, and shape are adopted in this paper. However, the simplicity of basic concept definitions sometimes leads to vague correlations with recognition targets and significant variability among individuals, limiting their adaptability to specific tasks. To address these issues, this study proposes a Basic Concept-Guided Deep Embedding Clustering (BCG-DEC) method that can effectively discover task-specific composite concepts. BCG-DEC methodically analyzes the deep semantic information of radar data through four distinct stages from the perspective of concept discovery, ensuring that the concepts discovered accurately conform to the task-specific property of MMW radar target recognition. The experimental results show that the proposed method not only expands the number of concepts but also effectively solves the problem of difficulty in annotating basic concepts. In the ROD2021 MMW radar explainability experiments, the concepts proved crucial for recognizing specific categories of radar targets.