The Internet of Audio Things (IoAuT) is an emerging research field positioned at the intersection of the Internet of Things, sound and music computing, artificial intelligence, and human-computer interaction. The IoAuT refers to the networks of computing devices embedded in physical objects (Audio Things) dedicated to the production, reception, analysis and understanding of audio in distributed environments. Audio Things, such as nodes of wireless acoustic sensor networks, are connected by an infrastructure that enables multidirectional communication, both locally and remotely. In this paper, we first review the state of the art of this field, then we present a vision for the IoAuT and its motivations. In the proposed vision, the IoAuT enables the connection of digital and physical domains by means of appropriate information and communication technologies, fostering novel applications and services based on auditory information. The ecosystems associated with the IoAuT include interoperable devices and services that connect humans and machines to support human-human and human-machines interactions. We discuss challenges and implications of this field, which lead to future research directions on the topics of privacy, security, design of Audio Things, and methods for the analysis and representation of audio-related information.

Noise pollution reduction in the environment is a major challenge from a societal and health point of view. To implement strategies to improve sound environments, experts need information on existing noise. The first source of information is based on the elaboration of noise maps using software, but with limitations on the realism of the maps obtained, due to numerous calculation assumptions. The second is based on the use of measured data, in particular through professional measurement observatories, but in limited numbers for practical and financial reasons. More recently, numerous technical developments, such as the miniaturization of electronic components, the accessibility of low-cost computing processors and the improved performance of electric batteries, have opened up new prospects for the deployment of low-cost sensor networks for the assessment of sound environments. Over the past fifteen years, the literature has presented numerous experiments in this field, ranging from proof of concept to operational implementation. The purpose of this article is firstly to review the literature, and secondly, to identify the expected technical characteristics of the sensors to address the problem of noise pollution assessment. Lastly, the article will also put forward the challenges that are needed to respond to a massive deployment of low-cost noise sensors.

This paper introduces a model for simulating environmental acoustic scenes that abstracts temporal structures from audio recordings. This model allows us to explicitly control key morphological aspects of the acoustic scene and to isolate their impact on the performance of the system under evaluation. Thus, more information can be gained on the behavior of an evaluated system, providing guidance for further improvements. To demonstrate its potential, this model is employed to evaluate the performance of nine state of the art sound event detection systems submitted to the IEEE DCASE 2013 Challenge. Results indicate that the proposed scheme is able to successfully build datasets useful for evaluating important aspects of the performance of sound event detection systems, such as their robustness to new recording conditions and to varying levels of background audio.

As part of the Agence Nationale de Recherche Caractérisation des ENvironnements SonorEs urbains (Characterization of urban sound environments) project, a questionnaire was sent in January 2019 to households in a 1 km2 study area in the city of Lorient, France, to which about 318 responded. The main objective of this questionnaire was to collect information about the inhabitants' perception of the sound environments in their neighborhoods, streets, and dwellings. In the same study area, starting mid-2019, about 70 sensors were continuously positioned, and 15 of them were selected for testing sound source recognition models. The French lockdown due to the COVID-19 crisis occurred during the project, and the opportunity was taken to send a second questionnaire during April 2020. About 31 of the first 318 first survey respondents answered this second questionnaire. This unique longitudinal dataset, both physical and perceptual, allows the undertaking of an analysis from different perspectives of such a period. The analysis reveals the importance of integrating source recognition tools, soundscape observation protocol, in addition to physical level analysis, to accurately describe the changes in the sound environment.

Instrumentalplaying techniques such as vibratos, glissandos, and trills often denote musical expressivity, both in classical and folk contexts. However, most existing approaches to music similarity retrieval fail to describe timbre beyond the so-called “ordinary” technique, use instrument identity as a proxy for timbre quality, and do not allow for customization to the perceptual idiosyncrasies of a new subject. In this article, we ask 31 human participants to organize 78 isolated notes into a set of timbre clusters. Analyzing their responses suggests that timbre perception operates within a more flexible taxonomy than those provided by instruments or playing techniques alone. In addition, we propose a machine listening model to recover the cluster graph of auditory similarities across instruments, mutes, and techniques. Our model relies on joint time–frequency scattering features to extract spectrotemporal modulations as acoustic features. Furthermore, it minimizes triplet loss in the cluster graph by means of the large-margin nearest neighbor (LMNN) metric learning algorithm. Over a dataset of 9346 isolated notes, we report a state-of-the-art average precision at rank five (AP@5) of .%. An ablation study demonstrates that removing either the joint time–frequency scattering transform or the metric learning algorithm noticeably degrades performance.

The emerging field of computational acoustic monitoring aims at retrieving high-level information from acoustic scenes recorded by some network of sensors. These networks gather large amounts of data requiring analysis. To decide which parts to inspect further, we need tools that automatically mine the data, identifying recurring patterns and isolated events. This requires a similarity measure for acoustic scenes that does not impose strong assumptions on the data. The state of the art in audio similarity measurement is the “bag-of-frames” approach, which models a recording using summary statistics of short-term audio descriptors, such as mel-frequency cepstral coefficients (MFCCs). They successfully characterise static scenes with little variability in auditory content, but cannot accurately capture scenes with a few salient events superimposed over static background. To overcome this issue, we propose a two-scale representation which describes a recording using clusters of scattering coefficients. The scattering coefficients capture short-scale structure, while the cluster model captures longer time scales, allowing for more accurate characterization of sparse events. Evaluation within the acoustic scene similarity framework demonstrates the interest of the proposed approach.