Most human activity recognition (HAR) systems are based on computer vision or wearable sensors. However, these methods have limitations, such as privacy concerns, the need for high computational power, and the requirement for frequent battery recharging. In this paper, we present a distributed multisensor system for the recognition of human activities based on ambient audio and vibration data. The system comprises several ambient multisensor nodes (AMSNs) located within a smart home environment, along with a data transfer and analysis system. The data transfer and analysis system includes an IoT gateway and an MQTT broker. The data obtained by the AMSNs is classified using various machine learning algorithms. We utilized a ResNet model to analyze ambient acoustic signals and an encoder network for the classification of vibrational data. An empirical evaluation of the proposed approachwas conducted using a dataset recorded in a real-world environment and different combinations of modalities and sensor setups. The results indicate that audio and vibration-based data can be leveraged for accurate activity detection, eliminating the need for specialized sensor equipment. These findings have several implications for smart home environments, such as improving user experience and comfort or enhancing safety and security for inhabitants.

The detection of human activities is an important step in automated systems to understand the context of given situations. It can be useful for applications like healthcare monitoring, smart homes, and energy management systems for buildings. To achieve this, a sufficient data basis is required. The presented dataset contains labeled recordings of 25 different activities of daily living performed individually by 14 participants. The data were captured by five multisensors in supervised sessions in which a participant repeated each activity several times. Flawed recordings were removed, and the different data types were synchronized to provide multi-modal data for each activity instance. Apart from this, the data are presented in raw form, and no further filtering was performed. The dataset comprises ambient audio and vibration, as well as infrared array data, light color and environmental measurements. Overall, 8615 activity instances are included, each captured by the five multisensor devices. These multi-modal and multi-channel data allow various machine learning approaches to the recognition of human activities, for example, federated learning and sensor fusion.