Human activity recognition (HAR) is fundamental to many services in smart buildings. However, providing sufficiently robust activity recognition systems that could be confidently deployed in an ordinary real environment remains a major challenge. Much of the research done in this area has mainly focused on recognition through pre-segmented sensor data. In this paper, real-time human activity recognition based on streaming sensors is investigated. The proposed methodology incorporates dynamic event windowing based on spatio-temporal correlation and the knowledge of activity trigger sensor to recognize activities and record new events. The objective is to determine whether the last event that just happened belongs to the current activity, or if it is the sign of the start of a new activity. For this, we consider the correlation between sensors in view of what can be seen in the history of past events. The proposed algorithm contains three steps: verification of sensor correlation (SC), verification of temporal correlation (TC), and determination of the activity triggering the sensor. The proposed approach is applied to a real case study: the “Aruba” dataset from the CASAS database. F1 score is used to assess the quality of the segmentation. The results show that the proposed approach segments several activities (sleeping, bed to toilet, meal preparation, eating, housekeeping, working, entering home, and leaving home) with an F1 score of 0.63–0.99.

Recently, deep learning (DL) approaches have been extensively employed to recognize human activities in smart buildings, which greatly broaden the scope of applications in this field. Convolutional neural networks (CNN), well known for feature extraction and activity classification, have been applied for estimating human activities. However, most CNN-based techniques usually focus on divided sequences associated to activities, since many real-world employments require information about human activities in real time. In this work, an online human activity recognition (HAR) framework on streaming sensor is proposed. The methodology incorporates real-time dynamic segmentation, stigmergy-based encoding, and classification with a CNN2D. Dynamic segmentation decides if two succeeding events belong to the same activity segment or not. Then, because a CNN2D requires a multi-dimensional format in input, stigmergic track encoding is adopted to build encoded features in a multi-dimensional format. It adopts the directed weighted network (DWN) that takes into account the human spatio-temporal tracks with a requirement of overlapping activities. It represents a matrix that describes an activity segment. Once the DWN for each activity segment is determined, a CNN2D with a DWN in input is adopted to classify activities. The proposed approach is applied to a real case study: the “Aruba” dataset from the CASAS database.

As living standards improve, the growing demand for energy, comfort, and health monitoring drives the increased importance of innovative solutions. Real-time recognition of human activities (HAR) in smart homes is of significant relevance, offering varied applications to improve the quality of life of fragile individuals. These applications include facilitating autonomy at home for vulnerable people, early detection of deviations or disruptions in lifestyle habits, and immediate alerting in the event of critical situations. The first objective of this work is to develop a real-time HAR algorithm in embedded equipment. The proposed approach incorporates the event dynamic windowing based on space-temporal correlation and the knowledge of activity trigger sensors to recognize activities in the case of a record of new events. The second objective is to approach the HAR task from the perspective of edge computing. In concrete terms, this involves implementing a HAR algorithm in a “home box”, a low-power, low-cost computer, while guaranteeing performance in terms of accuracy and processing time. To achieve this goal, a HAR algorithm was first developed to perform these recognition tasks in real-time. Then, the proposed algorithm is ported on three hardware architectures to be compared: (i) a NUCLEO-H753ZI microcontroller from ST-Microelectronics using two programming languages, C language and MicroPython; (ii) an ESP32 microcontroller, often used for smart-home devices; and (iii) a Raspberry-PI, optimizing it to maintain accuracy of classification of activities with a requirement of processing time, memory resources, and energy consumption. The experimental results show that the proposed algorithm can be effectively implemented on a constrained resource hardware architecture. This could allow the design of an embedded system for real-time human activity recognition.

Fault diagnosis and maintenance of a whole-building system is a complex task to perform. A wide range of available building fault detection and diagnosis (FDD) tools are only capable of performing fault detection using behavioral constraints analysis. However, the validity of the detected symptom is always questionable. In this work, we introduce the concept of the contextual test with validity constraints, in the context of building fault diagnostics. Thanks to a common formalization of the proposed heterogeneous tests, rule-, range-, and model-based tests can be combined in the same diagnostic analysis that reduces the whole-building modeling effort. The proposed methodology comprises the minimum diagnostic explanation feature that can significantly improve the knowledge of the building facility manager. A bridge diagnosis approach is used to describe the multiple fault scenarios. The proposed methodology is validated on an experimental building called the center for studies and design of prototypes (CECP) building located in Angers, France.

Building Energy Management (BEM) and monitoring systems should not only consider HVAC systems and building physics but also human behaviors. These systems could provide information and advice to occupants about the significance of their practices with regard to the current state of a dwelling. It is also possible to provide services such as assistance to the elderly, comfort and health monitoring. For this, an intelligent building must know the daily activities of its residents and the algorithms of the smart environment must track and recognize the activities that the occupants normally perform as part of their daily routine. In the literature, deep learning is one of effective supervised learning model and cost-efficient for real-time HAR, but it still struggles with the quality of training data (missing values in time series and non-annotated event), the variability of data, the data segmentation and the ontology of activities. In this work, recent research works, existing algorithms and related challenges in this field are firstly highlighted. Then, new research directions and solutions (performing fault detection and diagnosis for drift detection, multi-label classification modeling for multi-occupant classification, new indicators for training data quality, new metrics weighted by the number of representations in dataset to handle the issue of missing data and finally language processing for complex activity recognition) are suggested to solve them respectively and to improve this field.