The rapid increase in the senior population is posing serious challenges to national healthcare systems. Hence, innovative tools are needed to early detect health issues, including cognitive decline. Several clinical studies show that it is possible to identify cognitive impairment based on the locomotion patterns of the elderly. In this work, we investigate the use of sensor data and deep learning to recognize those patterns in instrumented smart-homes. In order to get rid of the noise introduced by indoor constraints and activity execution, we introduce novel visual feature extraction methods for locomotion data. Our solution relies on locomotion trace segmentation, image-based extraction of salient features from locomotion segments, and vision-based deep learning. We carried out extensive experiments with a large dataset acquired in a smart-home test bed from 153 seniors, including people with cognitive diseases. Results show that our system can accurately recognize the cognitive status of the senior, reaching a macro- $$F_1$$F 1 score of 0.873 for the three categories that we target: cognitive health, mild cognitive impairment, and dementia. Moreover, an experimental comparison shows that our system outperforms state-of-the-art methods.

Locomotion characteristics and movement patterns are reliable indicators of neurodegenerative diseases (NDDs). This survey provides a systematic literature review of locomotion data mining systems for supporting NDD diagnosis. We discuss techniques for discovering low-level locomotion indicators, sensor data acquisition and processing methods, and NDD detection algorithms. The survey presents a comprehensive discussion on the main challenges for this active area, including the addressed diseases, locomotion data types, duration of monitoring, employed algorithms, and experimental validation strategies. We also identify prominent open challenges and research directions regarding ethics and privacy issues, technological and usability aspects, and availability of public benchmarks.

This survey article explores graph-based approaches to multimodal human activity recognition in indoor environments, emphasizing their relevance to advancing multimodal representation and reasoning. With the growing importance of integrating diverse data sources such as sensor events, contextual information, and spatial data, effective human activity recognition methods are essential for applications in smart homes, digital health, and more. We review various graph-based techniques, highlighting their strengths in encoding complex relationships and improving activity recognition performance. Furthermore, we discuss the computational efficiencies and generalization capabilities of these methods across different environments. By providing a comprehensive overview of the state-of-the-art in graph-based human activity recognition, this article aims to contribute to the development of more accurate, interpretable, and robust multimodal systems for understanding human activities in indoor settings.

The proliferation of sensors in smart homes makes it possible to monitor human activities, routines, and complex behaviors in an unprecedented way. Hence, human activity recognition has gained increasing attention over the last few years as a tool to improve healthcare and well-being in several applications. However, most existing activity recognition systems rely on cameras or wearable sensors, which may be obtrusive and may invade the user’s privacy, especially at home. Moreover, extracting expressive features from a stream of data provided by heterogeneous smart-home sensors is still an open challenge. In this paper, we investigate a novel method to detect activities of daily living by exploiting unobtrusive smart-home sensors (i.e., passive infrared position sensors and sensors attached to everyday objects) and vision-based deep learning algorithms, without the use of cameras or wearable sensors. Our method relies on depicting the locomotion traces of the user and visual clues about their interaction with objects on a floor plan map of the home, and utilizes pre-trained deep convolutional neural networks to extract features for recognizing ongoing activity. One additional advantage of our method is its seamless extendibility with additional features based on the available sensor data. Extensive experiments with a real-world dataset and a comparison with state-of-the-art approaches demonstrate the effectiveness of our method.

With the global population aging, the demand for technologies facilitating independent living, especially for those with cognitive impairments, is increasing. This paper addresses this need by conducting a comprehensive evaluation of the Rostock Kitchen Task Assessment dataset, a pivotal resource in kitchen task activity recognition. Our study begins with an in-depth introduction, emphasizing the increasing prevalence of neurodegenerative disorders and the crucial role of assistive technologies. Our contributions encompass a systematic literature review, design and implementation of a working prototype of our envisioned system, refinement of the Rostock Kitchen Task Assessment dataset, creation of a semantically annotated dataset, extraction of statistical features, comparative analysis, and rigorous model performance assessment. The core of our work is the thorough evaluation and benchmarking of different activity recognition approaches using the aforementioned Rostock Kitchen Task Assessment dataset. Our experimental results demonstrate that despite encountering an imbalance problem in the dataset, the fusion of the Hidden Markov Model and Random Forest leads to superior results, achieving a weighted-averaged $F_{1}$ -score of 74.10% for all available activities and 81.40% for the most common actions in the Rostock Kitchen Task Assessment dataset. Moreover, through systematic analysis, we identify strengths and suggest potential refinements, thereby advancing the field of kitchen activity recognition. This offers valuable insights for researchers and practitioners in assistive and remote care technologies.

The ubiquity of sensors in smart-homes facilitates the support of independent living for older adults and enables cognitive assessment. Notably, there has been a growing interest in utilizing movement traces for identifying signs of cognitive impairment in recent years. In this study, we introduce an innovative approach to identify abnormal indoor movement patterns that may signal cognitive decline. This is achieved through the non-intrusive integration of smart-home sensors, including passive infrared sensors and sensors embedded in everyday objects. The methodology involves visualizing user locomotion traces and discerning interactions with objects on a floor plan representation of the smart-home, and employing different image descriptor features designed for image analysis tasks and synthetic minority oversampling techniques to enhance the methodology. This approach distinguishes itself by its flexibility in effortlessly incorporating additional features through sensor data. A comprehensive analysis, conducted with a substantial dataset obtained from a real smart-home, involving 99 seniors, including those with cognitive diseases, reveals the effectiveness of the proposed functional prototype of the system architecture. The results validate the system’s efficacy in accurately discerning the cognitive status of seniors, achieving a macro-averaged F₁-score of 72.22% for the two targeted categories: cognitively healthy and people with dementia. Furthermore, through experimental comparison, our system demonstrates superior performance compared with state-of-the-art methods.