The Internet of Robotic Things is an emerging vision that brings together pervasive sensors and objects with robotic and autonomous systems. This survey examines how the merger of robotic and Internet of Things technologies will advance the abilities of both the current Internet of Things and the current robotic systems, thus enabling the creation of new, potentially disruptive services. We discuss some of the new technological challenges created by this merger and conclude that a truly holistic view is needed but currently lacking.

Worldwide demographic projections point to a progressively older population. This fact has fostered research on Ambient Assisted Living, which includes developments on smart homes and social robots. To endow such environments with truly autonomous behaviours, algorithms must extract semantically meaningful information from whichever sensor data is available. Human activity recognition is one of the most active fields of research within this context. Proposed approaches vary according to the input modality and the environments considered. Different from others, this paper addresses the problem of recognising heterogeneous activities of daily living centred in home environments considering simultaneously data from videos, wearable IMUs and ambient sensors. For this, two contributions are presented. The first is the creation of the Heriot-Watt University/University of Sao Paulo (HWU-USP) activities dataset, which was recorded at the Robotic Assisted Living Testbed at Heriot-Watt University. This dataset differs from other multimodal datasets due to the fact that it consists of daily living activities with either periodical patterns or long-term dependencies, which are captured in a very rich and heterogeneous sensing environment. In particular, this dataset combines data from a humanoid robot’s RGBD (RGB + depth) camera, with inertial sensors from wearable devices, and ambient sensors from a smart home. The second contribution is the proposal of a Deep Learning (DL) framework, which provides multimodal activity recognition based on videos, inertial sensors and ambient sensors from the smart home, on their own or fused to each other. The classification DL framework has also validated on our dataset and on the University of Texas at Dallas Multimodal Human Activities Dataset (UTD-MHAD), a widely used benchmark for activity recognition based on videos and inertial sensors, providing a comparative analysis between the results on the two datasets considered. Results demonstrate that the introduction of data from ambient sensors expressively improved the accuracy results.

Robotic ecologies are systems made out of several robotic devices, including mobile robots, wireless sensors and effectors embedded in everyday environments, where they cooperate to achieve complex tasks. This paper demonstrates how endowing robotic ecologies with information processing algorithms such as perception, learning, planning, and novelty detection can make these systems able to deliver modular, flexible, manageable and dependable Ambient Assisted Living (AAL) solutions. Specifically, we show how the integrated and self-organising cognitive solutions implemented within the EU project RUBICON (Robotic UBIquitous Cognitive Network) can reduce the need of costly pre-programming and maintenance of robotic ecologies. We illustrate how these solutions can be harnessed to (i) deliver a range of assistive services by coordinating the sensing & acting capabilities of heterogeneous devices, (ii) adapt and tune the overall behaviour of the ecology to the preferences and behaviour of its inhabitants, and also (iii) deal with novel events, due to the occurrence of new user׳s activities and changing user׳s habits