Recent studies in social robotics show that it can provide economic efficiency and growth in domains such as retail, entertainment, and active and assisted living (AAL). Recent work also highlights that users have the expectation of affordable social robotics platforms, providing focused and specific assistance in a robust manner. In this paper, we present the AMIRO social robotics framework, designed in a modular and robust way for assistive care scenarios. The framework includes robotic services for navigation, person detection and recognition, multi-lingual natural language interaction and dialogue management, as well as activity recognition and general behavior composition. We present AMIRO platform independent implementation based on a Robot Operating System (ROS). We focus on quantitative evaluations of each functionality module, providing discussions on their performance in different settings and the possible improvements. We showcase the deployment of the AMIRO framework on a popular social robotics platform—the Pepper robot—and present the experience of developing a complex user interaction scenario, employing all available functionality modules within AMIRO.
Social assistive robotics is at the forefront of the effort for ensuring independent living and inclusion, especially for the elderly and vulnerable members of the population. We consider that to be proactive is a core requirement of such robotic systems. One key aspect of being proactive is to understand what the user is doing. We explored the task of human action recognition in the context of a social robotic platform based on a framework that we proposed in earlier work. We implemented a processing pipeline specific to this task then collected a dataset using a Pepper robot. The proposed domain adaptation method to increase performance of the method showed promising results.
Multimodal interfaces seamlessly integrate two or more user inputs in a coordinated manner to enhance user interaction with the system. This work proposes a multimodal interface that makes use of hand tracking and speech interactions for Ambient Intelligent and Ambient Assisted Living environments. The system is composed of two main modules: Spoken Language Interaction module and Hand Tracking Interaction module. Spoken Language Interaction use speech recognition services to input the user utterance to a natural language understanding module from which the dialog manager will match the context and output the corresponding answer to the user. Hand Tracking Interactions is performed using a Microsoft Kinect device to detect the 3D coordinate of the hand which is stabilized and transformed to be used as a cursor on the screen. Tests have been run on the speech recognition and natural language understanding framework and reached an accuracy of 83.7% in offering the user a correct answer from the first try. The accuracy results of the Dialog manager in offering the user a correct output was 91.6% computed on a data set of 3900 sentences.
The field of socially assistive robotics is advancing steadily, finding an increasing number of applications in domains such as health and home care, public servicing, tourism or entertainment. The diversity of robotic units poses a challenge in terms of their programming, since application designers must either use manufacturer code, or build their own suite of behavior modules from ground up, using open-source platforms such as the Robot Operating System (ROS). The trade-off for design flexibility of the latter option is that there is a lack of ROS-based frameworks for building a behavioral life-cycle for assistive robots, or which allow them to be contextually connected to their environments. In this paper, we introduce the foundational work for AMIRO, a ROS-based framework for the development of assistive robotics applications. We present details of the core behavioral models for navigation, person recognition and communication and detail the means for organizing them into coherent complex behaviors. We further discuss the provisioning for context-aware decision making. The core behavior models are analyzed individually from a performance perspective and two very different evaluation scenarios are described showcasing the versatility of the proposed framework.
Internet of Things has rapidly developed over the last years, with sensors collecting extremely large amounts of data. Even though this process is becoming more and more efficient, there is an urgent need for a solution that processes, stores and extracts information in an adequate and efficient manner. Using some of the newest software technologies, this paper presents a data management and visualization solution for the environment data, collected by sensors installed on a robotic machine. In this paper, we present a new architecture for collecting, storing and managing sensor data. Our work-flow starts from the operating system installed on the robot, which is provided with a proper communication channel that allows messages to be exported in a non-relational database compatible format. The collected data is further aggregated, transformed and exposed using the back-end layer of a web application. Finally, the data is displayed using a front-end server, which provides interactive ways for data visualization.