Driver distraction is a leading factor in car crashes. With a goal to reduce traffic accidents and improve transportation safety, this study proposes a driver distraction detection system which identifies various types of distractions through a camera observing the driver. An assisted driving testbed is developed for the purpose of creating realistic driving experiences and validating the distraction detection algorithms. The authors collected a dataset which consists of images of the drivers in both normal and distracted driving postures. Four deep convolutional neural networks including VGG‐16, AlexNet, GoogleNet, and residual network are implemented and evaluated on an embedded graphic processing unit platform. In addition, they developed a conversational warning system that alerts the driver in real‐time when he/she does not focus on the driving task. Experimental results show that the proposed approach outperforms the baseline one which has only 256 neurons in the fully‐connected layers. Furthermore, the results indicate that the GoogleNet is the best model out of the four for distraction detection in the driving simulator testbed.

Social robots are coming to our homes and have already been used to help humans in a number of ways in geriatric care. This article aims to develop a framework that enables social robots to conduct regular clinical screening interviews in geriatric care, such as cognitive evaluation, falls’ risk evaluation, and pain rating. We develop a social robot with essential features to enable clinical screening interviews, including a conversational interface, face tracking, an interaction handler, attention management, robot skills, and cloud service management. Besides, a general clinical screening interview management (GCSIM) model is proposed and implemented. The GCSIM enables social robots to handle various types of clinical questions and answers, evaluate and score responses, engage interviewees during conversations, and generate reports on their well-being. These reports can be used to evaluate the progression of cognitive impairment, risk of falls, pain level, and so on by caregivers or physicians. Such a clinical screening capability allows for early detection and treatment planning in geriatric care. The framework was developed and implemented on our 3-D-printed social robot. It was tested on 30 older adults with different ages, achieved satisfying results, and received their high confidence and trust in the use of this robot for human well-being assessment. Note to Practitioners —This article is motivated by the goal of using a social robot to perform geriatric well-being assessment through clinical screening interviews. In order to conduct clinical screening interviews, the social robot needs the following essential features: having a verbal conversational interface, adapting to different types of clinical screening interviews, scoring and evaluating answers, having nondirective listening responses, and enabling directive listening responses. The proposed general clinical screening interview management (GCSIM) model demonstrates these capabilities on the social robot. The robot can give structured clinical screening interviews with different question–answer sheets. This will help advance assistive technologies for use by geriatric physicians, nurses, and social service professionals to keep older adults healthy, safe, and independent at home. Robots will become more and more essential in working alongside geriatric practitioners to help monitor older adults at home and to provide early detection and warning of cognitive/mental health problems, falls’ risk, and so on. This early detection property can improve quality-of-care and help older adults remain living at home.

Monitoring daily activities is essential for home service robots to take care of the older adults who live alone in their homes. In this article, we proposed a sound-based human activity monitoring (SoHAM) framework by recognizing sound events in a home environment. First, the method of context-aware sound event recognition (CoSER) is developed, which uses contextual information to disambiguate sound events. The locational context of sound events is estimated by fusing the data from the distributed passive infrared (PIR) sensors deployed in the home. A two-level dynamic Bayesian network (DBN) is used to model the intratemporal and intertemporal constraints between the context and the sound events. Second, dynamic sliding time window-based human action recognition (DTW-HaR) is developed to estimate active sound event segments with their labels and durations, then infer actions and their durations. Finally, a conditional random field (CRF) model is proposed to predict human activities based on the recognized action, location, and time. We conducted experiments in our robot-integrated smart home (RiSH) testbed to evaluate the proposed framework. The obtained results show the effectiveness and accuracy of CoSER, action recognition, and human activity monitoring. Note to Practitioners —This article is motivated by the goal to develop companion robots that can assist older adults living alone. Among many capabilities, monitoring human daily activities is an essential one for such robots. Though computer vision or wearable sensors-based methods have been developed by other researchers, they are not practical due to the privacy concern and intrusiveness. Sound-based daily activity recognition can address these concerns and offer a viable solution. In this regard, our proposed method adopts microphones on the robot and a small set of motion sensors distributed in the home. The proposed theoretical framework was tested in a small-scale mock-up apartment with promising results. Before such companion robots can be deployed to real homes for elderly care, there is a need to improve the robustness of the algorithms. More thorough tests in various realistic home environments should be conducted to fully evaluate the performance of the robots. In addition, privacy concern related to audio capture should be further mitigated.

Memory loss is a part of normative aging. Many older adults commonly forget to take prescribed medication, which can have an adverse effect on health. Therefore, it is important to provide older adults with a medication reminder service. There are several mobile apps and devices capable of reminding medication, but their user interfaces and operations are usually unfriendly to seniors. Considering the above demands and shortcomings, we proposed a conversation-based medication management system (CMMS). The CMMS uses a companion robot and the cloud to create medication reminders and check medication adherence. We implemented the CMMS in our ASCC (Advanced Sensing, Computation and Control) companion robot. To evaluate the CMMS, we tested our system through the mobile app end with 23 human subjects and the robot end with 15 human subjects. The feedback from the post-test survey shows that the convenience, usefulness and total rating of our system is 8.217, 8.696 and 8.478 out of 10 from the mobile app end and 9.000, 8.933 and 8.533 out of 10 from the robot end, respectively. The System Usability Scale (SUS) score of our system is 81.333 from the robot end users, which means the participants had a high satisfaction level when using the system.

Negative affects such as anger, fear, nervousness, depression, etc., may increase human's susceptibility to illness. In this letter, we propose a negative emotion management system that is able to recognize negative emotions through ECG signal and perform emotion regulation through a robot assistant, which has a potential for reducing health risks. A smart shirt is developed to collect the ECG signal from the human body. The robot assistant has the ability to engage in verbal conversations with humans. Recurrence Quantitative Analysis (RQA) is used to extract ECG features for emotion classification purpose. Along with our own dataset, two other public datasets, RECOLA and DECAF, are also used to evaluate our methodology. The detection of negative emotion can trigger the robot assistant to help the user get out of such situations through interactive conversations. We tested and evaluated the proposed framework through experiments. We also assessed the effectiveness of the interactions with the robot on the emotional well-being of older adults.

This paper proposes and implements an open framework of active auditory learning for a home service robot to serve the elderly living alone at home. The framework was developed to realize the various auditory perception capabilities while enabling a remote human operator to involve in the sound event recognition process for elderly care. The home service robot is able to estimate the sound source position and collaborate with the human operator in sound event recognition while protecting the privacy of the elderly. Our experimental results validated the proposed framework and evaluated auditory perception capabilities and human–robot collaboration in sound event recognition.