In order to realize fast real-time positioning after a mobile robot starts, this paper proposes an improved ORB-SLAM2 algorithm. Firstly, we proposed a binary vocabulary storage method and vocabulary training algorithm based on an improved Oriented FAST and Rotated BRIEF (ORB) operator to reduce the vocabulary size and improve the loading speed of the vocabulary and tracking accuracy. Secondly, we proposed an offline map construction algorithm based on the map element and keyframe database; then, we designed a fast reposition method of the mobile robot based on the offline map. Finally, we presented an offline visualization method for map elements and mapping trajectories. In order to check the performance of the algorithm in this paper, we built a mobile robot platform based on the EAI-B1 mobile chassis, and we implemented the rapid relocation method of the mobile robot based on improved ORB SLAM2 algorithm by using C++ programming language. The experimental results showed that the improved ORB SLAM2 system outperforms the original system regarding start-up speed, tracking and positioning accuracy, and human–computer interaction. The improved system was able to build and load offline maps, as well as perform rapid relocation and global positioning tracking. In addition, our experiment also shows that the improved system is robust against a dynamic environment.

As the problem of an aging population becomes more and more serious, social robots have an increasingly significant influence on human life. By employing regular question-and-answer conversations or wearable devices, some social robotics products can establish personal health archives. But those robots are unable to collect diet information automatically through robot vision or audition. A healthy diet can reduce a person’s risk of developing cancer, diabetes, heart disease, and other age-related diseases. In order to automatically perceive the dietary composition of the elderly by listening to people’s chatting, this paper proposed a chat-based automatic dietary composition perception algorithm (DCPA). DCPA uses social robot audition to understand the semantic information and percept dietary composition for Mandarin Chinese. Firstly, based on the Mel-frequency cepstrum coefficient and convolutional neural network, a speaker recognition method is designed to identify speech data. Based on speech segmentation and speaker recognition algorithm, an audio segment classification method is proposed to distinguish different speakers, store their identity information and the sequence of expression in a speech conversation. Secondly, a dietetic lexicon is established, and two kinds of dietary composition semantic understanding algorithms are proposed to understand the eating semantics and sensor dietary composition information. To evaluate the performance of the proposed DCPA algorithm, we implemented the proposed DCPA in our social robot platform. Then we established two categories of test datasets relating to a one-person and a multi-person chat. The test results show that DCPA is capable of understanding users’ dietary compositions, with an F1 score of 0.9505, 0.8940 and 0.8768 for one-person talking, a two-person chat and a three-person chat, respectively. DCPA has good robustness for obtaining dietary information.

Recently, deep learning has achieved remarkable breakthroughs in hyperspectral image (HSI) classification tasks, particularly with methods based on convolutional neural networks (CNNs) and transformers. However, these methods have several limitations: 1) the limited receptive field inherent in the convolutional layer greatly hampers capturing feature contextual information on a large scale and 2) transformers cannot establish strong local relationships, making it challenging to characterize complex dependencies between distant pixels and different bands in HSIs. Moreover, as the network complexity increases, so does the number of network parameters. We propose a novel network called the spectral-spatial convolutional fusion long-range dependence transformer network (LRDTN) for HSI classification to address these challenges. LRDTN comprises three key components: dynamic-dependent convolutional (DDC) module, the multiscale enhanced fusion (MsEF) module, and the local–global perception transformer (LGPT). Specifically, the DDC dynamically models local features, while the MsEF integrates information from different scales to capture contextual relationships in HSI features effectively. Additionally, the ability to mine and utilize HSI local-global features and complex long-range dependencies is enhanced by the proposed transformer variant, LGPT. Ultimately, through the ingeniously designed structure of the LRDTN, the model effectively maintains its performance while reducing the number of network parameters. Extensive experiments conducted on four typical HSI datasets, including urban areas, agricultural areas, and swamps, demonstrate the superiority of LRDTN over other state-of-the-art networks. The code is available at https://github.com/ybyangjing/LRDTN .

Memory loss is a part of normative aging. Many older adults commonly forget to take prescribed medication, which can have an adverse effect on health. Therefore, it is important to provide older adults with a medication reminder service. There are several mobile apps and devices capable of reminding medication, but their user interfaces and operations are usually unfriendly to seniors. Considering the above demands and shortcomings, we proposed a conversation-based medication management system (CMMS). The CMMS uses a companion robot and the cloud to create medication reminders and check medication adherence. We implemented the CMMS in our ASCC (Advanced Sensing, Computation and Control) companion robot. To evaluate the CMMS, we tested our system through the mobile app end with 23 human subjects and the robot end with 15 human subjects. The feedback from the post-test survey shows that the convenience, usefulness and total rating of our system is 8.217, 8.696 and 8.478 out of 10 from the mobile app end and 9.000, 8.933 and 8.533 out of 10 from the robot end, respectively. The System Usability Scale (SUS) score of our system is 81.333 from the robot end users, which means the participants had a high satisfaction level when using the system.

Negative affects such as anger, fear, nervousness, depression, etc., may increase human's susceptibility to illness. In this letter, we propose a negative emotion management system that is able to recognize negative emotions through ECG signal and perform emotion regulation through a robot assistant, which has a potential for reducing health risks. A smart shirt is developed to collect the ECG signal from the human body. The robot assistant has the ability to engage in verbal conversations with humans. Recurrence Quantitative Analysis (RQA) is used to extract ECG features for emotion classification purpose. Along with our own dataset, two other public datasets, RECOLA and DECAF, are also used to evaluate our methodology. The detection of negative emotion can trigger the robot assistant to help the user get out of such situations through interactive conversations. We tested and evaluated the proposed framework through experiments. We also assessed the effectiveness of the interactions with the robot on the emotional well-being of older adults.

In this paper, we presented a multimodal approach to monitoring older adults’ activities of daily living (ADLs) using the combination of a wearable device and a companion robot. A dynamic Bayesian network (DBN) model was developed for activity recognition which fuses different data, including location, object, sound event, body action and time. The walking action is detected as the transition between consecutive activities, which helps capture the inception of activities and save energy on the wearable device. Three tests were conducted to evaluate the proposed approach. First, multiple daily activities were simulated and evaluated the approach based on a public ADL dataset. Second, the proposed approach was tested based on an offline dataset collected in our smart home testbed which contains images, sound events, motion and time data. Third, the proposed approach was tested in real time and a web-based interface was developed which helps caregivers better monitor the ADLs of older adults and provide further assistance. In the offline test and the real time test, the results show that the system achieved 91% and 93% activity detection ratio, respectively, which significantly outperformed the baseline periodic sampling methods. In addition, the camera and microphone sensor trigger times were reduced from 1537 to 140 and 78, leading to energy reduction of 36.0% and 37.6% on the wearable device, respectively.