In order to realize fast real-time positioning after a mobile robot starts, this paper proposes an improved ORB-SLAM2 algorithm. Firstly, we proposed a binary vocabulary storage method and vocabulary training algorithm based on an improved Oriented FAST and Rotated BRIEF (ORB) operator to reduce the vocabulary size and improve the loading speed of the vocabulary and tracking accuracy. Secondly, we proposed an offline map construction algorithm based on the map element and keyframe database; then, we designed a fast reposition method of the mobile robot based on the offline map. Finally, we presented an offline visualization method for map elements and mapping trajectories. In order to check the performance of the algorithm in this paper, we built a mobile robot platform based on the EAI-B1 mobile chassis, and we implemented the rapid relocation method of the mobile robot based on improved ORB SLAM2 algorithm by using C++ programming language. The experimental results showed that the improved ORB SLAM2 system outperforms the original system regarding start-up speed, tracking and positioning accuracy, and human–computer interaction. The improved system was able to build and load offline maps, as well as perform rapid relocation and global positioning tracking. In addition, our experiment also shows that the improved system is robust against a dynamic environment.

As the problem of an aging population becomes more and more serious, social robots have an increasingly significant influence on human life. By employing regular question-and-answer conversations or wearable devices, some social robotics products can establish personal health archives. But those robots are unable to collect diet information automatically through robot vision or audition. A healthy diet can reduce a person’s risk of developing cancer, diabetes, heart disease, and other age-related diseases. In order to automatically perceive the dietary composition of the elderly by listening to people’s chatting, this paper proposed a chat-based automatic dietary composition perception algorithm (DCPA). DCPA uses social robot audition to understand the semantic information and percept dietary composition for Mandarin Chinese. Firstly, based on the Mel-frequency cepstrum coefficient and convolutional neural network, a speaker recognition method is designed to identify speech data. Based on speech segmentation and speaker recognition algorithm, an audio segment classification method is proposed to distinguish different speakers, store their identity information and the sequence of expression in a speech conversation. Secondly, a dietetic lexicon is established, and two kinds of dietary composition semantic understanding algorithms are proposed to understand the eating semantics and sensor dietary composition information. To evaluate the performance of the proposed DCPA algorithm, we implemented the proposed DCPA in our social robot platform. Then we established two categories of test datasets relating to a one-person and a multi-person chat. The test results show that DCPA is capable of understanding users’ dietary compositions, with an F1 score of 0.9505, 0.8940 and 0.8768 for one-person talking, a two-person chat and a three-person chat, respectively. DCPA has good robustness for obtaining dietary information.

To better characterize the differences in category features in Facial Expression Recognition (FER) tasks, and improve inter-class separability and intra-class compactness, we propose a Multiscale Facial Expression Recognition model based on dynamic global and static local attention (MFER) from the perspectives of intra-class and inter-class features. Firstly, we propose Dynamic global and Static local attention (DS Attention) mechanism that fuse contextual information, learn potential regions of global and local features between different expression categories, and represent feature discrepancies between categories to distinguish between different expression categories. Then, we design a Deep Smooth Feature loss function (DSF) to balance the probability difference of encoded intra-class features and promote intra-class features towards corresponding centers. Finally, we construct a Multiscale classifier method (Msc) to learn high-frequency and low-frequency information in the dimensional space, represent deep features of multiscale dimensional space, and alleviate sparse distribution problems in high-dimensional space. Experimental results on public datasets RAF-DB, AffectNet-7, AffectNet-8, and FERPlus show that the proposed model achieves state-of-the-art performance with recognition accuracies of 92.08%, 67.06%, 63.15%, and 91.09%, respectively. Code available at https://github.com/XuJ1E/MFER/.

Non‐suicide self‐injury (NSSI) can be dangerous and difficult for guardians or caregivers to detect in time. NSSI refers to when people hurt themselves even though they have no wish to cause critical or long‐lasting hurt. To timely identify and effectively prevent NSSI in order to reduce the suicide rates of patients with a potential suicide risk, the detection of NSSI based on the spatiotemporal features of indoor activities is proposed. Firstly, an NSSI behaviour dataset is provided, and it includes four categories that can be used for scientific research on NSSI evaluation. Secondly, an NSSI detection algorithm based on the spatiotemporal features of indoor activities (NssiDetection) is proposed. NssiDetection calculates the human bounding box by using an object detection model and employs a behaviour detection model to extract the temporal and spatial features of NSSI behaviour. Thirdly, the optimal combination schemes of NssiDetection is investigated by checking its performance with different behaviour detection methods and training strategies. Lastly, a case study is performed by implementing an NSSI behaviour detection prototype system. The prototype system has a recognition accuracy of 84.18% for NSSI actions with new backgrounds, persons, or camera angles.

Keyframe extraction can effectively help users quickly understand video content. Generally, keyframes should be representative of the video content and simultaneously be diverse to reduce redundancy. Aiming to find the features of frames and filter out representative frames of the video, we propose a method of keyframe recommendation based on feature intercross and fusion (KFRFIF). The method is inspired by the implied relations between keyframe-extraction problem and recommendation problem. First, we investigate the application of a recommendation framework to the keyframe extraction problem. Second, the architecture of the proposed KFRFIF is put forward. Then, an algorithm for extracting intra-frame image features based on the combination of multiple image descriptors is proposed. An algorithm for extracting inter-frame distance features based on the combination of multiple distance calculation methods is designed. Moreover, A recommendation model based on feature intercross and fusion is put forward. An ablation study is further performed to verify the effectiveness of the submodule. Ultimately, the experimental results on four datasets with five outstanding approaches indicate the superior performance of our approach.