Automatic engagement recognition is a technique that is used to measure the engagement level of people in a specific task. Although previous research has utilized expensive and intrusive devices such as physiological sensors and pressure-sensing chairs, methods using RGB video cameras have become the most common because of the cost efficiency and noninvasiveness of video cameras. Automatic engagement recognition methods using video cameras are usually based on hand-crafted features and a statistical temporal dynamics modeling algorithm. This paper proposes a data-driven convolutional neural networks (CNNs)-based engagement recognition method that uses only facial images from input videos. As the amount of data in a dataset of children's engagement is insufficient for deep learning, pre-trained CNNs are utilized for low-level feature extraction from each video frame. In particular, a new layer combination for temporal dynamics modeling is employed to extract high-level features from low-level features. Experimental results on a database created using images of children from kindergarten demonstrate that the performance of the proposed method is superior to that of previous methods. The results indicate that the engagement level of children can be gauged automatically via deep learning even when the available database is deficient.

Applying face alignment after face detection exerts a heavy influence on face recognition. Many researchers have recently investigated face alignment using databases collected from images taken at close distances and with low magnification. However, in the cases of home-service robots, captured images generally are of low resolution and low quality. Therefore, previous face alignment research, such as eye detection, is not appropriate for robot environments. The main purpose of this paper is to provide a new and effective approach in the alignment of small and blurred faces. We propose a face alignment method using the confidence value of Real-AdaBoost with a modified census transform feature. We also evaluate the face recognition system to compare the proposed face alignment module with those of other systems. Experimental results show that the proposed method has a high recognition rate, higher than face alignment methods using a manually-marked eye position.

A connected component analysis from a binary image is a popular character segmentation method but occasionally fails to segment the characters owing to image noise and uneven illumination. A multimethod binarization scheme that incorporates two or more binary images is a novel solution, but selection of binarization methods has never been analyzed before. This paper reveals the best combination of binarization methods and parameters and presents an in-depth analysis of the multimethod binarization scheme for better character segmentation. We carry out an extensive quantitative evaluation, which shows a significant improvement over conventional single-method binarization methods. Experiment results of six binarization methods and their combinations with different test images are presented.

The increase in elderly population is emerging as a serious social issue. The coronavirus pandemic has increased the number of elderly people suffering from depression and loneliness owing to the lack of face-to-face activities. In this study, we developed an integrated system for the human-care robot service, considering cognitive and emotional support for elderly people, and verified its stability and usefulness in the real world. We recruited 40 elderly people for an apartment testbed environment experiment and two elderly people living alone for a long time participated in the experiment at their homes. Quantitative experimental results were analyzed by comparing service success rates and user satisfaction in two different test environments to verify the stability of the service. Qualitative evaluations were also conducted through surveys and interviews to assess the usefulness of the service.