To assist personalized healthcare of elderly people, our interest is to develop a virtual caregiver system that retrieves the expression of mental and physical health states through human–computer interaction in the form of dialogue. The purpose of this paper is to implement and evaluate a virtual caregiver system using mobile chatbot. Unlike the conventional health monitoring approach, our key idea is to integrate a rule-based virtual caregiver system (called “Mind Monitoring” service) with the physical, mental, and social questionnaires into the mobile chat application. The elderly person receives one question from the mobile chatbot per day, and answers it by pushing the optional button or using a speech recognition technique. Furthermore, a novel method is implemented to quantify the answers, generate visual graphs, and send the corresponding summaries or advice to the specific elder. In the experimental evaluation, we applied it to eight elderly subjects and 19 younger subjects within 14 months. As main results, its effects were significantly improved by the proposed method, including the above 80% in the response rate, the accurate reflection of their real lives from the responses, and high usefulness of the feedback messages with software quality requirements and evaluation. We also conducted interviews with subjects for health analysis and improvement.

Multimodal interaction technology has become a key aspect of remote education by enriching student engagement and learning results as it utilizes the speech, gesture, and visual feedback as various sensory channels. This publication reflects on the latest breakthroughs in multimodal interaction and its usage in remote learning environments, including a multi-layered discussion that addresses various levels of learning and understanding. It showcases the main technologies, such as speech recognition, computer vision, and haptic feedback, that enable the visitors and learning portals to exchange data fluidly. In addition, we investigate the function of multimodal learning analytics in order to measure the cognitive and emotional states of students, targeting personalized feedback and refining instructional strategies. Though multimodal communication may bring a historical improvement to the mode of online education, the platform still faces many issues, such as media synchronization, higher computational demand, physical adaptability, and privacy concerns. These problems demand further research in the fields of algorithm optimization, access to technology guidance, and the ethical use of big data. This paper presents a systematic review of the application of multimodal interaction in remote education. Through the analysis of 25 selected research papers, this review explores key technologies, applications, and challenges in the field. By synthesizing existing findings, this study highlights the role of multimodal learning analytics, speech recognition, gesture-based interaction, and haptic feedback in enhancing remote learning.

In contrast to the physical activities of able-bodied people at home, most people who require long-term specific care (e.g., bedridden patients and patients who have difficulty walking) usually show more low-intensity slow physical activities with postural changes. Although the existing devices can detect data such as heart rate and the number of steps, they have been increasing the physical burden relying on long-term wearing. The purpose of this paper is to realize a noninvasive fine-grained home care monitoring system that is sustainable for people requiring special care. In the proposed method, we present a novel technique that integrates inexpensive camera devices and bone-based human sensing technologies to characterize the quality of in-home postural changes. We realize a local process in feature data acquisition once per second, which extends from a computer browser to Raspberry Pi. Our key idea is to regard the changes of the bounding box output by standalone pose estimation models in the shape and distance as the quality of the pose conversion, body movement, and positional changes. Furthermore, we use multiple servers to realize distributed processing that uploads data to implement home monitoring as a web service. Based on the experimental results, we conveyed our findings and advice to the subject that include where the daily living habits and the irregularity of home care timings needed improvement.

With the rapid advancement of mobile technology, e-learning has expanded significantly, making language learning more accessible than ever. At the same time, the rise of artificial intelligence (AI) technologies has opened new avenues for adaptive and personalized e-learning experiences. However, traditional e-learning methods remain limited by their reliance on static, predefined materials, which restricts equitable access to learning resources and fails to fully support lifelong learning. To address this limitation, this study proposes a location-based AI-driven e-learning system that dynamically generates language learning materials tailored to real-world contexts by integrating location-awareness technology with AI. This approach enables learners to acquire language skills that are directly applicable to their physical surroundings, thereby enhancing engagement, comprehension, and retention. Both objective evaluation and user surveys confirm the reliability and effectiveness of AI-generated language learning materials. Specifically, user surveys indicate that the generated content achieves a content relevance score of 8.4/10, an accuracy score of 8.8/10, a motivation score of 7.9/10, and a learning efficiency score of 7.8/10. Our method can reduce reliance on predefined content, allowing learners to access location-relevant learning resources anytime and anywhere, thereby improving accessibility and fostering lifelong learning in the context of sustainable education.

To capture scientific evidence in elderly care, a user-defined facial expression sensing service was proposed in our previous study. Since the time-series data of feature values have been growing at a high rate as the measurement time increases, it may be difficult to find points of interest, especially for detecting changes from the elderly facial expression, such as many elderly people can only be shown in a micro facial expression due to facial wrinkles and aging. The purpose of this paper is to implement a method to efficiently find points of interest (PoI) from the facial feature time-series data of the elderly. In the proposed method, the concept of changing point detection into the analysis of feature values is incorporated by us, to automatically detect big fluctuations or changes in the trend in feature values and detect the moment when the subject’s facial expression changed significantly. Our key idea is to introduce the novel concept of composite feature value to achieve higher accuracy and apply change-point detection to it as well as to single feature values. Furthermore, the PoI finding results from the facial feature time-series data of young volunteers and the elderly are analyzed and evaluated. By the experiments, it is found that the proposed method is able to capture the moment of large facial movements even for people with micro facial expressions and obtain information that can be used as a clue to investigate their response to care.

In today’s world, services are essential in daily life, and identifying each person’s unique needs is key to creating a human-centered society. Traditional research has used machine learning to recommend services based on user behavior logs without directly detecting individual needs. This study introduces a system called Needs Companion, which automatically detects individual service needs, laying the groundwork for accurate needs sensing. The system defines a needs data model based on the 6W1H framework, uses virtual agents for needs elicitation, and applies large language models (LLMs) to analyze and automatically extract needs. Experiments showed that the system could detect needs accurately and quickly. This research provides interpretable data for personalized services and contributes to fields like machine learning, human-centered design, and requirements engineering.