26 May 2023 in IEEE Transactions on Robotics
Reminiscence therapy (RT) can improve the mood and communication of persons living with Alzheimer's disease and Alzheimer's-disease-related dementias (PLWD). Traditional RT requires professionals' facilitation, limiting its accessibility to PLWD. Social robotics has the potential to facilitate RT, enabling accessible home-based RT. However, studies are needed to investigate how PLWD would perceive a robot-mediated reminiscence therapy (RMRT) and how to develop RMRT for positive user experience and successful adoption. In this article, we develop a prototype of RMRT using a humanoid social robot and test it with 12 participants (seven PLWD, two with mild cognitive impairment, and three informal caregivers). The robot automatically displays a memory trigger on its tablet and engages participants in a relatable conversation during RMRT. A mixed-method approach is employed to assess its acceptability and usability. Our results show that PLWD have an overall positive user experience with the RMRT. Participants laugh and sing along with the robot during RMRT and demonstrate intention to use it. In addition, we discuss a robot control method and several critical problems for RMRT. The RMRT can facilitate both verbal and nonverbal social interaction for PLWD and holds promise for engaging, personalized, and efficient home-based cognitive exercises for PLWD.
22 August 2025 in Frontiers in Robotics and AI
Multimodal perception is essential for enabling robots to understand and interact with complex environments and human users by integrating diverse sensory data, such as vision, language, and tactile information. This capability plays a crucial role in decision-making in dynamic, complex environments. This survey provides a comprehensive review of advancements in multimodal perception and its integration with decision-making in robotics from year 2004–2024. We systematically summarize existing multimodal perception-driven decision-making (MPDDM) frameworks, highlighting their advantages in dynamic environments and the methodologies employed in human-robot interaction (HRI). Beyond reviewing these frameworks, we analyze key challenges in multimodal perception and decision-making, focusing on technical integration and sensor noise, adaptation, domain generalization, and safety and robustness. Finally, we outline future research directions, emphasizing the need for adaptive multimodal fusion techniques, more efficient learning paradigms, and human-trusted decision-making frameworks to advance the HRI field.
31 January 2022 in International Journal of Social Robotics
1 October 2024 in International Journal of Social Robotics