14 April 2023 in IEEE Robotics and Automation Letters
Robots can tackle human–robot interaction (HRI) tasks through inverse reinforcement learning (IRL). However, offline IRL agents' performance is upper-bounded by experts. Limited demonstration fails to provide an overall picture of the environment, especially in real-world applications. To further enhance IRL's performance, we implement a cross-modal inverse reinforcement learning to reinforcement learning (IRL-to-RL) transition framework for a real-world HRI interaction task, in which a mall receptionist android promotes sanitizer usage. During the 10-day experiment, the android develops a more proactive and effective strategy than the human expert. Furthermore, we explore four decay modes of prior knowledge supervision and suggest a preferable pattern for practical use. Our results demonstrate the feasibility of the framework to assist robots in switching to diverse modalities, learning incrementally with a sparse reward function, and eventually outperforming the human expert. We anticipate our framework to inspire more IRL-to-better learning paradigms, facilitating robots to outcompete teachers in more real-world HRI applications.
3 June 2022 in IEEE Robotics and Automation Letters
For human-robot interaction (HRI), it is difficult to hand-craft all the rules for robots owing to diverse situations. Therefore, inverse reinforcement learning (IRL) is a potential solution that helps transfer human knowledge about interactions to robots. However, the feasibility of practically using IRL for HRI remains unknown. Here, we demonstrate a practical HRI application of IRL. An android was trained using IRL and acted as a receptionist to encourage visitors to practice hand hygiene in a shopping mall. We found that android learning through IRL has a competitive ability to a well-trained human operator on the reception task. Furthermore, we found that the android maintained high performance regardless of customer traffic. Our results demonstrate the potential of IRL in advancing the social HRI field. We anticipate that our work will be a starting point for using IRL in future HRI applications.
17 July 2024 in IEEE Access
Systems in the real world often consist of multiple subsystems interacting with each other, for example, the musculoskeletal system or human-human interaction. The measurement of temporal changes in these systems involves a multidimensional time series. This study introduces a novel framework for extracting features from multidimensional time series data with a group structure using self-supervised learning techniques. Specifically, we use a “lag operation,” which is a temporal shifting operation applied to the features of a certain group. We propose a self-supervised learning method for a neural network model that uses the data automatically generated by the lag operation and its corresponding operation labels to capture and quantify the interaction between groups. Upon completion of the training process, the representation space is obtained with the expectation that it will capture timing-dependent features within its boundaries. We define and calculate the interaction score, R-score, on the obtained space. To validate our approach, we apply the proposed methodology to an artificial oscillator and approximately 4 hours of conversational data to evaluate the R-score properties. From the results of the artificial data, the R-score increases when the connection between the groups is large. From the high R-score region of the representation space of the conversation data, we extract the data that contain social behaviors such as “eye contact,” “turn-taking,” and “smiling,” which are related to the interaction between the participants. The experimental results suggest that the proposed method can obtain a representation space for time series data with a group structure.