This study shows an autonomous android robot that can have a natural daily dialogue with humans. The dialogue system for daily dialogue is different from a task-oriented dialogue system in that it is not given a clear purpose or the necessary information. That is, it needs to generate an utterance in a situation where there is no clear request from humans. Therefore, to continue a dialogue with a consistent content, it is necessary to essentially change the design policy of dialogue management compared with the existing dialogue system. The purpose of our study is to constructively find out the dialogue system architecture for realizing daily dialogue through implementing an autonomous dialogue robot capable of daily natural dialogue. We defined the android’s desire necessary for daily dialogue and the dialogue management system in which the android changes its internal (mental) states in accordance to the desire and partner’s behavior and chooses a dialogue topic suitable for the current situation. The developed android could continue daily dialogue for about 10 min in the scene where the robot and partner met for the first time in the experiment. Moreover, a multimodal Turing test has shown that half of the participants had felt that the android was remotely controlled to some degree, that is, the android’s behavior was humanlike. This result suggests that the system construction method assumed in this study is an effective approach to realize daily dialogue, and the study discusses the system architecture for daily dialogue.

Hand gestures commonly occur in daily dialogue interactions, and have important functions in communication. In this study, we proposed and implemented an online processing for a speech-driven gesture motion generation in an android robot dialogue system. Issues on motion overlaps and speech interruptions by the dialogue partner were taken into account. We then conducted two experiments to evaluate the effects of occasional dis-synchrony between the generated motions and speech, and the effects of holding duration control after speech interruptions. Evaluation results indicated that gestures should not be delayed by more than 400ms relative to the speech utterances. Evaluation of the second experiment indicated that gesture holding durations around 0.5 to 2 seconds after an interruption look natural, while longer durations may cause impression of displeasure by the robot.

Cybernetic Avatars (CAs) are controlled by an operator through an interface that communicates movements, voice, or the intent of action. The operator can use the CA to perform activities remotely. In other words, the CA is the operator’s alter ego. Therefore, the CA should have humanlike presence and lifelikeness. This chapter introduces related research, focusing on the development of a humanlike and life-like CA along with its interface technology.

Recent advancements in deep learning and large-scale language models have driven interest in creating conversational robots capable of heartful interactions with humans. However, most existing systems are tailored for specific contexts and lack a comprehensive framework to assist humans in daily activities. Developing a fully autonomous robot that can operate alongside humans and interact with them in dynamic environments is a complex task that requires addressing particular software and hardware demands. These demands often conflict, resulting in trade-offs based on the intended application. To overcome this challenge, we propose a body design that leverages passive dynamics. We illustrate how the essential devices for computation, sensing, and actuation can be integrated into the robot without compromising its embodiment requirements. Conversely, we utilize the weight and placement of these devices to achieve the physical characteristics needed for a robotic body that is well-suited for coexistence with humans. We validate this approach through the development of Indy, a mobile communication robot platform designed to coexist with humans. The employed mechanisms enable Indy to maintain an upright posture without active control and ensure safe physical interactions. Experimental results indicate that the generated body movement positively impacts users’ perceptions, promoting a sense of anthropomorphism and enhancing the interaction experience. This confirms that the selected design approach not only provides mechanical advantages but also improves the natural appearance of the robot’s movements.

This study explores the potential benefits of robots having the capability to anticipate people’s mental states in an exercise context. We designed 80 utterances for a robot with associated gestures that exhibit a range of emotional characteristics and then performed a 23-person data collection to investigate the effects of these robot behaviors on human mental states during exercise. The results of cluster analysis revealed that (1) utterances with similar meanings had the same effect and (2) the effects of a certain cluster on different people depend on their emotional state. On the basis of these findings, we proposed a robotic system that anticipates the effect of utterances on the individual’s future mental state, thereby choosing utterances that can positively impact the individual. This system incorporates three main features: (1) associating the relevant events detected by sensors with a user’s emotional state; (2) anticipating the effects of robot behavior on the user’s future mental state to choose the next behavior that maximizes the anticipated gain; and (3) determining appropriate times to provide coaching feedback, using predefined rules in the motion module for timing decisions. To evaluate the proposed system’s overall performance comprehensively, we compare robots equipped with the system’s unique features to those lacking these features. We design the baseline condition that lacks these unique features, opting for periodic random selection of utterances for interaction based on the current context. We conducted a 21-person experiment to evaluate the system’s performance. We found that participants perceived the robot to have a good understanding of their mental states and that they enjoyed the exercises more and put in more effort due to the robot’s encouragement.