Gestures play an integral role in human communication. Our research aims to develop a gesture understanding system that allows for better interpretation of human instructions in household robotics settings. We conducted an experiment with 34 participants who used pointing gestures to teach concepts to an assistant. Gesture data were analyzed using manual annotations (MAXQDA) and the computational methods of pose estimation and k-means clustering. The study revealed that participants tend to maintain consistent pointing styles, with one-handed pointing and index finger gestures being the most common. Gaze and pointing often co-occur, as do leaning forward and pointing. Using our gesture categorization algorithm, we analyzed gesture information values. As the experiment progressed, the information value of gestures remained stable, although the trends varied between participants and were associated with factors such as age and gender. These findings underscore the need for gesture recognition systems to balance generalization with personalization for more effective human–robot interaction.

The number of smart devices is expected to exceed 100 billion by 2050, and many will feature conversational user interfaces. Thus, methods for generating appropriate prosody for the responses of embodied conversational agents will be very important. This paper presents the results of the “Talk to Kotaro” experiment, which was conducted to better understand how people from different cultural backgrounds react when listening to prosody and phone choices for the IPA symbol-based gibberish speech of the virtual embodied conversational agent Kotaro. It also presents an analysis of the responses to a post-experiment Likert scale questionnaire and the emotions estimated from the participants’ facial expressions, which allowed one to obtain a phone embedding matrix and to conclude that there is no common cross-cultural baseline impression regarding different prosody parameters and that similarly sounding phones are not close in the embedding space. Finally, it also provides the obtained data in a fully anonymous data set.

This paper introduces a novel VGG-16-based visual navigation architecture for a differential drive robot, Social Plantroid, using a temporal vision mosaic, a novel approach which joins current and previous robot vision frames for heading estimation. As a minor contribution, it integrates a novel sunlight/shadow detection algorithm using Gabor filters. The neural network is trained with simulated data employing the artificial potential field method, which is another novelty for map-less robot navigation. Virtual and real-world experiments validate the effectiveness of this architecture in obstacle avoidance and navigation.

We present iChores, an open-source, robot-agnostic interface designed to facilitate intuitive human interaction for household tasks, enabling users to direct robots through natural language and deictic gestures (pointing) to perform complex chores, such as tidying a room. The system utilizes a modular microservice architecture that integrates state-of-the-art vision, natural language understanding (NLU), and reasoning modules. We detail the implementation of the framework, provide a public repository for reproducibility, and present an evaluation of 30 scenarios of increasing complexity. The general architecture of iChores and its various modules are briefly described. The implementation is publicly available through our GitHub project page, where all modules are provided as their own repositories.