The demand for a conversational system with digital humans has increased with the development of artificial intelligence. Latency can occur in such conversational systems because of natural language processing and network issues, which can deteriorate the user’s performance and the availability of the systems. There have been attempts to mitigate user-perceived latency by using conversational fillers in human–agent interaction and human–robot interaction. However, non-verbal cues, such as gestures, have received less attention in such attempts, despite their essential roles in communication. Therefore, we designed gestural fillers for the digital humans. This study examined the effects of whether the conversation type and gesture filler matched or not. We also compared the effects of the gestural fillers with conversational fillers. The results showed that the gestural fillers mitigate user-perceived latency and affect the willingness, impression, competence, and discomfort in conversations with digital humans.

The demand for kiosk systems with embodied conversational agents has increased with the development of artificial intelligence. There have been attempts to utilize non-verbal cues, particularly virtual human (VH) eye contact, to enable human-like interaction. Eye contact with VHs can affect satisfaction with the system and the perception of VHs. However, when rendered in 2D kiosks, the gaze direction of a VH can be incorrectly perceived, due to a lack of stereo cues. A user study was conducted to examine the effects of the gaze behavior of VHs in multi-party conversations in a 2D display setting. The results showed that looking at actual speakers affects the perceived interpersonal skills, social presence, attention, co-presence, and competence in conversations with VHs. In a second study, the gaze perception was further examined with consideration of the Mona Lisa effect, which can lead users to believe that a VH rendered on a 2D display is gazing at them, regardless of the actual direction, within a narrow range. We also proposed the camera rotation angle fine tuning (CRAFT) method to enhance the users’ perceptual accuracy regarding the direction of the VH’s gaze.The results showed that the perceptual accuracy for the VH gaze decreased in a narrow range and that CRAFT could increase the perceptual accuracy.

The demand to utilize digital humans for conversation systems in multiple areas has increased. Time delays in a conversation with digital humans can be occurred because of network and natural language processing, which can deteriorate the users’ satisfaction and the availability of the digital humans. In this study, we designed a gestural filler that can be exhibited when delay occurs during a conversation with a digital human, and compared the effects of the gestural filler with conversational filler.

The adoption of kiosks in public spaces is steadily increasing, with a trend toward providing more natural user experiences through embodied conversational agents (ECAs). To achieve human-like interactions, ECAs should be able to appropriately gaze at the speaker. However, kiosks in public spaces often face challenges, such as ambient noise and overlapping speech from multiple people, making it difficult to accurately identify the speaker and direct the ECA’s gaze accordingly. In this paper, we propose a lightweight gaze control system that is designed to operate effectively within the resource constraints of kiosks and the noisy conditions common in public spaces. We first developed a speaker detection model that identifies the active speaker in challenging noise conditions using only a single camera and microphone. The proposed model achieved a 91.6% mean Average Precision (mAP) in active speaker detection and a 0.6% improvement over the state-of-the-art lightweight model (Light ASD) (as evaluated on the noise-augmented AVA-Speaker Detection dataset), while maintaining real-time performance. Building on this, we developed a gaze control system for ECAs that detects the dominant speaker in a group and directs the ECA’s gaze toward them using an algorithm inspired by real human turn-taking behavior. To evaluate the system’s performance, we conducted a user study with 30 participants, comparing the system to a baseline condition (i.e., a fixed forward gaze) and a human-controlled gaze. The results showed statistically significant improvements in social/co-presence and gaze naturalness compared to the baseline, with no significant difference between the system and human-controlled gazes. This suggests that our system achieves a level of social presence and gaze naturalness comparable to a human-controlled gaze. The participants’ feedback, which indicated no clear distinction between human- and model-controlled conditions, further supports the effectiveness of our approach.

Accurate flood risk assessments are critical for mitigating the impacts of pluvial flooding in densely populated urban areas. However, conventional flood modeling approaches often face limitations due to the lack of measurement information. To address this challenge, we develop, implement, and evaluate a novel framework that integrates crowdsourced data, such as citizen observations, with process-based modeling to enhance the accuracy of urban flood risk assessments. The proposed method utilizes indicator co-kriging techniques to merge citizen-sourced data with auxiliary variables, including inundation maps generated from 1D-2D urban flood models driven by high-resolution radar rainfall estimates. The framework is applied to the Oncheon River catchment in Busan, South Korea, a region highly vulnerable to pluvial flooding due to its urbanization and complex hydrological conditions. To evaluate the method, synthetic citizen observation data were generated based on inundation maps. These synthetic experiments assess the influence of the spatial distribution and quality of citizen observations on urban flood risk predictions. This study examines the integration of citizen observations into urban flood modeling workflows to address uncertainties in models and observations. In particular, we investigate the extent to which distributed citizen observations enhance prediction accuracy and analyze the effects of model bias on the reliability of flood risk assessments. The study quantitatively evaluates the effects of citizen data quality and spatial distribution on the accuracy of urban flood risk mapping. Furthermore, a sensitivity analysis is conducted for co-kriging parameters, focusing on semivariogram model selection and its influence on prediction accuracy.