When responding to allegations of child sexual, physical, and psychological abuse, Child Protection Service (CPS) workers and police personnel need to elicit detailed and accurate accounts of the abuse to assist in decision-making and prosecution. Current research emphasizes the importance of the interviewer’s ability to follow empirically based guidelines. In doing so, it is essential to implement economical and scientific training courses for interviewers. Due to recent advances in artificial intelligence, we propose to generate a realistic and interactive child avatar, aiming to mimic a child. Our ongoing research involves the integration and interaction of different components with each other, including how to handle the language, auditory, emotional, and visual components of the avatar. This paper presents three subjective studies that investigate and compare various state-of-the-art methods for implementing multiple aspects of the child avatar. The first user study evaluates the whole system and shows that the system is well received by the expert and highlights the importance of its realism. The second user study investigates the emotional component and how it can be integrated with video and audio, and the third user study investigates realism in the auditory and visual components of the avatar created by different methods. The insights and feedback from these studies have contributed to the refined and improved architecture of the child avatar system which we present here.

This paper presents a detailed study of an AI-driven platform designed for the training of child welfare and law enforcement professionals in conducting investigative interviews with maltreated children. It achieves a subjective simulation of interview situation through the integration of fine-tuned GPT-3 models within the Unity framework. The study recruited participants from a range of backgrounds, including professionals experienced in conducting investigative interviews and individuals with academic qualifications in psychology, criminology, or related disciplines. To assess the effectiveness of this tool, a multi-method evaluation approach was utilized, incorporating both quantitative analyses and qualitative interviews. The quantitative methods included mixed-effects models, which provided insights into how effects such as the type of virtual environment (2D vs. 3D), scenario variations, virtual reality (VR) familiarity, and professional expertise influence the user experience. Additionally, structural equation modeling (SEM) provided deeper insights into the relationships between variables, offering a comprehensive understanding of how they collectively impact the user experience. The qualitative method included a detailed semi-structured interview that provided a deeper understanding of user experiences and perceptions. The findings indicate significant advantages of the 3D environment in enhancing Flow and Virtual Fidelity ; however, the 2D environment was favored for Usability . Despite the 3D environment’s potential for greater immersion, the discomfort associated with VR head-mounted displays (HMDs) led some users to prefer the 2D setup. Familiarity with VR technology positively influenced user perception, indicating that prior exposure can mitigate some of the Avatar Interaction Comfort issues. Additionally, the Hand Movement Perception was better understood in scenarios with sensitive themes. As user experience increased, participants had a more positive view of the Age-Appropriate Response . Furthermore, the dialog systems effectiveness, particularly Response Relevance and Detailed Responses , played a significant role in Empathy Elicitation , often outweighing Virtual Fidelity . However, Emotion in facial expressions and Responsiveness were two factors that negatively impacted the effectiveness of the tool, indicating areas that need improvement in the future.

The current study aims to evaluate and compare the subjective quality of an AI-based training system developed for conducting child interviews, focusing on the distinction between immersive 3D (using virtual reality) and 2D desktop environments. To this end, a structured user study was conducted, involving 36 participants who were exposed to these two distinct environments. The study evaluated various aspects of user experience, namely presence, usability, visual fidelity, emotion, responsiveness, appropriateness, and training effectiveness. The findings reveal significant differences in user experience between the 2D and 3D environments. Notably, the 3D environment enhanced presence, visual fidelity, training effectiveness, and empathy. In contrast, the 2D environment was favored for usability. The study highlights the potential of immersive VR while also pointing out the need to improve the system response and emotional expressiveness of the avatars.

Dynamic facial emotion is essential for believable AI-generated avatars; however, most systems remain visually inert, limiting their utility in simulations such as virtual training for investigative interviews with abused children. We introduce and evaluate a real-time architecture fusing Unreal Engine 5 MetaHuman rendering with NVIDIA Omniverse Audio2Face to translate vocal prosody into high-fidelity facial expressions on photorealistic child avatars. Due to limitations in synthetic voice options, both avatars were voiced using a young adult female TTS model, selected from two different systems in an attempt to better match each character. This compromise introduces a confounding factor. Voice-age mismatches and prosodic or emotional differences may disrupt audiovisual alignment. We implemented a two-PC setup decoupling language and speech synthesis from GPU-intensive rendering, designed to support low-latency interaction in desktop and VR environments. A between-subjects study ($N=70$) using audio+visual and visual-only conditions assessed perceptual impacts as participants rated emotional clarity, facial realism, and empathy for two avatars expressing joy, sadness, and anger. Our results show that while avatars could express emotions recognizably, particularly sadness and joy, recognition of anger dropped markedly without audio, highlighting the role of vocal cues in conveying high-arousal states. Interestingly, silencing the clips improved perceived realism by removing mismatches between facial animation and voice, especially where age or emotional tone were incongruent. These findings highlight that perceived believability hinges on the interplay between audiovisual congruence and facial geometry: a mismatched voice can undermine even well-crafted expressions, while a congruent one can enhance weaker visuals. This trade-off presents an ongoing challenge for designing emotionally coherent avatars in sensitive training contexts.

This paper explores advancements in real-time talking-head generation, focusing on overcoming challenges in Audio Feature Extraction (AFE), which often introduces latency and limits responsiveness in real-time applications. To address these issues, we propose and implement a fully integrated system that replaces conventional AFE models with OpenAI’s Whisper, leveraging its encoder to optimize processing and improve overall system efficiency. Our evaluation of two open-source real-time models across three different datasets shows that Whisper not only accelerates processing but also improves specific aspects of rendering quality, resulting in more realistic and responsive talking-head interactions. Although interviewer training systems are considered a potential application, the primary contribution of this work is the improvement of the technical foundations necessary for creating responsive AI avatars. These advancements enable more immersive interactions and expand the scope of AI-driven applications, including educational tools and simulated training environments.

This paper examines the integration of real-time talking-head generation for interviewer training, focusing on overcoming challenges in Audio Feature Extraction (AFE), which often introduces latency and limits responsiveness in real-time applications. To address these issues, we propose and implement a fully integrated system that replaces conventional AFE models with Open AI's Whisper, leveraging its encoder to optimize processing and improve overall system efficiency. Our evaluation of two open-source real-time models across three different datasets shows that Whisper not only accelerates processing but also improves specific aspects of rendering quality, resulting in more realistic and responsive talking-head interactions. These advancements make the system a more effective tool for immersive, interactive training applications, expanding the potential of AI-driven avatars in interviewer training.