Virtual agents are artificial intelligence systems that can interact with users in virtual reality (VR), providing users with companionship and entertainment. Virtual pets have become the most popular virtual agents due to their many benefits. However, haptic interaction with virtual pets involves two challenges: the rapid construction of various haptic proxies, and the design of agent-initiated active interaction. In this paper, we propose a modular haptic agent (MHA) prototype system, enabling the tactile simulation and encountered-type haptic interaction of common virtual pet agents through a modular design method and a haptic mapping method. Meanwhile, the MHA system with haptic interaction is actively initiated by the agents according to the user’s intention, which makes the virtual agents appear more autonomous and provides a better experience of human–agent interaction. Finally, we conduct three user studies to demonstrate that the MHA system has more advantages in terms of realism, interactivity, attraction, and raising user emotions. Overall, MHA is a system that can build multiple companion agents, provide active interaction and has the potential to quickly build diverse haptic agents for an intelligent and comfortable virtual world.

Virtual reality enables us to access and interact with immersive virtual environments anytime and anywhere in various fields such as entertainment, training, and education. However, users immersed in virtual scenes remain physically connected to their real-world surroundings, which can pose safety and immersion challenges. Although virtual scene synthesis has attracted widespread attention, many popular methods are limited to generating purely virtual scenes independent of physical environments or simply mapping physical objects as obstacles. To this end, we propose a scene agent that synthesizes situated 3D virtual scenes as a kind of ubiquitous embodied interface in VR for users. The scene agent synthesizes scenes by perceiving the user's physical environment as well as inferring the user's demands. The synthesized scenes maintain the affordances of the physical environment, enabling immersive users to interact with the physical environment and improving the user's sense of security. Meanwhile, the synthesized scenes maintain the style described by the user, improving the user's immersion. The comparison results show that the proposed scene agent can synthesize virtual scenes with better affordance maintenance, scene diversity, style maintenance, and 3D intersection over union compared to baselines. To the best of our knowledge, this is the first work that achieves in situ scene synthesis with virtual-real affordance consistency and user demand.

Demonstrative learning in virtual reality (VR) is a pivotal learning strategy for knowledge transfer for embodied agents. While existing studies have extensively explored agents’ knowledge transfer through self-demonstrative learning (SDL) or teacher-demonstrative learning (TDL), there has been limited focus on a system that integrates both paradigms. This article proposes a comprehensive system that combines the SDL paradigm with the TDL paradigm in VR from a top-down perspective. The system involves using directed probabilistic graphs (DPGs) for knowledge representation, constructing detectors and actuators based on object fluents and atomic actions, representing knowledge acquired from both learning paradigms on the DPGs, and incorporating knowledge integration and visualization. Through system evaluation, we show the advantages of integrating two demonstrative learning paradigms, including increased learning efficiency, mitigating demonstrator’s deficiencies, and more logical task execution. The study also reveals that a dynamically decreasing fusion factor with the learning progresses, performs well across different correct percentages of teacher demonstrations. Additionally, we show a more decentralized demonstration in the middle of the agent’s learning progress maximizing learning efficiency when demonstrating only a few atomic actions. Finally, we assess the system’s generalizability and delineate its boundaries. With the ongoing development and the increasing availability of human data in VR, we anticipate that our system can be applied to future scenarios of efficient human-agent knowledge transfer in human-agent symbiosis.