AudienceMR is designed as a multi-user mixed reality space that seamlessly extends the local user space to become a large, shared classroom where some of the audience members are seen seated in a real space, and more members are seen through an extended portal. AudienceMR can provide a sense of the presence of a large-scale crowd/audience with the associated spatial context. In contrast to virtual reality (VR), however, with mixed reality (MR), a lecturer can deliver content or conduct a performance from a real, actual, comfortable, and familiar local space, while interacting directly with real nearby objects, such as a desk, podium, educational props, instruments, and office materials. Such a design will elicit a realistic user experience closer to an actual classroom, which is currently prohibitive owing to the COVID-19 pandemic. This paper validated our hypothesis by conducting a comparative experiment assessing the lecturer’s experience with two independent variables: (1) an online classroom platform type, i.e., a 2D desktop video teleconference, a 2D video screen grid in VR, 3D VR, and AudienceMR, and (2) a student depiction, i.e., a 2D upper-body video screen and a 3D full-body avatar. Our experiment validated that AudienceMR exhibits a level of anxiety and fear of public speaking closer to that of a real classroom situation, and a higher social and spatial presence than 2D video grid-based solutions and even 3D VR. Compared to 3D VR, AudienceMR offers a more natural and easily usable real object-based interaction. Most subjects preferred AudienceMR over the alternatives despite the nuisance of having to wear a video see-through headset. Such qualities will result in information conveyance and an educational efficacy comparable to those of a real classroom, and better than those achieved through popular 2D desktop teleconferencing or immersive 3D VR solutions.

We present ASAP, a system that uses virtual humans to Automatically generate Storyboards And Pre-visualized scenes from movie scripts. In our ASAP system, virtual humans play the role of actors. To visualize the screenplay scene, our system understands the movie script, which is the text data, and then facilitates the automatic generation of the following virtual human's non-/verbal behavior: (1) co-speech gesture, (2) facial expression, and (3) body movements. First of all, co-speech gestures are created from dialogue paragraphs using a text-to-gesture model trained with 2D videos and 3D motion-captured data. Next, for the facial expressions, we interpret the actors' emotions in the parenthetical paragraphs and then adjust the virtual human's face animation to reflect emotions such as anger and sadness. For body movements, our system extract action entities from action paragraphs (e.g., subject, target, and action) and then combine sets of animations to make animation sequences (e.g., a man's act of sitting on a bed). As soon as possible, ASAP can reduce the amount of time, money, and labor-intensive work that needs to be done in the early stages of filmmaking.

Serendipity is the phenomenon of people making unexpected and beneficial discoveries. While research on the mechanism and effectiveness of serendipity in information acquisition has been actively conducted, little attempt has been made to quantify serendipity when it occurs. In this paper, we present a quantitative model that measures serendipity experienced by users in a hypertext environment. In order to propose an evaluation model that measures how probable users would experience serendipitous moments in the process of an active search, we define a serendipitous discovery as an unexpected discovery that can happen during a sidetracked search. The proposed model consists of three parts: (a) pre-encountering—how early the user falls into the sidetracked search in the process of an active search; (b) post-encountering—the degree of interests of the entire process from the active search to obtaining unxpected information; and (c) discovery—the degree of the unexpectedness of the information obtained from the discovery. We evaluated the proposed model against examples with different structures and the potential serendipity values computed indicated the difference between the spaces in a meaningful way.

In light of the COVID-19 pandemic, wearing a mask is crucial to avoid contracting infectious diseases. However, wearing a mask is known to impair communication functions. This study aims to address the communication difficulties caused by wearing a mask and provide a strategy for aiding in understanding the speaker’s speech through facial animation. Facial animation is generated in real-time, and upper facial information is processed to detect the speaker’s emotions, generating a lower facial expression. In addition, the system detects the mask’s shape and enables accurate registration in the proper position. This technology can improve communication and alleviate challenges associated with communication between persons wearing face masks.