21 February 2022 in Wireless Communications and Mobile Computing
Augmented reality (AR) technology, with its unique immersive interactive experience of virtual reality integration, can show children abstract concepts and theories in an intuitive way. In this paper, we improve the high-precision position and low delay interaction of AR recognition area through edge calculation, so as to enhance the immersion of children’s chord knowledge learning. Meanwhile, multisource fusion perception of audio and touch is used, and the AI algorithm model is used to monitor physical events (cloud AI training and edge execution) and effectively realize intelligent services such as chord combination, melody perception, and control. Chord knowledge in music theory is presented to children through multichannels (kinesthetic, visual, auditory, etc.) situationally, to improve children’s learning efficiency and experience. We invited 12 children to participate in user experiments to test usability of ChordAR. The test results showed that children can master the method of playing the game through simple learning, ultimately correctly imitating C, G, and E chords, and experience full immersion in the game. Finally, we discuss the positive effects of ChordAR on children’s learning and creativity and make suggestions for future AR games.
14 April 2023 in Proceedings of the ACM on Human-Computer Interaction
Traditional wall-sized displays mostly only support side-by-side co-located collaboration, while transparent displays naturally support face-to-face interaction. Many previous works assume transparent displays support collaboration. Yet it is unknown how exactly its afforded face-to-face interaction can support loose or close collaboration, especially compared to the side-by-side configuration offered by traditional large displays. In this paper, we used an established experimental task that operationalizes different collaboration coupling and layout locality, to compare pairs of participants collaborating side-by-side versus face-to-face in each collaborative situation. We compared quantitative measures and collected interview and observation data to further illustrate and explain our observed user behavior patterns. The results showed that the unique face-to-face collaboration brought by transparent display can result in more efficient task performance, different territorial behavior, and both positive and negative collaborative factors. Our findings provided empirical understanding about the collaborative experience supported by wall-sized transparent displays and shed light on its future design.