Research on Augmented Reality (AR)/Mixed Reality (MR) remote collaboration for physical tasks remains a compelling and dynamic area of study. AR systems have been developed which transmit virtual annotations between remote collaborators, but there has been little research on how haptic feedback can also be shared. In this paper, we present a Virtual Reality (VR)-Spatial Augmented Reality (SAR) remote collaborative system that provides haptic feedback with tangible interaction between a local worker and a remote expert helper. Using this system, we conducted a within-subject user study to compare two interfaces for remote collaboration between a local worker and expert helper, one with mid-air free drawing (MFD) and one with tangible physical drawing (TPD). The results showed that there were no significant differences with respect to performance time and operation errors. However, users felt that the TPD interface supporting passive haptic feedback could significantly improve the remote experts’ user experience in VR. Our research provides useful information on the way for gesture- and gaze-based multimodal interaction supporting haptic feedback in AR/MR remote collaboration on physical tasks.

This paper investigates the effect of using augmented reality (AR) annotations and two different gaze visualizations, head pointer (HP) and eye gaze (EG), in an AR system for remote collaboration on physical tasks. First, we developed a spatial AR remote collaboration platform that supports sharing the remote expert’s HP or EG cues. Then the prototype system was evaluated with a user study comparing three conditions for sharing non-verbal cues: (1) a cursor pointer (CP), (2) HP and (3) EG with respect to task performance, workload assessment and user experience. We found that there was a clear difference between these three conditions in the performance time but no significant difference between the HP and EG conditions. When considering the perceived collaboration quality, the HP/EG interface was statistically significantly higher than the CP interface, but there was no significant difference for workload assessment between these three conditions. We used low-cost head tracking for the HP cue and found that this served as an effective referential pointer. This implies that in some circumstances, HP could be a good proxy for EG in remote collaboration. Head pointing is more accessible and cheaper to use than more expensive eye-tracking hardware and paves the way for multi-modal interaction based on HP and gesture in AR remote collaboration.