The digitalization is transforming the very nature of factories, from automated systems to intelligent ones. In this process, industrial robots play a key role. Even if repeatability, precision and velocity of the industrial manipulators enable reaching considerable production levels, factories are required to face an increasingly competitive market, which requires being able to dynamically adapt to different situations and conditions. Hence, facilities are moving toward systems that rely on the collaboration between humans and machines. Human workers should understand the behavior of the robots, placing trust in them to properly collaborate. If a fault occurs on a manipulator, its movements are suddenly stopped for security reasons, thus workers may not be able to understand what happened to the robot. Therefore, the operators’ stress and anxiety may increase, compromising the human-robot collaborative scenario. This work fits in this context and it proposes an adaptive Augmented Reality system to display industrial robot faults by means of the Microsoft HoloLens device. Starting from the methodology employed to identify which virtual metaphors best evoke robot faults, an adaptive modality is presented to dynamically display the metaphors in positions close to the fault location, always visible from the user and not occluded by the manipulator. A comparison with a non adaptive modality is proposed to assess the effectiveness of the adaptive solution. Results show that the adaptive modality allows users to recognize faults faster and with fewer movements than the non adaptive one, thus overcoming the limitation of the narrow field-of-view of the HoloLens device.

Nowadays the market is becoming increasingly competitive, factories are required not only to enhance the product quality but also to reduce manufacturing and maintenance times. In an industrial context, modern factories are composed by many automated systems, such as industrial robots, which can perform different tasks. Although industrial robots are becoming more powerful and efficient, human workers are still required to accomplish different operations, such as training and maintenance procedures. The proposed research aims to assess a remote interaction system in an industrial training collaborative mixed-reality (CMR) environment. A remote expert user is capable of explaining a training procedure to an unskilled local user. Remote and local users interact using different interaction systems: the remote operator gives assistance using an immersive Virtual Reality (VR) device, whereas the local user interacts using a wearable Augmented Reality (AR) device. A comparison between an interaction based on the presence of a virtual human and one based on the use of abstract icons is proposed. In the first case, a virtual 3D representation of the remote technician is shown to the local user by using AR: the remote technician can pinpoint the components involved in the training procedure and the local user can visualize the instructions through some animations of the virtual avatar. In the second case, the local user cannot see a 3D representation of the remote technician; on the other hand, different 3D models, such as animated icons, are displayed to the local operator through AR depending on the component pinpointed by the remote technician in the virtual environment. Each 3D icon should suggest to the local user which component has to be manipulated at the current step of the procedure. Preliminary results suggest that the interface that requires less resources to be developed and managed should be preferred. Although in no audio condition the virtual avatar may improve the sense of presence of the remote technician, the use of abstract metaphors seems to be of primary importance to successfully complete an industrial task.

Human–robot collaboration (HRC) is a new and challenging discipline that plays a key role in Industry 4.0. Digital transformation of industrial plants aims to introduce flexible production lines able to adapt to different products quickly. In this scenario, HRC can be a booster to support flexible manufacturing, thus introducing new interaction paradigms between humans and machines. Augmented reality (AR) can convey much important information to users: for instance, information related to the status and the intention of the robot/machine the user is collaborating with. On the other hand, traditional input interfaces based on physical devices, gestures, and voice might be precluded in industrial environments. Brain–computer interfaces (BCIs) can be profitably used with AR devices to provide technicians solutions to effectively collaborate with robots. This paper introduces a novel BCI–AR user interface based on the NextMind and the Microsoft Hololens 2. Compared to traditional BCI interfaces, the NextMind provides an intuitive selection mechanism based on visual cortex signals. This interaction paradigm is exploited to guide a collaborative robotic arm for a pick and place selection task. Since the ergonomic design of the NextMind allows its use in combination with the Hololens 2, users can visualize through AR the different parts composing the artifact to be assembled, the visual elements used by the NextMind to enable the selections, and the robot status. In this way, users’ hands are always free, and the focus can be always on the objects to be assembled. Finally, user tests are performed to evaluate the proposed system, assessing both its usability and the task’s workload; preliminary results are very encouraging, and the proposed solution can be considered a starting point to design and develop affordable hybrid-augmented interfaces to foster real-time human–robot collaboration.

Global Positioning Satellite (GPS) systems sample points on the Earth’s surface with meter accuracy. Real-Time Kinematic (RTK) devices improve GPS performances by providing real-time correction data from ground stations, achieving centimeter accuracy. Reliable tracking approaches are essential for Augmented Reality (AR) applications, especially for outdoor scenarios, which still present unsolved challenges. AR handheld tracking capabilities have been greatly improved by integrating visual tracking approaches with RTK devices, whereas little is known about combining wearable AR interfaces with RTK systems. Although wearable AR devices are intrinsically designed for AR applications, their performance dramatically reduces in large outdoor areas, comprising the user experience. Hence, this paper provides a rigorous evaluation of a small-size RTK device that does not need any additional software integration to collect positional data. The main goal of the assessment is to verify whether its integration with a wearable AR device is advantageous or not. The evaluation has been performed considering both static and dynamic scenarios in open-sky and urban areas. The results show that the RTK device can achieve 1 cm accuracy when used in open-sky areas. In contrast, its accuracy dramatically reduces in the proximity of buildings and obstacles, showing average errors ranging from 76 to 2561%. Since wearable AR devices have an average accuracy of 2 cm, the outcomes indicate that RTK devices should be combined with wearable AR devices only when the RTK device is far from obstacles. On the contrary, the positional data should be completely avoided when barriers surround the RTK device.