Personalized production is moving the progress of industrial automation forward, and demanding new tools for improving the decision-making of the operators. This paper presents a new, projection-based augmented reality system for assisting operators during electronic component assembly processes. The paper describes both the hardware and software solutions, and depicts the results obtained during a usability test with the new system.

The nature of industrial manufacturing processes and the continuous need to adapt production systems to new demands require tools to support workers during transitions to new processes. At the early stage of transitions, human error rate is often high and the impact in quality and production loss can be significant. Over the past years, eXtended Reality (XR) technologies (such as virtual, augmented, immersive, and mixed reality) have become a popular approach to enhance operators’ capabilities in the Industry 4.0 paradigm. The purpose of this research is to explore the usability of dialogue-based XR enhancement to ease the cognitive burden associated with manufacturing tasks, through the augmentation of linked multi-modal information available to support operators. The proposed Interactive XR architecture, using the Spoken Dialogue Systems’ modular and user-centred architecture as a basis, was tested in two use case scenarios: the maintenance of a robotic gripper and as a shop-floor assistant for electric panel assembly. In both cases, we have confirmed a high user acceptance rate with an efficient knowledge communication and distribution even for operators without prior experience or with cognitive impairments, therefore demonstrating the suitability of the solution for assisting human workers in industrial manufacturing processes. The results endorse an initial validation of the Interactive XR architecture to achieve a multi-device and user-friendly experience to solve industrial processes, which is flexible enough to encompass multiple tasks.

One of the most challenging problems in robot-assisted surgical systems is to provide surgical realism at interactive simulation rates. The proposed visual tracking system can track and register object deformations in real time using a physically based formulation, despite the occlusions produced by the robotic system itself. The results obtained provide an accurate visual representation of the deformed solid and will thus enable new assistance approaches to help surgeons during surgical procedures.

This paper describes a method for calibrating multi camera and multi laser 3D triangulation systems, particularly for those using Scheimpflug adapters. Under this configuration, the focus plane of the camera is located at the laser plane, making it difficult to use traditional calibration methods, such as chessboard pattern-based strategies. Our method uses a conical calibration object whose intersections with the laser planes generate stepped line patterns that can be used to calculate the camera-laser homographies. The calibration object has been designed to calibrate scanners for revolving surfaces, but it can be easily extended to linear setups. The experiments carried out show that the proposed system has a precision of 0.1 mm.

In the recent years, due to the technological advancements brought by Industry 4.0, flexible in-line quality control processes that analyze 100% of produced parts targeting Zero Defect Manufacturing have become a reality. The electronics industry in general and the manufacturing of PCBAs in particular have also adopted this paradigm, where quality inspection applies especially to the most critical and error-prone feature: the solder joints between PCBs and components. When the literature related to AI-based solder joint quality inspection is analyzed, we find a lack of applications in real production lines, since most of them are limited to lab scenarios. Therefore, we present a deep learning-based method for solder joint defect detection, which is deployed in a real manufacturing line. The mentioned system is trained using the database we captured using an ad hoc robotic system with onboard front and lateral cameras designed also in this work, which we also describe in detail. The deployment of the defect detector in the real production line is done using that same robotic system too, ensuring the coherence and compatibility between the training data and the real inference scenario.