Visual technologies have an indispensable role in safety-critical applications, where tasks must often be performed through teleoperation. Due to the lack of stereoscopic and motion parallax depth cues in conventional images, alignment tasks pose a significant challenge to remote operation. In this context, machine vision can provide mission-critical information to augment the operator’s perception. In this paper, we propose a retro-reflector marker-based teleoperation aid to be used in hostile remote handling environments. The system computes the remote manipulator’s position with respect to the target using a set of one or two low-resolution cameras attached to its wrist. We develop an end-to-end pipeline of calibration, marker detection, and pose estimation, and extensively study the performance of the overall system. The results demonstrate that we have successfully engineered a retro-reflective marker from materials that can withstand the extreme temperature and radiation levels of the environment. Furthermore, we demonstrate that the proposed maker-based approach provides robust and reliable estimates and significantly outperforms a previous stereo-matching-based approach, even with a single camera.

Calibration of multiple cameras is a critical step in most vision enhancement systems. Target-based calibration approaches are known to provide accurate and stable results. However, they require manually performed capture procedures. This paper presents a generalization of a widely used single-camera target-based calibration algorithm to the case of n cameras. In order to obtain fully repeatable results, we propose the elimination of the manual capture step using a programmable robotic arm. Furthermore, we investigate the use of the position feedback provided by the robot. This is done specifically for the case of calibrating cameras without assumptions on their positions and overlapping of their fields of view. Results show that automatically captured images provide more accurate calibration results than the classical approach. Additionally, calibration of fully non-overlapping setups is made possible through our approach.

3D imaging is important for enabling autonomous operation of smart mobile work machines. Different sensing techniques have different characteristics, which affect the choice of sensors for each application. We study the depth resolution of three different active imaging technologies and stereoscopic depth estimation with multiple different baselines. We test the effect of depth measurement abilities in measuring average depth, angles between planar structures and diameter from logs. Active sensors show their robustness, while stereoscopic depth estimation follows the error behavior expected from the theory. However, in practice we find that the effect of poor depth resolution can be significantly reduced by averaging over multiple measurements from different points and using more sophisticated stereo processing.

ITER's working environment is characterized by extreme conditions, that deem maintenance and inspection tasks to be carried out through remote handling. 3D Node is a hardware/software module that extracts critical information from the remote environment during fine alignment tasks using an eye-in-hand camera system and updates the models behind the virtual reality-based remote handling platform. In this work we develop a retro-reflective marker-based version of 3D Node that estimates the pose of a planar target, the knuckle of the cassette locking system, using the markers attached to its surface. We demonstrate a pin-tool insertion task using these methods. Results show that our approach works reliably with a single low-resolution camera and outperforms the previously researched stereo depth estimation based approaches. We conclude that retro-reflective marker-based tracking has the potential to be a key enabler for remote handling operations in ITER.