High-quality video observations are very much needed in underwater environments for the monitoring of several ecosystem indicators and to support the sustainable development and management of almost all activities in the ocean. Reliable video observations are however challenging to collect, because of the generally poor visibility conditions and the difficulties to deploy cost-effective sensors and platforms in the marine environment. Visibility in water is regulated by natural light availability at different depths, and by the presence of suspended particles, scattering incident light in all directions. Those elements are also largely variable in time and space, making it difficult to identify technological solutions that can be used in all conditions. By combining state-of-the-art “time of flight” (ToF) image sensors and innovative pulsed laser illumination, we have developed a range-gated camera system (UTOFIA) that enables affordable and enhanced 3D underwater imaging at high resolution. This range-gated solution allows users to eliminate close-range backscattering, improving quality of the images and providing information on the distance of each illuminated object, hence giving access to real-time 3D measurements. Furthermore, as the system is based on pulsed laser light, it is almost independent of natural light conditions and can achieve similar performances at an extended depth range. We use this system to collect observations in different oceanographic conditions and for different applications, including aquaculture monitoring, seafloor mapping, litter identifications and structure inspection. Performances are evaluated by comparing images to regular cameras and by using standard targets to assess accuracy and precision of distance measurements. We suggest that this type of technology can become a standard in underwater 3D imaging to support the future development of the ocean economy.

We present a range-gated camera system designed for real-time (10 Hz) 3D estimation underwater. The system uses a fast-shutter CMOS sensor ($1280\times 1024$) customized to facilitate gating with 1.67 ns (18.8 cm in water) delay steps relative to the triggering of a solid-state actively $Q$-switched 532 nm laser. A depth estimation algorithm has been carefully designed to handle the effects of light scattering in water, i.e., forward and backward scattering. The raw range-gated signal is carefully filtered to reduce noise while preserving the signal even in the presence of unwanted backscatter. The resulting signal is proportional to the number of photons that are reflected during a small time unit (range), and objects will show up as peaks in the filtered signal. We present a peak-finding algorithm that is robust to unwanted forward scatter peaks and at the same time can pick out distant peaks that are barely higher than peaks caused by sensor and intensity noise. Super-resolution is achieved by fitting a parabola around the peak, which we show can provide depth precision below 1 cm at high signal levels. We show depth estimation results when scanning a range of 8 m (typically 1–9 m) at 10 Hz. The results are dependent on the water quality. We are capable of estimating depth at distances of over 4.5 attenuation lengths when imaging high albedo targets at low attenuation lengths, and we achieve a depth resolution $(\sigma )$ ranging from 0.8 to 9 cm, depending on signal level.

High-precision underwater 3D cameras are required to automate many of the traditional subsea inspection, maintenance and repair (IMR) operations. In this paper we introduce a novel multi-frequency phase stepping (structured light) method for high-precision 3D estimation even in turbid water. We introduce an adaptive phase-unwrapping procedure which uses the phase-uncertainty to determine the highest frequency that can be reliably unwrapped. Light scattering adversely affects the phase estimate. We propose to remove the effect of forward scatter with an unsharp filter and a model-based method to remove the backscatter effect. Tests in varying turbidity show that the scatter correction removes the adverse effect of scatter on the phase estimates. The adaptive frequency unwrapping with scatter correction results in images with higher accuracy and precision and less phase unwrap errors than the Gray-Code Phase Stepping (GCPS) approach.

Manipulation tasks on subsea instalments require extremely precise detection and localization of objects of interest. This problem is referred to as “pose estimation”. In this work, we present a framework for detecting and predicting 6DoF pose for relevant objects (fish-tail, gauges, and valves) on a subsea panel under varying water turbidity. A deep learning model that takes 3D vision data as an input is developed, providing a more robust 6D pose estimate. Compared to the 2D vision deep learning model, the proposed method reduces rotation and translation prediction error by ($-\Delta {0.39}^{\circ }$) and translation ($-\Delta 6.5$ mm), respectively, in high turbid waters. The proposed approach is able to provide object detection as well as 6D pose estimation with an average precision of 91%. The 6D pose estimation results show ${2.59}^{\circ }$ and $6.49$ cm total average deviation in rotation and translation as compared to the ground truth data on varying unseen turbidity levels. Furthermore, our approach runs at over 16 frames per second and does not require pose refinement steps. Finally, to facilitate the training of such model we also collected and automatically annotated a new underwater 6D pose estimation dataset spanning seven levels of turbidity.