Images taken under water usually suffer from the problems of quality degradation, such as low contrast, blurring details, color deviations, non-uniform illumination, etc. As an important problem in image processing and computer vision, the restoration and enhancement of underwater image are necessary for numerous practical applications. Over the last few decades, underwater image restoration and enhancement have been attracting an increasing amount of research effort. However, a comprehensive and in-depth survey of related achievements and improvements is still missing, especially the survey of underwater image dataset which is a key issue in underwater image processing and intelligent application. In this exposition, we first summarize more than 120 studies about the latest progress in underwater image restoration and enhancement, including the techniques, datasets, available codes, and evaluation metrics. We analyze the contributions and limitations of existing methods to facilitate the comprehensive understanding of underwater image restoration and enhancement. Furthermore, we provide detailed objective evaluations and analysis of the representative methods on five types of underwater scenarios, which verifies the applicability of these methods in different underwater conditions. Finally, we discuss the potential challenges and open issues of underwater image restoration and enhancement and suggest possible research directions in the future.

To distinguish underwater images from natural images is one of the challenge of collecting and generation of underwater image data. Common image classification and recognition models classify the objects in an image depending on the saliency while suppressing the background. In this article, an inception–attention network (IA-Net), a convolutional neural network (CNN)-based model to classify the underwater images from natural images is reported, in which an inception–attention (I-A) module is constructed to simulate the visual correlation mechanism of classifying images taken from special environments such as fog, nighttime and under water. It is illustrated that the context background is as important as the salient object when understanding the underwater images. We executed experiments on a data set, which consists of 4000 underwater images and 5000 nonunderwater images, and demonstrate that the proposed IA-Net achieves an accuracy of 99.3 $\%$ on underwater image classification, which is significantly better than classical image classification networks, such as AlexNet, InceptionV3, and ResNet. In addition, the comparative experiments prove that the IA-Net is superior to other networks when distinguishing underwater images from foggy, nighttime images and fish images taken in nonunderwater environments, although these images have indistinguishable characteristics with underwater images. Moreover, we demonstrate the I-A structure we proposed can be used to boost the performance of the existing object recognition networks. By substituting the inception module with the I-A module, the Inception-ResNetV2 network achieves a 10.7 $\%$ top-1 error rate on the subset of ILSVRC-2012, which further illustrates the effectiveness of the correlation between the image background and subjective perception in improving the performance of the visual analysis tasks.

A new restoration model for underwater images is presented, based on the dark channel reflection-illumination decomposition and local backscattering lighting estimation, to promote the clarity of edge detail and the colorfulness of the output image. For turbid offshore environments, a novel underwater image restoration method is further introduced by applying a statistical prior to the offshore attenuation coefficients. It is shown that the estimated transmission map with lighting-reflection decomposition, rather than dividing the dark channel by the maximal backscattering light as the other methods do, provides better clarity and color restoration. Detailed qualitative and quantitative analysis on hundreds of underwater images is performed, which demonstrates that the proposed method outperforms the state-of-the-art algorithms on the images taken in offshore water characterized by a heavy concentration of colored dissolved organic matter and total suspended matter, and is suitable for fast underwater processing.

Generative adversarial network (GAN)-based underwater image enhancement methods improve image quality by encoding a raw image and an adversarial image with the corresponding high-quality version. Although these methods have recently attracted significant attention, the lack of referred clear underwater imagery forces GAN-based underwater image enhancement models to be trained with synthetic or enhanced underwater images, limiting their applicability and performance. This article proposes a novel self-adversarial GAN (SA-GAN) to enhance underwater images by referring to paired raw and high-quality natural images. Specifically, a self-adversarial mode is designed that attaches a further constraint to the generation procedure. By applying two pairwise image quality discriminators, the generators are supervised with a stronger decision boundary to generate better quality than the high-quality natural image and the last-generated image. This is a novel settlement of the limitation caused by the adversary system with the synthetic or enhanced underwater images, realizing the quality transfer from natural images to distorted underwater images. Several experiments on real underwater images and two commonly used underwater image data sets demonstrate that the proposed method subjectively and objectively performs better than current methods in restoring the coloration of underwater images.

The complex underwater environment usually leads to the problem of quality degradation in underwater images, and the distortion of sharpness and color are the main factors to the quality of underwater images. The paper discloses an underwater sequence image dataset called TankImage-I with gradually changing sharpness and color distortion collected in a pool. TankImage-I contains two plane targets, a total of 78 images. It includes two lighting conditions and three different water transparency. The imaging distance is also changed during the photographing process. The paper introduces the relevant details of the photographing process, and provides the measurement results of the sharpness and color distortion of the sequence images. In addition, we verify the performance of 14 image quality assessment methods on TankImage-I, and analyze the results of 14 image quality assessment methods from the aspects of sharpness and color, which provides a reference for the design and improvement of underwater image quality assessment algorithm and underwater imaging system design.