Underwater images are often acquired in sub-optimal lighting conditions, in particular at profound depths where the absence of natural light demands the use of artificial lighting. Low-lighting images impose a challenge for both manual and automated analysis, since regions of interest can have low visibility. A new framework capable of significantly enhancing these images is proposed in this article. The framework is based on a novel dehazing mechanism that considers local contrast information in the input images, and offers a solution to three common disadvantages of current single image dehazing methods: oversaturation of radiance, lack of scale-invariance and creation of halos. A novel low-lighting underwater image dataset, OceanDark, is introduced to assist in the development and evaluation of the proposed framework. Experimental results and a comparison with other underwater-specific image enhancement methods show that the proposed framework can be used for significantly improving the visibility in low-lighting underwater images of different scales, without creating undesired dehazing artifacts.
20 December 2022 in Science of The Total Environment
An increasing number of marine conservation initiatives rely on data from Automatic Identification System (AIS) to inform marine vessel traffic associated impact assessments and mitigation policy. However, a considerable proportion of vessel traffic is not captured by AIS in many regions of the world. Here we introduce two complementary techniques for collecting traffic data in the Canadian Salish Sea that rely on optical imagery. Vessel data pulled from imagery captured using a shore-based autonomous camera system ("Photobot") were used for temporal analyses, and data from imagery collected by the National Aerial Surveillance Program (NASP) were used for spatial analyses. The photobot imagery captured vessel passages through Boundary Pass every minute (Jan-Dec 2017), and NASP data collection occurred opportunistically across most of the Canadian Salish Sea (2017-2018). Based on photobot imagery data, we found that up to 72 % of total vessel passages through Boundary Pass were not broadcasting AIS, and in some vessel categories this proportion was much higher (i.e., 96 %). We fit negative binomial General Linearized Models to our photobot data and found a strong seasonal variation in non-AIS, and a weekend/weekday component that also varied by season (interaction term p < 0.0001). Non-AIS traffic was much higher during the summer (Apr-Sep) and during the weekend (Sat-Sun), reflecting patterns in recreational vessel traffic not obligated to broadcast AIS. Negative binomial General Additive Models based on the NASP data revealed strong spatial associations with distance from shore (up to 10 km) and non-AIS vessel traffic for both summer and winter seasons. There were also associations between non-AIS vessels and marina and anchorage densities, particularly during the winter, which again reflect seasonal recreational vessel traffic patterns. Overall, our GAMs explained 20-37 % of all vessel traffic during the summer and winter, and highlighted subregions where vessel traffic is under represented by AIS.
7 July 2023 in IEEE Journal of Oceanic Engineering
Echosounders are used by fisheries and ocean observatories, but significant manual effort is required to classify species of interest within multifrequency echograms. This article investigates the use of modified U-Net convolutional neural networks for the pixel-level classification of biological and physical data in echogram images with accurate classification of herring and salmon schools, bubbles, and the sea surface. Data were collected on the coast of British Columbia, Canada, over two years using an Acoustic Zooplankton and Fish Profiler at four frequencies (67, 125, 200, 455 kHz). In addition, simulated data (water depth and solar elevation angle) provide spatial and temporal context to improve the quality of predictions. Redundancy is built into the model by using a tiling strategy during training and classification. During training, using a limited set of annotated data, translational augmentation encodes the U-Nets with robust features that enable applications for alternate deployment configurations (lower sampling rates or alternate water depths). To ensure broad applicability, these networks were trained to classify echograms with noise left intact. The best-performing model classifies herring, salmon, and bubble classes with $\rm {F_{1}}$ scores of 93.0%, 87.3%, and 86.5%, respectively. The results are accurate even when multiple classes are in close proximity, thus, retaining biological data that would otherwise be discarded due to surface bubble noise.
17 May 2023 in IEEE Journal of Oceanic Engineering
This paper addresses the detection of finfish and krill in echograms. Finfish, in particular Pacific hake, are used both as human food and fish meal. Krill, harvested for aquaculture and aquariums, are a primary food source for finfish, including hake. Thus, spatial distributions of hake follow that of krill. Stock assessments need an accurate differentiation of krill from finfish (hake) in acoustic echograms. This paper proposes a semantic segmentation paradigm for the pixel-level classification of multi-frequency information to detect co-occurring finfish and krill. This paradigm is highly relevant for identifying cloud-like, diffuse krill aggregations that are intertwined with small, often sparse and sometimes dense schools of finfish. We propose U-MSAA-Net, a deep learning U-Net-like framework with novel multi-scale additive attention (MSAA) modules. MSAA modules allow us to leverage all contextual and local information from feature maps available at any given level of the decoding phase of the network, yielding an efficient suppression of the feature responses from regions with lesser semantic value. Experimental results on a new finfish and krill data set spanning across nine months of acoustic data and covering various situations show that U-MSAA-Net outperforms both traditional, texture-based machine learning methods, and deep learning methods based on state-of-the-art semantic segmentation networks. Additional experiments on a data set containing schools of herring and salmon confirm the versatility of U-MSAA-Net and its superiority in terms of accuracy and ability to detect schools of varying sizes. U-MSAA-Net is the first step in creating a comprehensive tool for stock and ecosystem assessments.
Sablefish are a commercially important species in the northeast Pacific Ocean, typically caught using longline traps in a tightly regulated fishery. Mandatory electronic monitoring (EM) systems record deck activities to support compliance, but reviewing footage for trap hauling remains a manual, time-intensive process. This paper introduces TrapCounter, a method that automatically counts active fishing traps in EM videos by combining deep learning-based detection, tracking, and optical flow. We integrate the RetinaNet object detection architecture with the StrongSORT tracker to properly detect and track active traps as they are being hauled, addressing such challenges as varied poses and appearance due to the semi-transparent nature of traps and crowded deck environments. To reduce overcounting, we add an optical flow-based counting module that improves the tracker’s active trap count accuracy. We also release DTCFT, the first public dataset for trap detection, tracking, and counting, built from actual commercial and survey fishing vessel footage captured under varying conditions. Experiments show that TrapCounter outperforms approaches based on YOLOv9, Faster R-CNN, DeepSORT, and ByteTrack, streamlining EM workflows and enhancing EM scalability in commercial and survey fisheries. Dataset: https://github.com/alisoltani7596/TrapCounter