Semantic segmentation is a widespread image analysis task; in some applications, it requires such high accuracy that it still has to be done manually, taking a long time. Deep learning‐based approaches can significantly reduce such times, but current automated solutions may produce results below expert standards. We propose agLab, an interactive tool for the rapid labelling and analysis of orthoimages that speeds up semantic segmentation. TagLab follows a human‐centered artificial intelligence approach that, by integrating multiple degrees of automation, empowers human capabilities. We evaluated TagLab's efficiency in annotation time and accuracy through a user study based on a highly challenging task: the semantic segmentation of coral communities in marine ecology. In the assisted labelling of corals, TagLab increased the annotation speed by approximately 90% for nonexpert annotators while preserving the labelling accuracy. Furthermore, human–machine interaction has improved the accuracy of fully automatic predictions by about 7% on average and by 14% when the model generalizes poorly. Considering the experience done through the user study, TagLab has been improved, and preliminary investigations suggest a further significant reduction in annotation times.

Animal forests promote marine habitats morphological complexity and functioning. The red gorgonian, Paramuricea clavata, is a key structuring species of the Mediterranean coralligenous habitat and an indicator species of climate effects on habitat functioning. P. clavata metrics such as population structure, morphology and biomass inform on the overall health of coralligenous habitats, but the estimation of these metrics is time and cost consuming, and often requires destructive sampling. As a consequence, the implementation of long-term and wide-area monitoring programmes is limited. This study proposes a novel and transferable Structure from Motion (SfM) based method for the estimation of gorgonian population structure (i.e., maximal height, density, abundance), morphometries (i.e., maximal width, fan surface) and biomass (i.e., coenenchymal Dry Weight, Ash Free Dried Weight). The method includes the estimation of a novel metric (3D canopy surface) describing the gorgonian forest as a mosaic of planes generated by fitting multiple 5 cm × 5 cm facets to a SfM generated point cloud. The performance of the method is assessed for two different cameras (GoPro Hero4 and Sony NEX7). Results showed that for highly dense populations (17 colonies/m2), the SfM-method had lower accuracies in estimating the gorgonians density for both cameras (60% to 89%) than for medium to low density populations (14 and 7 colonies/m2) (71% to 100%). Results for the validation of the method showed that the correlation between ground truth and SfM estimates for maximal height, maximal width and fan surface were between R2 = 0.63 and R2 = 0.9, and R2 = 0.99 for coenenchymal surface estimation. The methodological approach was used to estimate the biomass of the gorgonian population within the study area and across the coralligenous habitat between −25 to −40 m depth in the Portofino Marine Protected Area. For that purpose, the coenenchymal surface of sampled colonies was obtained and used for the calculations. Results showed biomass values of dry weight and ash free dry weight of 220 g and 32 g for the studied area and to 365 kg and 55 Kg for the coralligenous habitat in the Marine Protected Area. This study highlighted the feasibility of the methodology for the quantification of P. clavata metrics as well as the potential of the SfM-method to improve current predictions of the status of the coralligenous habitat in the Mediterranean sea and overall management of threatened ecosystems.

Detecting the impacts of natural and anthropogenic disturbances that cause declines in organisms or changes in community composition has long been a focus of ecology. However, a tradeoff often exists between the spatial extent over which relevant data can be collected, and the resolution of those data. Recent advances in underwater photogrammetry, as well as computer vision and machine learning tools that employ artificial intelligence (AI), offer potential solutions with which to resolve this tradeoff. Here, we coupled a rigorous photogrammetric survey method with novel AI-assisted image segmentation software in order to quantify the impact of a coral bleaching event on a tropical reef, both at an ecologically meaningful spatial scale and with high spatial resolution. In addition to outlining our workflow, we highlight three key results: (1) dramatic changes in the three-dimensional surface areas of live and dead coral, as well as the ratio of live to dead colonies before and after bleaching; (2) a size-dependent pattern of mortality in bleached corals, where the largest corals were disproportionately affected, and (3) a significantly greater decline in the surface area of live coral, as revealed by our approximation of the 3D shape compared to the more standard planar area (2D) approach. The technique of photogrammetry allows us to turn 2D images into approximate 3D models in a flexible and efficient way. Increasing the resolution, accuracy, spatial extent, and efficiency with which we can quantify effects of disturbances will improve our ability to understand the ecological consequences that cascade from small to large scales, as well as allow more informed decisions to be made regarding the mitigation of undesired impacts.

The semantic segmentation of underwater imagery is an important step in the ecological analysis of coral habitats. To date, scientists produce fine-scale area annotations manually, an exceptionally time-consuming task that could be efficiently automatized by modern CNNs. This paper extends our previous work presented at the 3DUW’19 conference, outlining the workflow for the automated annotation of imagery from the first step of dataset preparation, to the last step of prediction reassembly. In particular, we propose an ecologically inspired strategy for an efficient dataset partition, an over-sampling methodology targeted on ortho-imagery, and a score fusion strategy. We also investigate the use of different loss functions in the optimization of a Deeplab V3+ model, to mitigate the class-imbalance problem and improve prediction accuracy on coral instance boundaries. The experimental results demonstrate the effectiveness of the ecologically inspired split in improving model performance, and quantify the advantages and limitations of the proposed over-sampling strategy. The extensive comparison of the loss functions gives numerous insights on the segmentation task; the Focal Tversky, typically used in the context of medical imaging (but not in remote sensing), results in the most convenient choice. By improving the accuracy of automated ortho image processing, the results presented here promise to meet the fundamental challenge of increasing the spatial and temporal scale of coral reef research, allowing researchers greater predictive ability to better manage coral reef resilience in the context of a changing environment.

Visual sampling techniques represent a valuable resource for a rapid, non-invasive data acquisition for underwater monitoring purposes. Long-term monitoring projects usually requires the collection of large quantities of data, and the visual analysis of a human expert operator remains, in this context, a very time consuming task. It has been estimated that only the 1-2% of the acquired images are later analyzed by scientists (Beijbom et al., 2012). Strategies for the automatic recognition of benthic communities are required to effectively exploit all the information contained in visual data. Supervised learning methods, the most promising classification techniques in this field, are commonly affected by two recurring issues: the wide diversity of marine organism, and the small amount of labeled data. In this work, we discuss the advantages offered by the use of annotated high resolution ortho-mosaics of seabed to classify and segment the investigated specimens, and we suggest several strategies to obtain a considerable per-pixel classification performance although the use of a reduced training dataset composed by a single ortho-mosaic. The proposed methodology can be applied to a large number of different species, making the procedure of marine organism identification an highly adaptable task.