Recent advances in deep learning, including the development of AlexNet, Residual Network (ResNet), and transfer learning, offer unprecedented classification accuracy in the field of machine vision. A developing application of deep learning is the automated identification and management of aquatic invasive plants. Classification of submersed aquatic vegetation (SAV) presents a unique challenge, namely, the lack of a single source of sensor data that can produce robust, interpretable images across a variable range of depth, turbidity, and lighting conditions. This paper focuses on the development of a multi-sensor (RGB and hydroacoustic) classification system for SAV that is robust to environmental conditions and combines the strengths of each sensing modality. The detection of invasive Hydrilla verticillata (hydrilla) is the primary goal. Over 5000 aerial RGB and hydroacoustic images were generated from two Florida lakes via an unmanned aerial vehicle and boat-mounted sonar unit, and tagged for neural network training and evaluation. Classes included “HYDR”, containing hydrilla; “NONE”, lacking SAV, and “OTHER”, containing SAV other than hydrilla. Using a transfer learning approach, deep neural networks with the ResNet architecture were individually trained on the RGB and hydroacoustic datasets. Multiple data fusion methodologies were evaluated to ensemble the outputs of these neural networks for optimal classification accuracy. A method incorporating logic and a Monte Carlo dropout approach yielded the best overall classification accuracy (84%), with recall and precision of 84.5% and 77.5%, respectively, for the hydrilla class. The training and ensembling approaches were repeated for a DenseNet model with identical training and testing datasets. The overall classification accuracy was similar between the ResNet and DenseNet models when averaged across all approaches (1.9% higher accuracy for the ResNet vs. the DenseNet).

Timely, accurate maps of invasive plant species are critical for making appropriate management decisions to eliminate emerging target populations or contain infestations. High-resolution aerial imagery is routinely used to map, monitor, and detect invasive plant populations. While conventional image interpretation involving human analysts is straightforward, it can require high demands for time and resources to produce useful intelligence. We compared the performance of human analysts with a custom Retinanet-based deep convolutional neural network (DNN) for detecting individual miconia (Miconia calvescens DC) plants, using high-resolution unmanned aerial system (UAS) imagery collected over lowland tropical forests in Hawai’i. Human analysts (n = 38) examined imagery at three linear scrolling speeds (100, 200 and 300 px/s), achieving miconia detection recalls of 74 ± 3%, 60 ± 3%, and 50 ± 3%, respectively. The DNN achieved 83 ± 3% recall and completed the image analysis in 1% of the time of the fastest scrolling speed tested. Human analysts could discriminate large miconia leaf clusters better than isolated individual leaves, while the DNN detection efficacy was independent of leaf cluster size. Optically, the contrast in the red and green color channels and all three (i.e., red, green, and blue) signal to clutter ratios (SCR) were significant factors for human detection, while only the red channel contrast, and the red and green SCRs were significant factors for the DNN. A linear cost analysis estimated the operational use of a DNN to be more cost effective than human photo interpretation when the cumulative search area exceeds a minimum area. For invasive species like miconia, which can stochastically spread propagules across thousands of ha, the DNN provides a more efficient option for detecting incipient, immature miconia across large expanses of forested canopy. Increasing operational capacity for large-scale surveillance with a DNN-based image analysis workflow can provide more rapid comprehension of invasive plant abundance and distribution in forested watersheds and may become strategically vital to containing these invasions.

Miconia is a highly invasive plant species with incipient plants occupying remote areas of Hawaiian watersheds. Management of these incipient plants is integral to current containment strategies. Herbicide Ballistic Technology (HBT) has been used for 8 years from helicopters as a precision approach to target individual plants. We have developed a prototype HBT applicator integrated onto an unmanned aircraft system, HBT-UAS, which offers the same precision approach with a semi-automated flight plan. Inclusion of the HBT payload resulted in statistically significant deviations from programmed flight plans compared to the unencumbered UAS, but the effect size was lower than that observed for different stages of flight. The additional payload of the HBT-UAS resulted in a large reduction in available flight time resulting a limited range of 22 m. The projectile spread of the HBT-UAS, within a 2–10 m range, had a maximum CEP of 1.87–5.58 cm. The most substantial limitation of the current prototype HBT-UAS is the available flight time. The use of larger capacity UAS and potential for beyond visual line of sight operations would result in a substantial improvement in the serviceable area and utility of the HBT-UAS for containment of invasive plants.