This paper presents a comprehensive overview of current deep-learning methods for automatic object classification of underwater sonar data for shoreline surveillance, concentrating mostly on the classification of vessels from passive sonar data and the identification of objects of interest from active sonar (such as minelike objects, human figures or debris of wrecked ships). Not only is the contribution of this work to provide a systematic description of the state of the art of this field, but also to identify five main ingredients in its current development: the application of deep-learning methods using convolutional layers alone; deep-learning methods that apply biologically inspired feature-extraction filters as a preprocessing step; classification of data from frequency and time–frequency analysis; methods using machine learning to extract features from original signals; and transfer learning methods. This paper also describes some of the most important datasets cited in the literature and discusses data-augmentation techniques. The latter are used for coping with the scarcity of annotated sonar datasets from real maritime missions.

The illegal exploitation of protected marine environments has consistently threatened the biodiversity and economic development of coastal regions. Extensive monitoring in these – often remote – areas is challenging. Machine learning methods are useful in object detection and classification tasks and have the potential to underpin techniques for the development of robust monitoring systems to overcome this problem. However, development is hindered due to the limited number of publicly available labelled and curated datasets. Furthermore, there are relatively few open-source state-of-the-art methods to be used for evaluation. This paper presents an investigation of automated classification methods using underwater acoustic signals to infer the presence and type of vessels navigating in coastal regions. Various combinations of deep convolutional neural network architectures, and preprocessing filter layers, were evaluated using a new dataset based on a subset of the extensive open-source Ocean Networks Canada hydrophone data. Tests were conducted in which VGGNet and ResNet networks were applied to classify the input data. The data was preprocessed using either Constant Q Transform (CQT), Gammatone, Mel spectrogram, or a combination of these filters. With over 97% accuracy, using all three preprocessing representations simultaneously yielded the most reliable result. However, high accuracies of 94.95% were achieved using CQT as the preprocessing filter for a ResNet-based convolutional neural network, providing a trade-off between model complexity and accuracy; a result that is more than 10% higher than previously reported approaches. This more accurate classifier for underwater acoustics could be used as a reliable autonomous monitoring system in maritime environments.

Computer vision algorithms that make use of optical flow are constantly increasing in complexity, especially in the context of elaborated algorithms that are heavily inspired by biology. To develop upon and utilise these algorithms for realworld tasks, their extensive parameter sets need to be tuned. Due to algorithmic complexity, and non-linearities present throughout their parameter set, this is no small task. Using an adaptive genetic algorithm, which itself is biologically-inspired, we look at the performance and behaviour of the tuning when significant changes have been made to a low speed rotational velocity optical flow estimation algorithm. We validate that previously reported changes to the optical flow estimator yielded a fitness increase of over 30% when compared to the baseline model the changes were made to, and a 15% increase once that baseline model was also tuned. This improvement would be extremely unlikely without the aid of evolutionary computation algorithms. This shows that even an extremely complex computer vision algorithm with many parameters, 36 in this case, can be tuned to an operating point, facilitating the continued development work on the vision algorithm by allowing for the validation and quantification of algorithmic changes.

Vision is a potent source of information, not just for humans, but for robots as well. Processing visual information is a computationally expensive task, one that is often difficult to accomplish in real-time on embedded hardware. In the broad field of visual research exists egomotion estimation, the process of determining self-motion from optical flow. Here we show a technological adaptation and implementation of a complex, biologically-inspired egomotion estimator that has previously only been simulated. Through efficient biologically-inspired signal conditioning and motion estimation techniques, rotational velocity estimations have been achieved at 100 frames per second on embedded hardware within an operating range of 2.7°/s to 72.1°/s with comparable accuracy to traditional sensors. These findings open the possibilities for egomotion estimation on embedded platforms which could be used to complement existing rotational velocity estimators under complex environmental conditions.

The visual systems of insects are relatively simple compared to humans. However, they enable navigation through complex environments where insects perform exceptional levels of obstacle avoidance. Biology uses two separable modes of optic flow to achieve this: rapid gaze fixation (rotational motion known as saccades); and the inter-saccadic translational motion. While the fundamental process of insect optic flow has been known since the 1950’s, so too has its dependence on contrast. The surrounding visual pathways used to overcome environmental dependencies are less well known. Previous work has shown promise for low-speed rotational motion estimation, but a gap remained in the estimation of translational motion, in particular the estimation of the time to impact. To consistently estimate the time to impact during inter-saccadic translatory motion, the fundamental limitation of contrast dependence must be overcome. By adapting an elaborated rotational velocity estimator from literature to work for translational motion, this paper proposes a novel algorithm for overcoming the contrast dependence of time to impact estimation using nonlinear spatio-temporal feedforward filtering. By applying bioinspired processes, approximately 15 points per decade of statistical discrimination were achieved when estimating the time to impact to a target across 360 background, distance, and velocity combinations: a 17-fold increase over the fundamental process. These results show the contrast dependence of time to impact estimation can be overcome in a biologically plausible manner. This, combined with previous results for low-speed rotational motion estimation, allows for contrast invariant computational models designed on the principles found in the biological visual system, paving the way for future visually guided systems.