With the evolution of machine learning and deep learning, more and more researchers have utilized these methods in the field of underwater acoustic target recognition. In these studies, convolutional neural networks (CNNs) are the main components of recognition models. In recent years, a neural network model Transformer that uses a self-attention mechanism was proposed and achieved good performance in deep learning. In this paper, we propose a Transformer-based underwater acoustic target recognition model STM. To the best of our knowledge, this is the first work to introduce Transformer into the underwater acoustic field. We compared the performance of STM with CNN, ResNet18, and other multi-class algorithm models. Experimental results illustrate that under two commonly used dataset partitioning methods, STM achieves 97.7% and 89.9% recognition accuracy, respectively, which are 13.7% and 50% higher than the CNN Model. STM also outperforms the state-of-the-art model CRNN-9 by 3.1% and ResNet18 by 1.8%.

Whale vocal calls contain valuable information and abundant characteristics that are important for classification of whale sub-populations and related biological research. In this study, an effective data-driven approach based on pre-trained Convolutional Neural Networks (CNN) using multi-scale waveforms and time-frequency feature representations is developed in order to perform the classification of whale calls from a large open-source dataset recorded by sensors carried by whales. Specifically, the classification is carried out through a transfer learning approach by using pre-trained state-of-the-art CNN models in the field of computer vision. 1D raw waveforms and 2D log-mel features of the whale-call data are respectively used as the input of CNN models. For raw waveform input, windows are applied to capture multiple sketches of a whale-call clip at different time scales and stack the features from different sketches for classification. When using the log-mel features, the delta and delta-delta features are also calculated to produce a 3-channel feature representation for analysis. In the training, a 4-fold cross-validation technique is employed to reduce the overfitting effect, while the Mix-up technique is also applied to implement data augmentation in order to further improve the system performance. The results show that the proposed method can improve the accuracies by more than 20% in percentage for the classification into 16 whale pods compared with the baseline method using groups of 2D shape descriptors of spectrograms and the Fisher discriminant scores on the same dataset. Moreover, it is shown that classifications based on log-mel features have higher accuracies than those based directly on raw waveforms. The phylogeny graph is also produced to significantly illustrate the relationships among the whale sub-populations.