Side-scan sonar (SSS) has become an important tool for ocean exploration due to its practicality and reliability. Existing approaches for SSS image classification mainly rely on deep convolutional neural networks (DCNNs). Although effective, DCNN-based methods fail to classify SSS images with specific target samples that are completely absent from the training data. In this work, a global context external-attention network (GCEANet) is proposed to produce pseudo-SSS images corresponding to the absent categories for zero-shot SSS image classification. First, the optical and acoustic images are taken as content and style image inputs to the network, respectively. Second, a global context (GC) block is proposed to extract the context semantics of the style features and fuse them with the content features. Finally, an external attention (EA) block is proposed to extract the potential correlation between content and style features to generate pseudo-SSS images with optical content. In addition, a contrastive loss based on image reconstruction is proposed to optimize the cross-modal feature space fusion. Extensive experiments demonstrate the effectiveness of our approach in efficiently generating realistic pseudo-SSS samples. The experimental results on real SSS sample datasets show that our method outperforms other state-of-the-art (SOTA) methods in the field of zero-shot SSS image classification. The code is available at https://github.com/baizhongyu/GCEANet.

Harmonic Drives (HDs) are key components of industrial robots, and damages or failures of the HDs can lead to robot paralysis or even cause operational accidents. To prevent this from happening, it is of great importance to design proper algorithms for fault diagnosis of the HDs, especially in real HD manufacturing lines. In this paper, a fault diagnosis method was proposed to convert multiple vibration signals into symmetrized dot pattern (SDP) images, and to use a ConvNeXt model with Transformer training procedures to classify fault features of the HDs under various fault states and working conditions. The method was experimentally validated on three datasets with a diagnostic accuracy of up to 98.5% and 95.3% under constant and variable conditions, respectively, both higher over those using either state-of-the-art convolutional neural networks or Transformers. This was the first time where the SDP and the ConvNeXt were unitedly used to give superior visibility, interpretability, accuracy and efficiency for HD fault diagnosis, indicating good feasibility of using the ConvNeXt as a backbone for this task.

Side-scan sonar (SSS) is attracting increasing attention in ocean exploration for its utility and stability on autonomous underwater vehicles (AUVs). Existing SSS image recognition methods mainly employ deep neural networks (DNNs) for various tasks. However, the effectiveness of DNN-based approaches is limited in situations with zero samples. In this letter, a multi-feature fusion self-attention network (MFSANet) is proposed to generate SSS images of novel categories, transforming this problem into a conventional supervised learning problem. Specifically, optical-acoustic image pairs are used as inputs to the network to synthesize pseudo-SSS images. First, the shallow and deep features of the input images are extracted by different layers of the encoder. Then, the long-range dependencies of the acoustic images are efficiently modeled with the proposed simplified self-attention module (SSAM). Finally, the acoustic features are appropriately aggregated to each position of the optical features to efficiently generate pseudo-SSS samples for training the classification network. In addition, a novel contrastive loss is proposed to optimize the cross-modal feature space distribution. Experimental results demonstrate that our method can efficiently generate high-quality pseudo-SSS samples, which improves the accuracy of zero-shot SSS image recognition.

Using non-invasive technology to control prosthetics remains a real-life challenge due to the low acceptability of prosthetics stemming from their unnatural motion trajectory. The continuous angle estimation of fingers based on surface electromyography (sEMG) can effectively improve the unnatural motion trajectory. This paper proposes the Transformer using Shifted Windows and convolution networks (SCTNet) to achieve high-precision, fine-grained and high naturalness finger continuous angle estimation. The shifted windows is used to encrease receptive field and capture multi-scale sEMG signals. The Transformer is employed to capture the long-term dependencies within multi-scale sEMG signals, facilitating the extraction of global features, while CNN is utilized to extract local features. We use twenty subjects of Ninapro DB2 dataset to test the model, and compared with LS-TCN, CNN-Attention, ConvLSTM and LSTM models. The Pearson correlation coefficient (PCC), the normalized root means square error (NRMSE), and the coefficient of determination (R2) of the SCTNet was 84.35%, 8.85%, and 69.65%. Compared with the indicator results of other models, the SCTNet has improved by 1% (PCC), 0.15% (NRMSE), and 2.13% (R2) in each index, and achieved competitive results in real-time computing performance. The results indicate that the SCTNet can effectively estimate the natural continuous finger joint angles.

Bursting interference that causes a sudden and significant change in surface electromyography (sEMG) characteristics, can reduce the stability and security of myoelectric assistive robots. Current adaptation strategies for progressive-generated interference are incapable of dealing with bursting interference. To address this problem, an incremental learning and fault-tolerant classifier (ILFTC) was proposed by combining a Gaussian mixture model (GMM) ensemble and linear discriminant analysis (LDA), in conjunction with online update and marginalization schemes. Subsequently, an ILFTC-based myoelectric pattern recognition (MPR) strategy was developed to improve the robustness of MPR against multiple interferences, including outlier motion and missing/fault data owing to electrode loosening. Experiments on hand/wrist motions were conducted to validate the anti-interference performance of the ILFTC. Experimental results showed that the ILFTC could effectively resist the two types of bursting interference and produce a significant improvement in the recognition performance over traditional classifiers, as well as the methods presented in previous studies. The results show that the proposed method has the potential to enhance the robustness of myoelectric assistive robots.

Arbitrary style transfer is attracting increasing attention in the computer vision community due to its application flexibility. Existing approaches directly fuse deep style features with deep content features or adaptively normalize content features for global statistical matching. Although effective, it is prone to suffer from local unnatural outputs and artifacts owing to the lack of exploring the global contextual semantic distribution of style image features. In this paper, a novel global context self-attentional network (GCSANet) is proposed to efficiently generate high-quality stylized results based on the global semantic spatial distributions of style images. First, a context modeling module is proposed to aggregate the depth features of style images into global context features. Then, channel-wise interdependencies are captured with the feature transform module. Finally, the style features are appropriately aggregated to each location of the content image. In addition, novel external contrastive losses are proposed to balance the distribution of content and style features to ensure the reasonableness of the texture patterns in the stylized images. The ablation studies validate the effectiveness of the proposed components. Various quantitative and qualitative experiments demonstrate the superiority of our method for real-time arbitrary image/video style transfer.