Remaining useful life (RUL) prediction is one of the key technologies of condition-based maintenance (CBM), which is important to maintain the reliability and safety of industrial equipment. Massive industrial measurement data has effectively improved the performance of the data-driven-based RUL prediction method. While deep learning has achieved great success in RUL prediction, existing methods have difficulties in processing long sequences and extracting information from the sensor and time step aspects. In this article, we propose dual-aspect self-attention based on transformer (DAST), a novel deep RUL prediction method, which is an encoder–decoder structure purely based on self-attention without any recurrent neural network (RNN)/convolution neural network (CNN) module. DAST consists of two encoders, which work in parallel to simultaneously extract features of different sensors and time steps. Solely based on self-attention, the DAST encoders are more effective in processing long data sequences and are capable of adaptively learning to focus on more important parts of the input. Moreover, the parallel feature extraction design avoids the mutual influence of information from two aspects. Experiments on two widely used turbofan engines datasets show that our method significantly outperforms the state-of-the-art RUL prediction methods.

Remaining useful life (RUL) prediction plays an important role in prognostics and health management (PHM) and can significantly enhance equipment reliability and safety in various engineering applications. Accurate RUL prediction enables proactive maintenance planning, helping prevent potential hazards and economic losses caused by equipment failures. Recently, while deep learning-based methods have swept the RUL prediction field, most existing methods still have difficulties in simultaneously extracting multiscale global and local degradation feature information from raw multi-sensor monitoring data. To address these issues, we propose a novel multiscale global and local self-attention-based network (MGLSN) for RUL prediction. MGLSN consists of global and local feature extraction subnetworks based on self-attention, which work in parallel to simultaneously extract the global and local degradation features of equipment and can adaptively focus on more important parts. While the global network captures long-term dependencies between time steps, the local network focuses on modeling local temporal dynamics. The design of parallel feature extraction can avoid the mutual influence of information from global and local aspects. Moreover, MGLSN adopts a multiscale feature extraction design (multiscale self-attention and convolution) to capture the global and local degradation patterns at different scales, which can be combined to better reflect the degradation trend. Experiments on the widely used Commercial Modular Aero-Propulsion System Simulation (CMAPSS), New CMAPSS (N-CMAPSS), and International Conference on Prognostics and Health Management 2008 challenge datasets provided by NASA show that MGLSN significantly outperforms state-of-the-art RUL prediction methods and has great application prospects in the field of PHM.

Music genre classification (MGC) is the basis for the efficient organization, retrieval, and recommendation of music resources, so it has important research value. Convolutional neural networks (CNNs) have been widely used in MGC and achieved excellent results. However, CNNs cannot model global features well due to the influence of the local receptive field; these global features are crucial for classifying music signals with temporal properties. Transformers can capture long-range dependencies within an image thanks to adopting the self-attention mechanism. Nevertheless, there are still performance and computational cost gaps between Transformers and existing CNNs. In this paper, we propose a hybrid architecture (CNN-TE) based on CNN and Transformer encoder for MGC. Specifically, we convert the audio signals into mel spectrograms and feed them into a hybrid model for training. Our model employs a CNN to initially capture low-level and localized features from the spectrogram. Subsequently, these features are processed by a Transformer encoder, which models them globally to extract high-level and abstract semantic information. This refined information is then classified using a multi-layer perceptron. Our experiments demonstrate that this approach surpasses many existing CNN architectures when tested on the GTZAN and FMA datasets. Notably, it achieves these results with fewer parameters and a faster inference speed.