Self-supervised learning (SSL) has recently achieved impressive performance on various time series tasks. The most prominent advantage of SSL is that it reduces the dependence on labeled data. Based on the pre-training and fine-tuning strategy, even a small amount of labeled data can achieve high performance. Compared with many published self-supervised surveys on computer vision and natural language processing, a comprehensive survey for time series SSL is still missing. To fill this gap, we review current state-of-the-art SSL methods for time series data in this article. To this end, we first comprehensively review existing surveys related to SSL and time series, and then provide a new taxonomy of existing time series SSL methods by summarizing them from three perspectives: generative-based, contrastive-based, and adversarial-based. These methods are further divided into ten subcategories with detailed reviews and discussions about their key intuitions, main frameworks, advantages and disadvantages. To facilitate the experiments and validation of time series SSL methods, we also summarize datasets commonly used in time series forecasting, classification, anomaly detection, and clustering tasks. Finally, we present the future directions of SSL for time series analysis.

This article proposes a valve stiction detection strategy based on a convolutional neural network. Considering the commonly existing characteristics of industrial time-series signals, the strategy is developed to learn features on multiple timescales automatically. Unlike the traditional approaches using hand-crafted features, the proposed strategy can automatically learn representative features on the time-series data collected from industrial control loops. The strategy is composed of two complementary data conversion methods, a mixed feature learning stage and a fusion decision stage, and it has the following merits: 1) the interaction of different pairs of time series can be effectively captured; and 2) the whole process of feature learning is automatic, and no manual feature extraction is needed. The effectiveness of the proposed strategy is evaluated through the comprehensive data, including the International Stiction Data Base, and the real data collected from the real hardware experimental system and the industrial environment. Compared with four traditional methods and three deep-learning-based methods, the experimental results demonstrate that the proposed strategy outperforms the other methods. Besides performance evaluation, we give the implementation procedure of practical application of the proposed strategy and provide the detailed analysis from the perspective of the data conversion methods and the number of timescales.

Using neural networks to build a reliable fault detection model is an attractive topic in industrial processes but remains challenging due to the lack of labeled data. We propose a feature learning approach for industrial time-series data based on self-supervised contrastive learning to tackle this challenge. The proposed approach consists of two components: data transformation and representation learning. The data transformation converts the raw time-series to temporal distance matrices capable of storing temporal and spatial information. The representation learning component uses a convolution-based encoder to encode the temporal distance matrices to embedding representations. The encoder is trained using a new constraint called multitimescale feature consistent constraint. Finally, a fault detection framework for the valve stiction detection task is developed based on the feature learning method. The proposed framework is evaluated not only on an industrial benchmark dataset but also on a hardware experimental system and real industrial environments.

Building reliable fault detection systems through deep neural networks is an appealing topic in industrial scenarios. In these contexts, the representations extracted by neural networks on available labeled time-series data can reflect system states. However, this endeavor remains challenging due to the necessity of labeled data. Self-supervised contrastive learning (SSCL) is one of the effective approaches to deal with this challenge, but existing SSCL-based models suffer from sampling bias and representation bias problems. This article introduces a debiased contrastive learning framework for time-series data and applies it to industrial fault detection tasks. This framework first develops the multigranularity augmented view generation method to generate augmented views at different granularities. It then introduces the momentum clustering contrastive learning strategy and the expert knowledge guidance mechanism to mitigate sampling bias and representation bias, respectively. Finally, the experiments on a public bearing fault detection dataset and a widely used valve stiction detection dataset show the effectiveness of the proposed feature learning framework.

The time series self-supervised contrastive learning framework has succeeded significantly in industrial fault detection scenarios. It typically consists of pretraining on abundant unlabeled data and fine-tuning on limited annotated data. However, the two-phase framework faces three challenges: Sampling bias, task-agnostic representation issue, and angular-centricity issue. These challenges hinder further development in industrial applications. This article introduces a debiased contrastive learning with supervision guidance (DCLSG) framework and applies it to industrial fault detection tasks. First, DCLSG employs channel augmentation to integrate temporal and frequency domain information. Pseudolabels based on momentum clustering operation are assigned to extracted representations, thereby mitigating the sampling bias raised by the selection of positive pairs. Second, the generated supervisory signal guides the pretraining phase, tackling the task-agnostic representation issue. Third, the angular-centricity issue is addressed using the proposed Gaussian distance metric measuring the radial distribution of representations. The experiments conducted on three industrial datasets (ISDB, CWRU, and practical datasets) validate the superior performance of the DCLSG compared to other fault detection methods.

Self-supervised learning (SSL) has recently achieved impressive performance on various time series tasks. The most prominent advantage of SSL is that it reduces the dependence on labeled data. Based on the pre-training and fine-tuning strategy, even a small amount of labeled data can achieve high performance. Compared with many published self-supervised surveys on computer vision and natural language processing, a comprehensive survey for time series SSL is still missing. To fill this gap, we review current state-of-the-art SSL methods for time series data in this article. To this end, we first comprehensively review existing surveys related to SSL and time series, and then provide a new taxonomy of existing time series SSL methods by summarizing them from three perspectives: generative-based, contrastive-based, and adversarial-based. These methods are further divided into ten subcategories with detailed reviews and discussions about their key intuitions, main frameworks, advantages and disadvantages. To facilitate the experiments and validation of time series SSL methods, we also summarize datasets commonly used in time series forecasting, classification, anomaly detection, and clustering tasks. Finally, we present the future directions of SSL for time series analysis.