Deep transfer learning (DTL), which incorporates new ideas from deep neural networks into transfer learning (TL), has achieved excellent success in computer vision, text classification, behavior recognition, and natural language processing. As a branch of machine learning, DTL applies end-to-end learning to overcome the drawback of traditional machine learning that regards each dataset individually. Although some valuable and impressive general surveys exist on TL, special attention and recent advances in DTL are lacking. In this survey, we first review more than 50 representative approaches of DTL in the last decade and systematically summarize them into four categories. In particular, we further divide each category into subcategories according to models, functions, and operation objects. In addition, we discuss recent advances in TL in other fields and unsupervised TL. Finally, we provide some possible and exciting future research directions.

This article proposes an efficient nonlinear process monitoring method (DCCA-SCO) by integrating canonical correlation analysis (CCA), deep autoencoder neural networks (DAENNs), and sparsity-constrained optimization (SCO). Specifically, DAENNs are first used to learn a nonlinear function automatically, which characterizes intrinsic features of the original process data. Then, the CCA is performed in that low-dimensional representation space to extract the most correlated variables. In addition, the SCO is imposed to reduce the redundancy of the hidden representation. Unlike other deep CCA methods, the DCCA-SCO provides a new nonlinear method that is able to learn a nonlinear mapping with a sparse prior. The validity of the proposed DCCA-SCO is extensively demonstrated on the benchmark Tennessee Eastman (TE) process and the diesel generator process. In particular, compared with the classical CCA, the fault detection rate is increased by 8.00% for the fault IDV(11) in the TE process.

This article proposes a novel modeling method for the stochastic nonlinear degradation process by using the relevance vector machine (RVM), which can describe the nonlinearity of degradation process more flexibly and accurately. Compared with the existing methods, where degradation processes are modeled as the Wiener process with a nonlinear drift function formulized as the power law or exponential law, this kind of modeling method can characterize degradation processes with more nonlinear behavior. Instead of modeling the drift coefficient of the Wiener process directly, the weighted combination of basis functions is utilized to express the increment of the Wiener process and the parameters are calculated by a sparse Bayesian learning algorithm. Based on the proposed model, a numerical approximation formula for the probability density function (PDF) of the remaining useful life (RUL) is derived. Finally, comparison studies, including a numerical simulation and a practical case, are provided to demonstrate the effectiveness and the accuracy of the proposed methods for RUL estimation.

With the rapid development of science and technology, high-dimensional data have been widely used in various fields. Due to the complex characteristics of high-dimensional data, it is usually distributed in the union of several low-dimensional subspaces. In the past several decades, subspace clustering (SC) methods have been widely studied as they can restore the underlying subspace of high-dimensional data and perform fast clustering with the help of the data self-expressiveness property. The SC methods aim to construct an affinity matrix by the self-representation coefficient of high-dimensional data and then obtain the clustering results using the spectral clustering method. The key is how to design a self-expressiveness model that can reveal the real subspace structure of data. In this survey, we focus on the development of SC methods in the past two decades and present a new classification criterion to divide them into three categories based on the purpose of clustering, i.e., low-rank sparse SC, local structure preserving SC, and kernel SC. We further divide them into subcategories according to the strategy of constructing the representation coefficient. In addition, the applications of SC methods in face recognition, motion segmentation, handwritten digits recognition, and speech emotion recognition are introduced. Finally, we have discussed several interesting and meaningful future research directions.

In order to improve the performance of canonical correlation analysis (CCA) based methods for process monitoring, this paper proposes a novel process monitoring approach using the structured joint sparse canonical correlation analysis (SJSCCA). Technically, the graph Laplacian could incorporate structured variable correlation information and the joint sparsity could discard useless variables. The developed two-stage alternating direction method of multipliers is shown to be very efficient because each derived subproblem has a closed-form solution or can be solved by fast solvers. In order to detect abnormal situations, T2 test statistic is adopted. The validity of SJSCCA is illustrated by the benchmark Tennessee Eastman process. The achieved results show that the proposed SJSCCA is able to improve the monitoring performance significantly in comparison with the existing state-of-the-art CCA-based methods.