Recurrent neural networks (RNNs) have been widely adopted in research areas concerned with sequential data, such as text, audio, and video. However, RNNs consisting of sigma cells or tanh cells are unable to learn the relevant information of input data when the input gap is large. By introducing gate functions into the cell structure, the long short-term memory (LSTM) could handle the problem of long-term dependencies well. Since its introduction, almost all the exciting results based on RNNs have been achieved by the LSTM. The LSTM has become the focus of deep learning. We review the LSTM cell and its variants to explore the learning capacity of the LSTM cell. Furthermore, the LSTM networks are divided into two broad categories: LSTM-dominated networks and integrated LSTM networks. In addition, their various applications are discussed. Finally, future research directions are presented for LSTM networks.

Remaining useful life estimation is central to the prognostics and health management of systems, particularly for safety-critical systems, and systems that are very expensive. We present a non-linear model to estimate the remaining useful life of a system based on monitored degradation signals. A diffusion process with a nonlinear drift coefficient with a constant threshold was transformed to a linear model with a variable threshold to characterize the dynamics and nonlinearity of the degradation process. This new diffusion process contrasts sharply with existing models that use a linear drift, and also with models that use a linear drift based on transformed data that were originally nonlinear. Both existing models are based on a constant threshold. To estimate the remaining useful life, an analytical approximation to the distribution of the first hitting time of the diffusion process crossing a threshold level is obtained in a closed form by a time-space transformation under a mild assumption. The unknown parameters in the established model are estimated using the maximum likelihood estimation approach, and goodness of fit measures are applied. The usefulness of the proposed model is demonstrated by several real-world examples. The results reveal that considering nonlinearity in the degradation process can significantly improve the accuracy of remaining useful life estimation.