Heart rate variability (HRV) contains important information about the modulation of the cardiovascular system. Various methods of nonlinear dynamics (e.g., estimating Lyapunov exponents) and complexity measures (e.g., correlation dimension or entropies) have been applied to HRV analysis. Permutation entropy, which was proposed recently, has been widely used in many fields due to its conceptual and computational simplicity. It maps a time series onto a symbolic sequence of permutation ranks. The original permutation entropy assumes the time series under study has a continuous distribution, thus equal values are rare and can be ignored by ranking them according to their order of emergence, or broken by adding small random perturbations to ensure every symbol in a sequence is different. However, when the observed time series is digitized with lower resolution leading to a greater number of equal values, or the equalities represent certain characteristic sequential patterns of the system, it may not be rational to simply ignore or break them. In the present paper, a modified permutation entropy is proposed that, by mapping the equal value onto the same symbol (rank), allows for a more accurate characterization of system states. The application of the modified permutation entropy to the analysis of HRV is investigated using clinically collected data. Results show that modified permutation entropy can greatly improve the ability to distinguish the HRV signals under different physiological and pathological conditions. It can characterize the complexity of HRV more effectively than the original permutation entropy.

Time-Series Mining (TSM) is an important research area since it shows great potential in practical applications. Deep learning models that rely on massive labeled data have been utilized for TSM successfully. However, constructing a large-scale well-labeled dataset is difficult due to data annotation costs. Recently, pre-trained models have gradually attracted attention in the time series domain due to their remarkable performance in computer vision and natural language processing. In this survey, we provide a comprehensive review of Time-Series Pre-Trained Models (TS-PTMs), aiming to guide the understanding, applying, and studying TS-PTMs. Specifically, we first briefly introduce the typical deep learning models employed in TSM. Then, we give an overview of TS-PTMs according to the pre-training techniques. The main categories we explore include supervised, unsupervised, and self-supervised TS-PTMs. Further, extensive experiments involving 27 methods, 434 datasets, and 679 transfer learning scenarios are conducted to analyze the advantages and disadvantages of transfer learning strategies, Transformer-based models, and representative TS-PTMs. Finally, we point out some potential directions of TS-PTMs for future work. The source code is available at https://github.com/qianlima-lab/time-series-ptms .

As the applications of large language models (LLMs) expand across diverse fields, their ability to adapt to ongoing changes in data, tasks, and user preferences becomes crucial. Traditional training methods with static datasets are inadequate for coping with the dynamic nature of real-world information. Lifelong learning, or continual learning, addresses this by enabling LLMs to learn continuously and adapt over their operational lifetime, integrating new knowledge while retaining previously learned information and preventing catastrophic forgetting. Our survey explores the landscape of lifelong learning, categorizing strategies into two groups based on how new knowledge is integrated: Internal Knowledge, where LLMs absorb new knowledge into their parameters through full or partial training, and External Knowledge, which incorporates new knowledge as external resources such as Wikipedia or APIs without updating model parameters. The key contributions of our survey include: (1) introducing a novel taxonomy to categorize the extensive literature of lifelong learning into 12 scenarios; (2) identifying common techniques across all lifelong learning scenarios and classifying existing literature into various technique groups; (3) highlighting emerging techniques such as model expansion and data selection, which were less explored in the pre-LLM era. Resources are available at https://github.com/qianlima-lab/awesome-lifelong-learning-methods-for-llm .

Time series with missing values (incomplete time series) are ubiquitous in real life on account of noise or malfunctioning sensors. Time-series imputation (replacing missing data) remains a challenge due to the potential for nonlinear dependence on concurrent and previous values of the time series. In this paper, we propose a novel framework for modeling incomplete time series, called a linear memory vector recurrent neural network (LIME-RNN), a recurrent neural network (RNN) with a learned linear combination of previous history states. The technique bears some similarity to residual networks and graph-based temporal dependency imputation. In particular, we introduce a linear memory vector [called the residual sum vector (RSV)] that integrates over previous hidden states of the RNN, and is used to fill in missing values. A new loss function is developed to train our model with time series in the presence of missing values in an end-to-end way. Our framework can handle imputation of both missing-at-random and consecutive missing inputs. Moreover, when conducting time-series prediction with missing values, LIME-RNN allows imputation and prediction simultaneously. We demonstrate the efficacy of the model via extensive experimental evaluation on univariate and multivariate time series, achieving state-of-the-art performance on synthetic and real-world data. The statistical results show that our model is significantly better than most existing time-series univariate or multivariate imputation methods.

Incomplete time series classification (ITSC) is an important issue in time series analysis since temporal data often has missing values in practical applications. However, integrating imputation (replacing missing data) and classification within a model often rapidly amplifies the error from imputed values. Reducing this error propagation from imputation to classification remains a challenge. To this end, we propose an Adversarial Joint-learning Recurrent Neural Network (AJ-RNN) for ITSC, an end-to-end model trained in an adversarial and joint learning manner. We train the system to categorize the time series as well as impute missing values. To alleviate the error introduced by each imputation value, we use an adversarial network to encourage the network to impute realistic missing values by distinguishing real and imputed values. Hence, AJ-RNN can directly perform classification with missing values and greatly reduce the error propagation from imputation to classification, boosting the accuracy. Extensive experiments on 68 synthetic datasets and 4 real-world datasets from the expanded UCR time series archive demonstrate that AJ-RNN achieves state-of-the-art performance. Furthermore, we show that our model can effectively alleviate the accumulating error problem through qualitative and quantitative analysis based on the trajectory of the dynamical system learned by the RNN.