With the advancement of deep learning algorithms and the growing availability of computational power, deep learning-based forecasting methods have gained significant importance in the domain of time series forecasting. In the past decade, there has been a rapid rise in time series forecasting approaches. This paper comprehensively reviews the advancements in deep learning-based forecasting models spanning 2014 to 2024. We provide a comprehensive examination of the capabilities of these models in capturing correlations among time steps and time series variables. Additionally, we explore methods to enhance the efficiency of long-term time series forecasting and summarize the diverse loss functions employed in these models. Moreover, this study systematically evaluates the effectiveness of these approaches in both univariate and multivariate time series forecasting tasks across diverse domains. We comprehensively discuss the strengths and limitations of various algorithms from multiple perspectives, analyze their capacity to capture different types of time series information, including trend and season patterns, and compare methods for enhancing the computational efficiency of these models. Finally, we summarize the experimental results and discuss the future directions in time series forecasting. Codes and datasets are available at https://github.com/TCCofWANG/Deep-Learning-based-Time-Series-Forecasting.

From the feature representation's point of view, the feature learning module of a convolutional neural network (CNN) is to transform an input pattern into a feature vector. This feature vector is then multiplied with a number of output weight vectors to produce softmax scores. The common training objective in CNNs is based on the softmax loss, which ignores the intra-class compactness. This brief proposes a constrained center loss (CCL)-based algorithm to extract robust features. The training objective of a CNN consists of two terms, softmax loss and CCL. The aim of the softmax loss is to push the feature vectors from different classes apart. Meanwhile, the CCL aims at clustering the feature vectors such that the feature vectors from the same classes are close together. Instead of using stochastic gradient descent (SGD) algorithms to learn all the connection weights and the cluster centers at the same time. Our CCL-based algorithm is based on the alternative learning strategy. We first fix the connection weights of the CNN and update the cluster centers based on an analytical formula, which can be implemented based on the minibatch concept. We then fix the cluster centers and update the connection weights for a number of SGD minibatch iterations. We also propose a simplified CCL (SCCL) algorithm. Experiments are performed on six commonly used benchmark datasets. The results demonstrate that the two proposed algorithms outperform several state-of-the-art approaches.

In urban transportation systems, missing data and noise contamination are almost inevitable. To address the challenge of imputing traffic data corrupted by noise and outliers in real-world scenarios, this paper proposes a novel algorithm based on spatiotemporal tensor completion. The proposed method transforms observed data into three-dimensional spatiotemporal tensors and utilizes tensor ring decomposition for data completion. Furthermore, spatial and temporal information is incorporated into the model by utilizing the graph Laplacian matrix. To handle outliers, they are treated as unknown parameters, and the ℓ₀-norm is introduced to ensure their sparsity, thereby achieving the Spatio-Temporal Tensor Completion model with ℓ₀-norm term (STTC-ℓ₀). The solution to the model is derived using the alternating optimization framework with the alternating direction method of multipliers. Then, we discuss the convergence of the solution method. To further enhance the efficiency of our proposed method, we combine the unrolling algorithm with our iterative optimization model, creating a lightweight and efficient neural network tailored for tensor completion, called STTC-ℓ₀-NN. Extensive experiments conducted on real datasets demonstrate the superiority of our proposed method over several state-of-the-art methods across various experimental scenarios. It is worth noting that STTC-ℓ₀-NN reduces computational time by one to two orders of magnitude compared to existing methods while maintaining or even improving imputation accuracy. The code is available at https://github.com/TCCofWANG/STTC-L0-and-STTC-L0-NN.

The objective of compressive sampling is to determine a sparse vector from an observation vector. This brief describes an analog neural method to achieve the objective. Unlike previous analog neural models which either resort to the ℓ₁-norm approximation or are with local convergence only, the proposed method avoids any approximation of the ℓ₁-norm term and is probably capable of leading to the optimum solution. Moreover, its computational complexity is lower than that of the other three comparison analog models. Simulation results show that the error performance of the proposed model is comparable to several state-of-the-art digital algorithms and analog models and that its convergence is faster than that of the comparison analog neural models.