Missing values, which are common in multivariate time series, is most important obstacle towards the utilization and interpretation of those data. Great efforts have been employed on how to accurately impute missing values in multivariate time series, and existing works either use deep learning networks to achieve deterministic imputations or aim at generating different plausible imputations by sampling multiple noises from a same distribution and then denoising them. However, these models either fall short of modeling the uncertainties of imputations due to their deterministic nature or perform poorly in terms of interpretability and imputation accuracy due to their ignorance of the correlations between the latent representations of both observed and missing values which are parts of samples from a same distribution. To this end, in this paper, we explicitly take the correlations between observed and missing values into account, and theoretically re-derive the Evidence Lower BOund (ELBO) of conditional diffusion model in the scenario of multivariate time series imputation. Based on the newly derived ELBO, we further propose a novel multivariate imputation diffusion model (MIDM) which is equipped with novel noise sampling, adding and denoising mechanisms for multivariate time series imputation, and the series of newly designed technologies jointly ensure the involving of the consistency between observed and missing values. Extensive experiments on both the tasks of multivariate time series imputation and forecasting witness the superiority of our proposed MIDM model on generating conditional estimations.

Accurate traffic prediction plays a vital role in intelligent transport managements and applications. However, in the vast majority of existing works, the focus is mainly on modeling spatiotemporal correlations in static traffic networks. Thus, the continuous expansion and evolution of traffic networks are ignored. In this work, we study the problem of traffic prediction with expanding road network structures under the continual learning paradigm. Considering the model prediction performance, efficiency, and data accessibility, a SpatioTemporal Knowledge Expansion and Consolidation (STKEC) framework is proposed. This framework contains an influence-based knowledge expansion strategy to help the spatiotemporal learning model integrate new spatiotemporal traffic patterns and a memory-augmented knowledge consolidation mechanism to preserve the learned spatiotemporal patterns without accessing the data in previous graphs. Extensive experiments are conducted on a large-scale dataset and verify the superior performance of STKEC in continual traffic prediction.

Spatiotemporal (ST) learning has become a crucial technique for urban digitalization. Due to expansions and dynamics of cities, current spatiotemporal models are inclined to suffer distribution shifts between training and testing sets, leading to the OOD delimma. However, few studies focus on such OOD problem in temporal regressions, let alone spatiotemporal learning. Spatiotemporal data usually reveals segment-level heterogeneity within periodicity and complex spatial dependencies, posing challenges to invariance extraction. In this paper, we find that ST relations make sense for generalization and devise a Causal ST learning framework, CauSTG, which enables invariant relation transferred to OOD scenarios. Specifically, we take temporal steps as environments, and transform spatial-temporal relations into learnable parameters. To tackle heterogeneity in periodicity, we partition temporal steps into sub-environments by identifying distinctive trend patterns, enabling re-organized samples trained separately. To extract invariance within ST observations, we propose a spatiotemporal consistency learner and a hierarchical invariance explorer to jointly filter out stable relations. Our spatiotemporal learner quantifies bi-directional spatial consistency and extracts disentangled seasonal-trend patterns via trainable parameters. Further, the hierarchical invariance explorer constructs variation-based filter to achieve both local and global invariances. Experiments reveal that CauSTG can increase at most 10.26% performance against best baselines, and visualized invariant relations can well interpret the physical rationales. The appendix and codes can be available in our Github repository.

The training and inference of Graph Neural Networks (GNNs) are costly when scaling up to large-scale graphs. Graph Lottery Ticket (GLT) has presented the first attempt to accelerate GNN inference on large-scale graphs by jointly pruning the graph structure and the model weights. Though promising, GLT encounters robustness and generalization issues when deployed in real-world scenarios, which are also long-standing and critical problems in deep learning ideology. In real-world scenarios, the distribution of unseen test data is typically diverse. We attribute the failures on out-of-distribution (OOD) data to the incapability of discerning causal patterns, which remain stable amidst distribution shifts. In traditional spase graph learning, the model performance deteriorates dramatically as the graph/network sparsity exceeds a certain high level. Worse still, the pruned GNNs are hard to generalize to unseen graph data due to limited training set at hand. To tackle these issues, we propose the Resilient Graph Lottery Ticket (RGLT) to find more robust and generalizable GLT in GNNs. Concretely, we reactivate a fraction of weights/edges by instantaneous gradient information at each pruning point. After sufficient pruning, we conduct environmental interventions to extrapolate potential test distribution. Finally, we perform last several rounds of model averages to further improve generalization. We provide multiple examples and theoretical analyses that underpin the universality and reliability of our proposal. Further, RGLT has been experimentally verified across various independent identically distributed (IID) and out-of-distribution (OOD) graph benchmarks. The source code for this work is available at https://github.com/Lyccl/RGLT for PyTorch implementation.

Traffic prediction is a crucial task in the Intelligent Transportation System (ITS), receiving significant attention from both industry and academia. Numerous spatio-temporal graph convolutional networks have emerged for traffic prediction and achieved remarkable success. However, these models have limitations in terms of generalization and scalability when dealing with Out-of-Distribution (OOD) graph data with both structural and temporal shifts. To tackle the challenges of spatio-temporal shift, we propose a framework called STONE by learning invariable node dependencies, which achieve stable performance in variable environments. STONE initially employs gated-transformers to extract spatial and temporal semantic graphs. These two kinds of graphs represent spatial and temporal dependencies, respectively. Then we design three techniques to address spatio-temporal shifts. Firstly, we introduce a Fréchet embedding method that is insensitive to structural shifts, and this embedding space can integrate loose position dependencies of nodes within the graph. Secondly, we propose a graph intervention mechanism to generate multiple variant environments by perturbing two kinds of semantic graphs without any data augmentations, and STONE can explore invariant node representation from environments. Finally, we further introduce an explore-to-extrapolate risk objective to enhance the variety of generated environments. We conduct experiments on multiple traffic datasets, and the results demonstrate that our proposed model exhibits competitive performance in terms of generalization and scalability.