Multivariate time series forecasting (MTSF) aims to learn temporal dynamics among variables to forecast future time series. Existing statistical and deep learning-based methods suffer from limited learnable parameters and small-scale training data. Recently, large language models (LLMs) combining time series with textual prompts have achieved promising performance in MTSF. However, we discovered that current LLM-based solutions fall short in learning disentangled embeddings. We introduce TimeCMA, an intuitive yet effective framework for MTSF via cross-modality alignment. Specifically, we present a dual-modality encoding with two branches: the time series encoding branch extracts disentangled yet weak time series embeddings, and the LLM-empowered encoding branch wraps the same time series with text as prompts to obtain entangled yet robust prompt embeddings. As a result, such a cross-modality alignment retrieves both disentangled and robust time series embeddings, ``the best of two worlds'', from the prompt embeddings based on time series and prompt modality similarities. As another key design, to reduce the computational costs from time series with their length textual prompts, we design an effective prompt to encourage the most essential temporal information to be encapsulated in the last token: only the last token is passed to downstream prediction. We further store the last token embeddings to accelerate inference speed. Extensive experiments on eight real datasets demonstrate that TimeCMA outperforms state-of-the-arts.

Accurately predicting the urban spatio-temporal data is critically important to various urban computing tasks for smart city related applications such as crowd flow prediction and traffic congestion prediction. Existing models especially deep learning based approaches require a large volume of training data, whose performance may degrade remarkably when the data is scarce. Recent works try to transfer knowledge from the intra-city or cross-city multi-modal spatio-temporal data. However, the careful design of what to transfer and how between the multi-modal spatio-temporal data needs to be determined in advance. There still lacks an end-to-end solution that can automatically capture the common cross-domain knowledge. In this paper, we propose a D eep A ttentive A daptation N etwork model named ST-DAAN to transfer cross-domain S patio- T emporal knowledge for urban crowd flow prediction. ST-DAAN first maps the raw spatio-temporal data of source domain and target domain to a common embedding space. Then domain adaptation is adopted on several domain-specific layers through adding a domain discrepancy penalty to explicitly match the mean embeddings of the two domain distributions. Considering the complex spatial correlation in many urban spatio-temporal data, a global attention mechanism is also designed to enable the model to capture broader spatial dependencies. Using urban crowd flow prediction as a demonstration, we conduct experiments on five real-world large datasets over both intra- and cross-city transfer learning. The results demonstrate that ST-DAAN outperforms state-of-the-art methods by a large margin.

Abnormal traffic incidents such as traffic accidents have become a significant health and development threat with the rapid urbanization of many countries. The challenges of accurate traffic risk forecasting are three-fold. First, traffic accident data in some areas of a city is sparse, especially for a fine-grained prediction, which may cause the zero inflation problem during model training. Second, the spatio-temporal correlations of the traffic accidents are rather complex and non-linear, which is difficult to capture by existing shallow models like regression. Third, the occurrence of traffic accidents can be significantly affected by various context features including weather, POI and road network features. To address the above challenges, this paper proposes a Multi-View Multi-Task Spatio-Temporal Networks (MVMT-STN) model to forecast fine- and coarse-grained traffic accident risks of a city simultaneously. Specifically, to address the data sparsity issue in a fine-grained prediction, we adopt a multi-task learning framework to jointly forecast both fine- and coarse-grained traffic accident risks by considering their spatial associations. We conduct extensive experiments over two large real traffic accident datasets. The results show that MVMT-STN improves the performance of traffic accident risk prediction in both fine- and coarse-grained prediction by a large margin compared with existing state-of-the-art.

With the proliferation of mobile sensing techniques, huge amounts of time series data are generated and accumulated in various domains, fueling plenty of real-world applications. In this setting, time series anomaly detection is practically important. It endeavors to identify deviant samples from the normal sample distribution in time series. Existing approaches generally assume that all the time series is available at a central location. However, we are witnessing the decentralized collection of time series due to the deployment of various edge devices. To bridge the gap between the decentralized time series data and the centralized anomaly detection algorithms, we propose a Parameter-efficient Federated Anomaly Detection framework named PeFAD with the increasing privacy concerns. PeFAD for the first time employs the pre-trained language model (PLM) as the body of the client's local model, which can benefit from its cross-modality knowledge transfer capability. To reduce the communication overhead and local model adaptation cost, we propose a parameter-efficient federated training module such that clients only need to fine-tune small-scale parameters and transmit them to the server for update. PeFAD utilizes a novel anomaly-driven mask selection strategy to mitigate the impact of neglected anomalies during training. A knowledge distillation operation on a synthetic privacy-preserving dataset that is shared by all the clients is also proposed to address the data heterogeneity issue across clients. We conduct extensive evaluations on four real datasets, where PeFAD outperforms existing state-of-the-art baselines by up to 28.74%.

Spatial Crowdsourcing (SC) is finding widespread application in today's online world. As we have transitioned from desktop crowdsourcing applications (e.g., Wikipedia) to SC applications (e.g., Uber), there is a sense that SC systems must not only provide effective task assignment but also need to ensure privacy. To achieve these often-conflicting objectives, we propose a framework, Task Assignment with Federated Preference Learning, that performs task assignment based on worker preferences while keeping the data decentralized and private in each platform center (e.g., each delivery center of an SC company). The framework includes a federated preference learning phase and a task assignment phase. Specifically, in the first phase, we build a local preference model for each platform center based on historical data. We provide means of horizontal federated learning that makes it possible to collaboratively train these local preference models under the orchestration of a central server. Specifically, we provide a practical method that accelerates federated preference learning based on stochastic controlled averaging and achieves low communication costs while considering data heterogeneity among clients. The task assignment phase aims to achieve effective and efficient task assignment by considering workers' preferences. Extensive evaluations on real data offer insight into the effectiveness and efficiency of the paper's proposals.

The widespread deployment of wireless and mobile devices results in a proliferation of decentralized spatio-temporal data. Many recent proposals that target deep learning for spatio-temporal prediction assume that all data is available at a central location and suffers from so-called catastrophic forgetting, where previously learned knowledge is entirely forgotten when new data arrives. Such proposals may face data privacy concerns and may experience deteriorating prediction performance when applied in decentralized settings where data streams into the system. To bridge the gap between decentralized training and spatio-temporal prediction on streaming data, we propose a unified federated continuous learning framework, which uses a horizontal federated learning mechanism for protecting data privacy and includes a global replay buffer with synthetic spatio-temporal data generated by the previously learned global model. For each client, we fuse the current training data with synthetic spatio-temporal data using a spatio-temporal mixup mechanism to preserve historical knowledge effectively, thus avoiding catastrophic forgetting. To enable holistic representation preservation, the local models at clients each integrates a general spatio-temporal autoencoder with a spatio-temporal simple siamese network that aims to ensure prediction accuracy and avoid holistic feature loss. Extensive experiments on real data offer insight into the effectiveness of the proposed framework.