The Intelligent Autonomous Transport System in 6G (6G-IATS) refers to the coordination of 6G, Artificial Intelligence (AI), and intelligent transportation systems, which is expected to revolutionize future intelligent transportation systems. In 6G-IATS, large-scale traffic flow prediction, affiliated with time series prediction, holds significant value for transportation planning and urban management. As an emerging AI method, Large Language Models (LLMs) have emerged prominently in time series forecasting. Unfortunately, it is challenging to achieve accurate and efficient large-scale traffic flow prediction by LLMs in 6G-IATS, due to the two issues: a) these LLMs fail to capture the spatio-temporal correlations in a large-scale road network, leading to limited prediction accuracy, and b) they process a substantial amount of training data on the central server, which imposes low training efficiency. Jointly considering the two concerns, this paper proposes a novel LLM and edge computing-based architecture for large-scale traffic flow prediction in 6G-IATS, called Spatio-Temporal Generative Large Language Model on Edge (STGLLM-E). In this architecture, we first decompose the entire large-scale road network into several subgraphs. To capture the spatio-temporal correlations, an LLM-based method named Spatio-Temporal Generative Large Language Model (STGLLM) including Spatio-Temporal Module (STM) and Generative Large Language Model (GLLM) is proposed. Secondly, to improve the training efficiency of the STGLLM-E, an edge training strategy based on edge servers is devised. Experiments are conducted on two real-world traffic flow datasets. The experimental results illustrate that the STGLLM-E is superior to the baselines in the prediction accuracy and the efficiency of training.

Spatio-Temporal (ST) data is pervasive on the various aspects in our daily lives. By mining the ST information from the data, we are able to predict trends in numerous domains. The Transformer and one of its more recent enhancements, foundation models, have achieved a remarkable success in such ST prediction. In this article, we first survey the state of the art of Transformers-related work, then introduce the network architecture of the Transformer and summarize the improvements to adapt to the ST prediction Transformer and foundation models, including module enhancement and adjustment. Subsequently, we categorize the ST Transformer and foundation models in selected applications in some relevant domains, mainly urban transportation, climate monitoring, and motion prediction. Next, we propose an evaluation method in the ST prediction with Transformers and foundation models, list the most relevant open-source datasets, evaluation metrics and performance analysis. Finally, we discuss some future directions on the task of ST prediction with Transformer and foundation models. Relevant papers and open-source resources have been collated and are continuously updated at: https://github.com/cyhforlight/Spatio-Temporal-Prediction-Transformer-Review .

The heterogeneous network formed by the deployment and interconnection of various network devices (e.g., sensors) has attracted widespread attention. $P{M}_{2.5}$ forecasting on the entire industrial region throughout mainland China is an important application of heterogeneous networks, which has great significance to factory management and human health travel. In recent times, Large Language Models (LLMs) have exhibited notability in terms of time series prediction. However, existing LLMs tend to forecast nationwide industry $P{M}_{2.5}$, which encounters two issues. First, most LLM-based models use centralized training, which requires uploading large amounts of data from sensors to a central cloud. This entire transmission process can lead to security risks of data leakage. Second, LLMs fail to extract spatiotemporal correlations in the nationwide sensor network (heterogeneous network). To tackle these issues, we present a novel framework entitled Spatio-Temporal Large Language Model with Edge Computing Servers (STLLM-ECS) to securely predict nationwide industry $P{M}_{2.5}$ in China. In particular, We initially partition the entire sensor network, located in the national industrial region, into several subgraphs. Each subgraph is allocated an edge computing server (ECS) for training and inference, avoiding the security risks caused by data transmission. Additionally, a novel LLM-based approach named Spatio-Temporal Large Language Model (STLLM) is developed to extract spatiotemporal correlations and infer prediction sequences. Experimental results prove the effectiveness of our proposed model.

Accurate urban $P{M}_{2.5}$ forecasting serves a crucial function in air pollution warning and human health monitoring. Recently, deep learning techniques have been widely employed for urban $P{M}_{2.5}$ forecasting. Unfortunately, two problems exist: (1) Most techniques are focused on training and prediction on a central cloud. As the number of monitoring sites grows and the data explodes, handling a large amount of data on the central cloud can cause tremendous computational pressures and increase the risk of data leakages. (2) Existing methods lack an adaptive layer to capture the varying impacts of different external factors (e.g., weather conditions, temperature, and wind speed). In this paper, a federated deep learning network (FedDeep) is developed for edge-assisted multi-urban $P{M}_{2.5}$ forecasting. First, we assign each urban region to an edge cloud server (ECS). An external spatio-temporal network (ESTNet) is then deployed on each ECS. Data from different urban regions are uploaded to the corresponding ECS for training, which avoids processing all the data on the central cloud and effectively alleviates computational pressure and data leakage issues. Second, in ESTNet, we develop a gating fusion layer to adaptively fuse external factors to improve prediction accuracy. Finally, we adopted $P{M}_{2.5}$ data collected from air quality monitoring sites in 13 prefecture-level cities, Jiangsu Province for validation. The experimental results proved that FedDeep outperformed the advanced baselines in terms of prediction accuracy and model efficiency.