Modeling the spatiotemporal relationship (STR) of traffic data is important yet challenging for existing graph networks. These methods usually capture features separately in temporal and spatial dimensions or represent the spatiotemporal data by adopting multiple local spatial-temporal graphs. The first kind of method mentioned above is difficult to capture potential temporal-spatial relationships, while the other is limited for long-term feature extraction due to its local receptive field. To handle these issues, the Synchronous Spatio-Temporal grAph Transformer (S²TAT) network is proposed for efficiently modeling the traffic data. The contributions of our method include the following: 1) the nonlocal STR can be synchronously modeled by our integrated attention mechanism and graph convolution in the proposed S²TAT block; 2) the timewise graph convolution and multihead mechanism designed can handle the heterogeneity of data; and 3) we introduce a novel attention-based strategy in the output module, being able to capture more valuable historical information to overcome the shortcoming of conventional average aggregation. Extensive experiments are conducted on PeMS datasets that demonstrate the efficacy of the S²TAT by achieving a top-one accuracy but less computational cost by comparing with the state of the art.

Noncontrastive self-supervised learning methods offer an effective alternative to contrastive approaches by avoiding the need for negative samples to avoid representation collapse. Noncontrastive learning methods explicitly or implicitly optimize the representation space, yet they often require large representation dimensions, leading to dimensional inefficiency. To provide negative samples, contrastive learning methods often require large batch sizes, thus regarded as sample inefficient, while noncontrastive learning methods require large representation dimensions, thus regarded as dimension inefficient. Although we have some understanding of the noncontrastive learning method, theoretical analysis of such phenomenon still remains largely unexplored. We present a theoretical analysis of the dimensional need for noncontrastive learning. We investigate the transfer between upstream representation learning and downstream tasks’ performance, demonstrating how noncontrastive methods implicitly increase interclass distances within the representation space and how the distance affects the model performance of evaluation performance. We prove that the performance of noncontrastive methods is affected by the output dimension and the number of latent classes, and illustrate why performance degrades significantly when the output dimension is substantially smaller than the number of latent classes. We demonstrate our findings through experiments on image classification experiments, and enrich the verification in audio, graph and text modalities. We also perform empirical evaluation for image models on extensive detection and segmentation tasks beyond classification that show satisfactory correspondence to our theorem.

With the rapid development of wireless technologies, many researchers use WiFi signals for human action recognition. Most of the previous methods are based on traditional machine learning algorithms, requiring researchers to manually extract time-domain or frequency-domain features from the obtained WiFi Channel State Information (CSI), but they can not effectively represent the CSI continuous features of human actions. In this paper, we propose an attentional temporal convolutional network (ATCN) for action recognition using CSI. By combining causal convolution and dilated convolution, the proposed ATCN guarantees no CSI features leakage from the future to the past and effectively expands the receptive field to maintain a longer memory size. Meanwhile, the self-attention mechanism is leveraged to assign more weights for the CSI data that is representative of human actions. To better verify and evaluate our proposed model, we build the Njtech-CSI-Action data set and make comparative experiments. The experimental results demonstrate that our Wi-ATCN outperforms previous methods and achieves the state-of-the-art performance.