With the fast development of various positioning techniques such as Global Position System (GPS), mobile devices and remote sensing, spatio-temporal data has become increasingly available nowadays. Mining valuable knowledge from spatio-temporal data is critically important to many real-world applications including human mobility understanding, smart transportation, urban planning, public safety, health care and environmental management. As the number, volume and resolution of spatio-temporal data increase rapidly, traditional data mining methods, especially statistics-based methods for dealing with such data are becoming overwhelmed. Recently deep learning models such as recurrent neural network (RNN) and convolutional neural network (CNN) have achieved remarkable success in many domains due to the powerful ability in automatic feature representation learning, and are also widely applied in various spatio-temporal data mining (STDM) tasks such as predictive learning, anomaly detection and classification. In this paper, we provide a comprehensive review of recent progress in applying deep learning techniques for STDM. We first categorize the spatio-temporal data into five different types, and then briefly introduce the deep learning models that are widely used in STDM. Next, we classify existing literature based on the types of spatio-temporal data, the data mining tasks, and the deep learning models, followed by the applications of deep learning for STDM in different domains including transportation, on-demand service, climate & weather analysis, human mobility, location-based social network, crime analysis, and neuroscience. Finally, we conclude the limitations of current research and point out future research directions.

Urban traffic passenger flows prediction is practically important to facilitate many real applications including transportation management and public safety. Recently, deep learning based approaches are proposed to learn the spatio-temporal characteristics of the traffic passenger flows. However, it is still very challenging to handle some complex factors such as hybrid transportation lines, mixed traffic, transfer stations, and some extreme weathers. Considering the multi-channel and irregularity properties of urban traffic passenger flows in different transportation lines, a more efficient and fine-grained deep spatio-temporal feature learning model is necessary. In this paper, we propose a deep irregular convolutional residual LSTM network model called DST-ICRL for urban traffic passenger flows prediction. We first model the passenger flows among different traffic lines in a transportation network into multi-channel matrices analogous to the RGB pixel matrices of an image. Then, we propose a deep learning framework that integrates irregular convolutional residential network and LSTM units to learn the spatial-temporal feature representations. To fully utilize the historical passenger flows, we sample both the short-term and long-term historical traffic data, which can capture the periodicity and trend of the traffic passenger flows. In addition, we also fuse other external factors further to facilitate a real-time prediction. We conduct extensive experiments on different types of traffic passenger flows datasets including subway, taxi and bus flows in Beijing as well as bike flows in New York. The results show that the proposed DST-ICRL significantly outperforms both traditional and deep learning based urban traffic passenger flows prediction methods.

CNNs, RNNs, GCNs, and CapsNets have shown significant insights in representation learning and are widely used in various text mining tasks such as large-scale multi-label text classification. Most existing deep models for multi-label text classification consider either the non-consecutive and long-distance semantics or the sequential semantics. However, how to coherently take them into account is still far from studied. In addition, most existing methods treat output labels as independent medoids, ignoring the hierarchical relationships among them, which leads to a substantial loss of useful semantic information. In this paper, we propose a novel hierarchical taxonomy-aware and attentional graph capsule recurrent CNNs framework for large-scale multi-label text classification. Specifically, we first propose to model each document as a word order preserved graph-of-words and normalize it as a corresponding word matrix representation preserving both the non-consecutive, long-distance and local sequential semantics. Then the word matrix is input to the proposed attentional graph capsule recurrent CNNs for effectively learning the semantic features. To leverage the hierarchical relations among the class labels, we propose a hierarchical taxonomy embedding method to learn their representations, and define a novel weighted margin loss by incorporating the label representation similarity. Extensive evaluations on three datasets show that our model significantly improves the performance of large-scale multi-label text classification by comparing with state-of-the-art approaches.

Traffic flow prediction has received rising research interest recently since it is a key step to prevent and relieve traffic congestion in urban areas. Existing methods mostly focus on road-level or region-level traffic prediction, and fail to deeply capture the high-order spatial-temporal correlations among the road links to perform a road network-level prediction. In this paper, we propose a network-scale deep traffic prediction model called TrafficGAN, in which Generative Adversarial Nets (GAN) is utilized to predict traffic flows under an adversarial learning framework. To capture the spatial-temporal correlations among the road links of a road network, both Convolutional Neural Nets (CNN) and Long-Short Term Memory (LSTM) models are embedded into TrafficGAN. In addition, we also design a deformable convolution kernel for CNN to make it better handle the input road network data. We extensively evaluate our proposal over two large GPS probe datasets in the arterial road network of downtown Chicago and Bay Area of California. The results show that TrafficGAN significantly outperforms both traditional statistical models and state-of-the-art deep learning models in network-scale short-term traffic flow prediction.

Accurately predicting the urban spatio-temporal data is critically important to various urban computing tasks for smart city related applications such as crowd flow prediction and traffic congestion prediction. Existing models especially deep learning based approaches require a large volume of training data, whose performance may degrade remarkably when the data is scarce. Recent works try to transfer knowledge from the intra-city or cross-city multi-modal spatio-temporal data. However, the careful design of what to transfer and how between the multi-modal spatio-temporal data needs to be determined in advance. There still lacks an end-to-end solution that can automatically capture the common cross-domain knowledge. In this paper, we propose a D eep A ttentive A daptation N etwork model named ST-DAAN to transfer cross-domain S patio- T emporal knowledge for urban crowd flow prediction. ST-DAAN first maps the raw spatio-temporal data of source domain and target domain to a common embedding space. Then domain adaptation is adopted on several domain-specific layers through adding a domain discrepancy penalty to explicitly match the mean embeddings of the two domain distributions. Considering the complex spatial correlation in many urban spatio-temporal data, a global attention mechanism is also designed to enable the model to capture broader spatial dependencies. Using urban crowd flow prediction as a demonstration, we conduct experiments on five real-world large datasets over both intra- and cross-city transfer learning. The results demonstrate that ST-DAAN outperforms state-of-the-art methods by a large margin.

Recently, recommender systems play a pivotal role in alleviating the problem of information overload. Latent factor models have been widely used for recommendation. Most existing latent factor models mainly utilize the interaction information between users and items, although some recently extended models utilize some auxiliary information to learn a unified latent factor for users and items. The unified latent factor only represents the characteristics of users and the properties of items from the aspect of purchase history. However, the characteristics of users and the properties of items may stem from different aspects, e.g., the brand-aspect and category-aspect of items. Moreover, the latent factor models usually use the shallow projection, which cannot capture the characteristics of users and items well. Deep neural network has shown tremendous potential to model the non-linearity relationship between users and items. It can be used to replace shallow projection to model the complex correlation between users and items. In this paper, we propose a Neural network based Aspect-level Collaborative Filtering model (NeuACF) to exploit different aspect latent factors. Through modelling the rich object properties and relations in recommender system as a heterogeneous information network, NeuACF first extracts different aspect-level similarity matrices of users and items, respectively, through different meta-paths, and then feeds an elaborately designed deep neural network with these matrices to learn aspect-level latent factors. Finally, the aspect-level latent factors are fused for the top-N recommendation. Moreover, to fuse information from different aspects more effectively, we further propose NeuACF++ to fuse aspect-level latent factors with self-attention mechanism. Extensive experiments on three real world datasets show that NeuACF and NeuACF++ significantly outperform both existing latent factor models and recent neural network models.