Social media datasets have been widely used in disaster assessment and management. When a disaster occurs, many users post messages in a variety of formats, e.g., image and text, on social media platforms. Useful information could be mined from these multimodal data to enable situational awareness and to support decision making during disasters. However, the multimodal data collected from social media contain a lot of irrelevant and misleading content that needs to be filtered out. Existing work has mostly used unimodal methods to classify disaster messages. In other words, these methods treated the image and textual features separately. While a few methods adopted multimodality to deal with the data, their accuracy cannot be guaranteed. This research seamlessly integrates image and text information by developing a multimodal fusion approach to identify useful disaster images collected from social media platforms. In particular, a deep learning method is used to extract the visual features from social media, and a FastText framework is then used to extract the textual features. Next, a novel data fusion model is developed to combine both visual and textual features to classify relevant disaster images. Experiments on a real-world disaster dataset, CrisisMMD, are performed, and the validation results demonstrate that the method consistently and significantly outperforms the previously published state-of-the-art work by over 3%, with a performance improvement from 84.4% to 87.6%.

We propose a Multimodal Machine Learning method for estimating the Photometric Redshifts of quasars (PhotoRedshift-MML for short), which has long been the subject of many investigations. Our method includes two main models, i.e. the feature transformation model by multimodal representation learning, and the photometric redshift estimation model by multimodal transfer learning. The prediction accuracy of the photometric redshift was significantly improved owing to the large amount of information offered by the generated spectral features learned from photometric data via the MML. A total of 415,930 quasars from Sloan Digital Sky Survey (SDSS) Data Release 17, with redshifts between 1 and 5, were screened for our experiments. We used |Δz| = |(z_phot − z_spec)/(1 + z_spec)| to evaluate the redshift prediction and demonstrated a |$4.04\%$| increase in accuracy. With the help of the generated spectral features, the proportion of data with |Δz| < 0.1 can reach |$84.45\%$| of the total test samples, whereas it reaches |$80.41\%$| for single-modal photometric data. Moreover, the Root Mean Square (RMS) of |Δz| is shown to decreases from 0.1332 to 0.1235. Our method has the potential to be generalized to other astronomical data analyses such as galaxy classification and redshift prediction.

With the development of urban science, researches on mining of urban big data have attracted more and more attention. One typical microcosm of urban big data is taxi trajectory data. Predicting the travel time between the two specified points accurately is great significance for applications, such as travel plan. However, the current approach just uses limited modality data or single model without considering their one-sidedness. This paper puts forward to one optimized method to estimate travel time, which is based on ensemble method with multi-modality urban big data, namely Travel Time Estimation-Ensemble (TTE-Ensemble). First, we extract the feature sub-vectors from the multi-modality data as the model input. Then we use the gradient boosting decision tree (GBDT) model to process the low dimensional simple features and adopt the deep neural network (DNN) model to handle high dimensional underlying features. Finally, the ensemble method was introduced to integrate the two model of GBDT and the DNN. Extensive experiments were conducted based on real datasets of origin-destination points in Chengdu and Shanghai, China. These experiments demonstrate the superiority of the TTE-Ensemble model.

We address the problem of data acquisition in large distributed wireless sensor networks (WSNs). We propose a method for data acquisition using the hierarchical routing method and compressive sensing for WSNs. Only a few samples are needed to recover the original signal with high probability since sparse representation technology is exploited to capture the similarities and differences of the original signal. To collect samples effectively in WSNs, a framework for the use of the hierarchical routing method and compressive sensing is proposed, using a randomized rotation of cluster-heads to evenly distribute the energy load among the sensors in the network. Furthermore, L1-minimization and Bayesian compressed sensing are used to approximate the recovery of the original signal from the smaller number of samples with a lower signal reconstruction error. We also give an extensive validation regarding coherence, compression rate, and lifetime, based on an analysis of the theory and experiments in the environment with real world signals. The results show that our solution is effective in a large distributed network, especially for energy constrained WSNs.

Location-based social networks (LBSN) allow users to socialize with friends by sharing their daily life experiences online. In particular, a large amount of check-ins data generated by LBSNs capture the visit locations of users and open a new line of research of spatio-temporal big data, i.e., the next point-of-interest (POI) recommendation. At present, while some advanced methods have been proposed for POI recommendation, existing work only leverages the temporal information of two consecutive LBSN check-ins. Specifically, these methods only focus on adjacent visit sequences but ignore non-contiguous visits, while these visits can be important in understanding the spatio-temporal correlation within the trajectory. In order to fully mine this non-contiguous visit information, we propose a multi-layer Spatio-Temporal deep learning attention model for POI recommendation, Spatio-Temporal Transformer Recommender (STTF-Recommender). To incorporate the spatio-temporal patterns, we encode the information in the user’s trajectory as latent representations into their embeddings before feeding them. To mine the spatio-temporal relationship between any two visited locations, we utilize the Transformer aggregation layer. To match the most plausible candidates from all locations, we develop on an attention matcher based on the attention mechanism. The STTF-Recommender was evaluated with two real-world datasets, and the findings showed that STTF improves at least 13.75% in the mean value of the Recall index at different scales compared with the state-of-the-art models.