The string similarity join is a basic operation of many applications that need to find all string pairs from a collection given a similarity function and a user-specified threshold. Recently, there has been considerable interest in designing new algorithms with the assistant of an inverted index to support efficient string similarity joins. These algorithms typically adopt a two-step filter-and-refine approach in identifying similar string pairs: 1) generating candidate pairs by traversing the inverted index; and 2) verifying the candidate pairs by computing the similarity. However, these algorithms either suffer from poor filtering power (which results in high verification cost), or incur too much computational cost to guarantee the filtering power. In this paper, we propose a multiple prefix filtering method based on different global orderings such that the number of candidate pairs can be reduced significantly. We also propose a parallel extension of the algorithm that is efficient and scalable in a MapReduce framework. We conduct extensive experiments on both centralized and Hadoop systems using both real and synthetic data sets, and the results show that our proposed approach outperforms existing approaches in both efficiency and scalability.

Set similarity join is an essential operation in big data analytics, e.g., data integration and data cleaning, that finds similar pairs from two collections of sets. To cope with the increasing scale of the data, distributed algorithms are called for to support large-scale set similarity joins. Multiple techniques have been proposed to perform similarity joins using MapReduce in recent years. These techniques, however, usually produce huge amounts of duplicates in order to perform parallel processing successfully as MapReduce is a shared-nothing framework. The large number of duplicates incurs on both large shuffle cost and unnecessary computation cost, which significantly decrease the performance. Moreover, these approaches do not provide a load balancing guarantee, which results in a skewness problem and negatively affects the scalability properties of these techniques. To address these problems, in this paper, we propose a duplicatefree framework, called FS-Join, to perform set similarity joins efficiently by utilizing an innovative vertical partitioning technique. FS-Join employs three powerful filtering methods to prune dissimilar string pairs without computing their similarity scores. To further improve the performance and scalability, FS-Join integrates horizontal partitioning. Experimental results on three real datasets show that FS-Join outperforms the state-of-theart methods by one order of magnitude on average, which demonstrates the good scalability and performance qualities of the proposed technique.

Considering wireless sensor network characteristics, this paper combines anomaly and mis-use detection and proposes an integrated detection model of cluster-based wireless sensor network, aiming at enhancing detection rate and reducing false rate. Adaboost algorithm with hierarchical structures is used for anomaly detection of sensor nodes, cluster-head nodes and Sink nodes. Cultural-Algorithm and Artificial-Fish–Swarm-Algorithm optimized Back Propagation is applied to mis-use detection of Sink node. Plenty of simulation demonstrates that this integrated model has a strong performance of intrusion detection.

The theory of traditional Chinese medicine (TCM) is an important part of Chinese culture. In the long history, there are a large number of excellent prescriptions, whose laws have been explored by many studies, but few works directly studied the generation of prescriptions. With the rapid development of deep learning, many applications of text generation using neural networks have emerged. Prescriptions are the doctors’ clinical experience and the results of neural networks also come from the accumulated experience. So, it is very feasible to apply deep learning techniques to the recommendation on TCM prescriptions. GAN and its variants have been applied in text generation recently. It has advantages in many aspects, such as the rapid speed of computation, the update of parameters by back propagation without Markov chain and more real data generation with two-players game. We attempted to know the important attributes of prescriptions and use these contents as the training data for variants of GAN to generate prescriptions. Specifically, we attempted to apply SeqGAN (Sequence Generative Adversarial Nets) and CGAN (Conditional Generative Adversarial Nets) to prescription generations. By underlying the knowledge of TCM, the prescriptions with different characteristics can be successfully generated. In the experiments, we conducted the comparative evaluations on the original data with other models. The results showed that applications in the innovation of prescription sequence generations have certain feasibility and significance, even can provide some reference values for the innovation of TCM.

With the rapid development of the aviation industry, it is particularly important to ensure the safe flight of aircraft. How to find potential hazards in the process of aircraft flight has always been one of the important topics of civil aviation research. At present, the Quick Access Recorder (QAR) is the most widely used equipment to store the data recorded on aircraft. QAR data contain a lot of valuable and unexplored information, which records the true status of the aircraft in detail. Therefore, finding abnormal data from QAR data lays an important foundation for obtaining the cause of abnormality and providing a guarantee for flight. In this paper, in order to discover the abnormal information in the QAR data, we applied a VAE-LSTM model with a multihead self-attention mechanism. Compared to the VAE and LSTM models alone, our model performs much better in anomaly detection and prediction, detecting all types of anomalies. We conducted extensive experiments on real-world QAR data sets to prove the efficiency and accuracy of our proposed neural network model. The experimental results proved that our proposed model can outperform state-of-the-art models under different experimental settings.

Flight Operations Quality Assurance (FOQA) is an internationally recognized solution to ensure the safety of civil aircraft flights based on Quick Access Recorder (QAR) data. The traditional approach to anomaly detection in civil aviation is to detect the over-limit values of monitoring parameters for each monitoring event based on the standards issued by civil aviation authorities. Usually, for each anomaly detection operation routine, this only works for one monitoring event. Furthermore, the causal analyses for the detected anomaly events are based on the relevant worker’s expertise. In order to improve the efficiency of FOQA, this paper proposes an automated anomaly detection and causal analysis method called $\mathsf{MAD}\text{-}\mathsf{XFP}$. Due to the unique industry characteristics of QAR data and the requirements of FOQA, feature engineering and hyper-parameter optimization techniques are utilized to enhance the machine learning model. The proposed method can monitor multiple events in one routine and provide a causal analysis. In the causal analysis process, the Shapley additive interpretation method is applied to produce analysis report for detected anomalies. Experimental evaluations are conducted on real civil aviation datasets. The experimental results show that the proposed method can efficiently and automatically detect different abnormal events with high precision in the approach phase and produce preliminary causal analysis.