Artificial intelligence, big data, machine learning, cloud computing and Internet of Things (IoT) are terms which have driven the 4th industrial revolution. The digital revolution has transformed the manufacturing industry into smart manufacturing through the development of intelligent systems. This paper presents a big data ecosystem for the implementation of fault detection and diagnosis in predictive maintenance with real industrial big data gathered directly from large-scale global manufacturing plants, aiming to provide a complete architecture which could be used in industrial IoT-based smart manufacturing in an industrial 4.0 system. The proposed architecture overcomes multiple challenges including big data ingestion, integration, transformation, storage, analytics and visualization in a real-time environment using various technologies such as the data lake, NoSQL database, Apache Spark, Apache Drill, Apache Hive, OPC Collector and other techniques. Transformation protocols, authentication and data encryption methods are also utilized to address data and network security issues. A MapReducebased distributed PCA model is designed for fault detection and diagnosis. In a large-scale manufacturing system, not all kinds of failure data are accessible, and the absence of labels precludes all the supervised methods in the predictive phase. Furthermore, the proposed framework takes advantage of some of the characteristics of PCA such as its ease of implementation on Spark, its simple algorithmic structure, and its real-time processing ability. All these elements are essential for smart manufacturing in the evolution to Industry 4.0. The proposed detection system has been implemented into the real-time industrial production system in a cooperated company, running for several years, and the results successfully provide an alarm warning several days before the fault happens. A test case involving several outages in 2014 is reported and analyzed in detail during the experiment section.

Traditional Intrusion Detection Systems (IDS) cannot cope with the increasing number and sophistication of cyberattacks such as Advanced Persistent Threats (APT) . Due to their high false-positive rate and the required effort of security experts to validate them, incidents can remain undetected for up to several months. As a result, enterprises suffer from data loss and severe financial damage. Recent research explored data provenance for Host-based Intrusion Detection Systems (HIDS) as one promising data source to tackle this issue. Data provenance represents information flows between system entities as Direct Acyclic Graph (DAG) . Provenance-based Intrusion Detection Systems (PIDS) utilize data provenance to enhance the detection performance of intrusions and reduce false-alarm rates compared to traditional IDS. This survey demonstrates the potential of PIDS by providing a detailed evaluation of recent research in the field, proposing a novel taxonomy for PIDS, discussing current issues, and potential future research directions. This survey aims to help and motivate researchers to get started in the field of PIDS by tackling issues of data collection, graph summarization, intrusion detection, and developing real-world benchmark datasets.

In recent years, the Internet of Things (IoT)-oriented smart manufacturing has become a prominent solution in realizing evolutional digital transformation. Missing data is one of the biggest problems for data preprocessing in an IoT architecture, and it is crucial that missing values are recovered to improve the reliability of monitoring applications. However, due to the high-frequency collection of sensor data, missing data in IoT brings new challenges. Several methods have been developed to recover missing IoT data by utilizing data from sensors which are geographically close to the sensor which is responsible for the missing data, or from sensors which provide data which is highly correlated to the missing data. In IoT systems, because of the transmission of a large volume of data over networks, common mode failures need to be considered where a single event can lead to the loss of data from a large number of sensors. In this situation, it would be infeasible to recover missing data from other sensors. To address this issue, in this paper, we focus on missing data imputation for large gaps in univariate time series data and propose an iterative framework using multiple segmented gap iteration called Itr-MS-STLecImp to provide the most appropriate values. The gap is firstly segmented into several pieces to initialize the missing value imputation process and then we iteratively run gap reconstruction and gap concatenation to obtain the final imputation results. We validate the proposed approach using sensor data collected from real manufacturing plants in Australia and the comparison results show that the proposed Itr-MS-STLecImp outperforms the state-of-the-art methods in terms of root mean square error. Under different gap-length conditions, the proposed approach consistently reduces the error rate more than the baseline algorithm, and the error reduction is greater when the lengths of the gaps increase, indicating that the performance is significantly improved. These analysis results further prove the effectiveness of the multiple segmentation of missing gaps and the iteration operation.

The Industrial Internet of Things (IIoT) enables intelligent predictive maintenance in smart manufacturing by incorporating IoT technologies, Big Data techniques, artificial intelligence, cloud computing, and other ever-developing enabling technologies. Although a large body of research has been conducted on IIoT based predictive maintenance, most work focuses on addressing only a part of the problem. However, predictive maintenance involves an ecosystem from ingesting data from sensors to displaying the results on a dashboard for engineers to visualize. With increasing requirements for real-time responses and privacy, integrating edge computing is no doubt a promising trend. This article proposes a complete and optimized IoT Big Data ecosystem embedded into a three-layer architecture for predictive maintenance applications. The proposed architecture consists of an edge layer, a cloud layer, and an application layer. The proposed edge infrastructure distributes the tasks effectively between the cloud layer and edge layer. On top of the architecture, different layers are integrated seamlessly to address reliability and scalability issues. In addition, an edge computing-assisted autoencoder is introduced and enabled by being deployed in a distributed manner to improve both performance and efficiency. For practical interest, we also provide an application programming interface-oriented implementation guideline for readers. To verify the proposed ecosystem, a real case study from industry is conducted to demonstrate the performance gain of edge computing-based Internet of Things systems in conjunction with the autoencoder-based deep learning technique.

The emergence of Industry 4.0 and the rapid advances in the Industrial Internet of Things (IIoT) have provided manufacturers with the ability to remotely monitor the process by deploying automatic fault detection in an IoT-based predictive maintenance system. However, the monitoring targets are now manufacturing plant-wide instead of being just a local area. Multiple types of faults are involved and the conventional centralized cloud computing-based IoT solutions always lead to a heavy burden on the network bandwidth due to the large amount of sensor data collected frequently that has to be transmitted to the central server and this leads to poor response time for the monitoring system. To address this problem, this article develops an artificial intelligence-assisted distributed system for manufacturing plant-wide predictive maintenance applications. The developed distributed system relies on the feature selection technique to identity an optimal feature subset for each type of fault and is enabled by deploying each independent model built on the obtained feature subset into different edge nodes. The distributed approach enables the data to be processed near the sensors, requiring less data to be transmitted to the central cloud server reducing network delay and delivering more accurate results. In addition, our proposed feature selection approach is especially designed to accommodate the characteristics of IIoT data such as the lack of labels. The effectiveness of the proposed method is validated using the widely used public Tennessee Eastman dataset.

With the rapid growth in the use of various smart digital sensors, the Internet of Things (IoT) is a swiftly growing technology, which has contributed significantly to Industry 4.0 and the promotion of IoT-based smart factories, which gives rise to the new challenges of big data analytics and the implementation of machine learning techniques. This article proposes a practical framework that combines IoT techniques, a data lake, data analysis, and cloud computing for manufacturing equipment health-state monitoring and diagnostics in smart manufacturing. It addresses all the required aspects in the realization of such a system and allows the seamless interchange of data and functionality. Due to the specific characteristics of IoT sensor data (low quality, redundant multisources, partial labeling), we not only provide a promising framework but also give detailed insights and pay considerable attention to data quality issues. In the proposed framework, an ingestion procedure is designed to manage data collection, data security, data transformation and data storage issues. To improve the quality of IoT big data, a high-noise feature filter is proposed for automated preliminary sensor selection to suppress noisy features, followed by a noisy data cleaning module to provide good quality data for unbiased diagnosis modeling. The proposed framework can achieve seamless integration between IoT big data ingestion from the physical factory and machine learning-based data analytics in the virtual systems. It is built on top of the Apache Spark processing engine, being capable of working in both big data and real-time environments. One case study has been conducted based on a four-stage syngas compressor from real industries, which won the Best Industry Application of IoT at the BigInsights Data & AI Innovation Awards. The experimental results demonstrate the effectiveness of both the proposed IoT-architecture and techniques to address the data quality issues.