Dataset quality significantly affects the effectiveness of a machine learning (ML) model in an intrusion detection system (IDS) for cyber-physical industrial control systems (CPS/ICS) and Industrial Internet of Things (IIoT). Existing surveys compare datasets qualitatively or along limited dimensions, whereas this review introduces quantitative documentation and decision-aid scoring across 23 ICS/OT/IIoT datasets. These datasets are analyzed along seven measurable axes, with their attacks mapped to MITRE ATT&CK for ICS tactics. Quantitatively, 14 of the 23 datasets (60.9%) are built on physical testbeds, and 22 of the 23 map to MITRE ATT&CK for ICS, spanning 11 of the 12 tactics. We introduce a checklist for documentation completeness (0–7) and a decision-aid rubric (0–15) covering realism, attack diversity, class imbalance, documentation, and reproducibility. Protocol coverage across these datasets is skewed toward Modbus (13 of 23 datasets, 57%), while many other protocols (such as Profinet and OPC UA) are underrepresented relative to their industry deployment. The available datasets show structural gaps in capturing multi-stage adversary behavior. In practice, dataset selection should pair a realism-anchored dataset with a high-reproducibility one, and account for protocol diversity and APT representation.

Dataset quality significantly affects the effectiveness of a machine learning (ML) model for an intrusion detection system (IDS) for cyber-physical industrial control systems (CPS/ICS) and Industrial Internet of Things (IIoT). Existing surveys compare datasets qualitatively or along limited dimensions, while this review introduces quantitative documentation and decision-aid scoring across 23 ICS/OT/IIoT datasets. These datasets are analyzed along seven measurable axes, with their attacks mapped to MITRE ATT&CK for ICS tactics. We introduce a checklist for documentation completeness (0–7) and a decision-aid rubric (0–15) covering realism, attack diversity, class imbalance, documentation, and reproducibility. Protocol coverage in datasets is skewed toward Modbus, while many other protocols (such as Profinet and OPC UA) are underrepresented relative to their deployment in industry. Available datasets show structural gaps in capturing multi-stage adversary behavior. In practice, dataset selection should pair a realism-anchored aspect with a high-reproducibility one, and account for protocol diversity and APT representation.

An explanation of an industrial intrusion detection can be trusted only as far as the chain behind it allows, namely the dataset, the detector, and the explanation. Four research questions test this chain on combinations of seven variants of industrial control system and industrial Internet of Things (ICS/IIoT) datasets with three model families (XGBoost, CNN, logistic regression), using two explainers, the KernelSHAP and the LIME. We find that the detector is worth explaining in 19 of 21 combinations. Each explainer mostly repeats its answer, where a new seed keeps about 9 of the top 10 features for KernelSHAP and 8 for LIME. The two explainers agree more than random feature lists would on 18 of 19 combinations. Yet, although both explainers are local, little of this agreement appears to be instance-specific, since lists for two different instances overlap nearly as much as lists for the same instance. In process datasets, where we injected attacks into known variables, their agreement showed no clear link to KernelSHAP being right, so agreement does not imply correctness. We propose four checks per explanation, one per research question. For reproducibility, the settings for each explanation are also reported.