The exponential growth of network intrusions necessitates the development of advanced artificial intelligence (AI) techniques for intrusion detection systems (IDSs). However, the reliance on AI for IDSs presents several challenges, including the performance variability of different AI models and the opacity of their decision-making processes, hindering comprehension by human security analysts. In response, we propose an end-to-end explainable AI (XAI) framework tailored to enhance the interpretability of AI models in network intrusion detection tasks. Our framework commences with benchmarking seven black-box AI models across three real-world network intrusion datasets, each characterized by distinct features and challenges. Subsequently, we leverage various XAI models to generate both local and global explanations, shedding light on the underlying rationale behind the AI models’ decisions. Furthermore, we employ feature extraction techniques to discern crucial model-specific and intrusion-specific features, aiding in understanding the discriminative factors influencing the detection outcomes. Additionally, our framework identifies overlapping and significant features that impact multiple AI models, providing insights into common patterns across different detection approaches. Notably, we demonstrate that the computational overhead incurred by generating XAI explanations is minimal for most AI models, ensuring practical applicability in real-time scenarios. By offering multi-faceted explanations, our framework equips security analysts with actionable insights to make informed decisions for threat detection and mitigation. To facilitate widespread adoption and further research, we have made our source code publicly available, serving as a foundational XAI framework for IDSs within the research community.

The exponential growth of intrusions on networked systems inspires new research directions on developing artificial intelligence (AI) techniques for intrusion detection systems (IDS). In particular, the need to understand and explain these AI models to security analysts (managing these IDS to safeguard their networks) motivates the usage of explainable AI (XAI) methods in real-world IDS. In this work, we propose an end-to-end framework to evaluate black-box XAI methods for network IDS. We evaluate both global and local scopes for these black-box XAI methods for network intrusion detection. We analyze six different evaluation metrics for two popular black-box XAI techniques, namely SHAP and LIME. These metrics are descriptive accuracy, sparsity, stability, efficiency, robustness, and completeness. They cover main metrics from network security and AI domains. We evaluate our XAI evaluation framework using three popular network intrusion datasets and seven AI methods with different characteristics. We release our codes for the network security community to access it as a baseline XAI framework for network IDS. Our framework shows the limitations and strengths of current black-box XAI methods when applied to network IDS.

The exponential growth of Internet of Things (IoT) systems inspires new research directions on developing artificial intelligence (AI) techniques for detecting anomalies in these IoT systems. One important goal in this context is to accurately detect and anticipate anomalies (or failures) in IoT devices and identify main characteristics for such anomalies to reduce maintenance cost and minimize downtime. In this paper, we propose an explainable AI (XAI) framework for enhancing anomaly detection in IoT systems. Our framework has two main components. First, we propose AI-based anomaly detection of IoT systems where we adapt two classes of AI methods (single AI methods, and ensemble methods) for anomaly detection in smart IoT systems. Such anomaly detection aims at detecting anomaly data (from deployed sensors or network traffic between IoT devices). Second, we conduct feature importance analysis to identify the main features that can help AI models identify anomalies in IoT systems. For this feature analysis, we use seven different XAI methods for extracting important features for different AI methods and different attack types. We test our XAI framework for anomaly detection through two real-world IoT datasets. The first dataset is collected from IoT-based manufacturing sensors and the second dataset is collected from IoT botnet attacks. For the IoT-based manufacturing dataset, we detect the level of defect for data from IoT sensors. For the IoT botnet attack dataset, we detect different attack classes from different kinds of botnet attacks on the IoT network. For both datasets, we provide extensive feature importance analysis using different XAI methods for our different AI models to extract the top features. We release our codes for the community to access it for anomaly detection and feature analysis for IoT systems and to build on it with new datasets and models. Taken together, we show that accurate anomaly detection can be achieved along with understanding top features that identify anomalies, paving the way for enhancing anomaly detection in IoT systems.

The advent of autonomous driving systems has given rise to pressing cybersecurity issues regarding the vulnerability of vehicular ad hoc networks (VANETs) to potential attacks. This critical security problem necessitates the application of artificial intelligence (AI) models for anomaly detection in VANETs of autonomous vehicles (AVs). However, the lack of explainability of such AI-based anomaly detection models presents challenges. This motivates an emerging research direction of utilizing explainable AI (XAI) techniques to elucidate the behaviors of anomaly detection models in AV networks. In this work, we propose an end-to-end XAI framework to interpret and visualize the anomaly detection classifications made by AI models securing VANETs. We evaluate the framework on two real-world autonomous driving datasets. The framework furnishes both global and local explanations for the black-box AI models using two XAI methods. Moreover, we introduce two novel feature selection techniques to identify the salient features contributing to anomaly detection, derived from the popular SHAP XAI method and the accuracy of six different black-box AI models.We compare our proposed feature selection approaches with six state-of-the-art feature selection techniques (including two wrapper-based feature selection methods), demonstrating superior performance on various evaluation metrics. To generalize the impact of our feature selection methods, we apply three independent classifiers to evaluate our proposed feature selection approaches. The novel feature selection methods effectively distill themost explanatory features, enhancing model interpretability. Finally, we assess the efficiency (how quickly the XAI models can yield explanatory findings) for each of the six black-box AI models we employed on our two datasets, identifying the most efficient model. By furnishing explanations and visualizations of anomaly detection by AI models, our XAI framework can help in enabling trust and transparency for securing vehicular networks.

Smart manufacturing systems are considered the next generation of manufacturing applications. One important goal of the smart manufacturing system is to rapidly detect and anticipate failures to reduce maintenance cost and minimize machine downtime. This often boils down to detecting anomalies within the sensor data acquired from the system which has different characteristics with respect to the operating point of the environment or machines, such as, the RPM of the motor. In this paper, we analyze four datasets from sensors deployed in manufacturing testbeds. We detect the level of defect for each sensor data leveraging deep learning techniques. We also evaluate the performance of several traditional and ML-based forecasting models for predicting the time series of sensor data. We show that careful selection of training data by aggregating multiple predictive RPM values is beneficial. Then, considering the sparse data from one kind of sensor, we perform transfer learning from a high data rate sensor to perform defect type classification. We release our manufacturing database corpus (4 datasets) and codes for anomaly detection and defect type classification for the community to build on it. Taken together, we show that predictive failure classification can be achieved, paving the way for predictive maintenance.

We consider a system consisting of multiple interdependent assets, and a set of defenders, each responsible for securing a subset of the assets against an attacker. The interdependencies between the assets are captured by an attack graph, where an edge from one asset to another indicates that if the former asset is compromised, an attack can be launched on the latter asset. Each edge has an associated probability of successful attack, which can be reduced via security investments by the defenders. In such scenarios, we investigate the security investments that arise under certain features of human decision-making that have been identified in behavioral economics. In particular, humans have been shown to perceive probabilities in a nonlinear manner, typically overweighting low probabilities and underweighting high probabilities. We show that suboptimal investments can arise under such weighting in certain network topologies. We also show that pure strategy Nash equilibria exist in settings with multiple (behavioral) defenders, and study the inefficiency of the equilibrium investments by behavioral defenders compared to a centralized socially optimal solution.