Machine learning (ML)-based Network Intrusion Detection Systems (NIDSs) can classify each network’s flow behavior as benign or malicious by detecting heterogeneous features, including both categorical and numerical features. However, the present ML-based NIDSs are deemed insufficient in terms of their ability to generalize, particularly in changing network environments such as the Internet of Things (IoT)-based smart home. Although IoT devices add so much to home comforts, they also introduce potential risks and vulnerabilities. Recently, many NIDS studies on other IoT scenarios, such as the Internet of Vehicles (IoV) and smart cities, focus on utilizing the telemetry data of IoT devices for IoT intrusion detection. Because when IoT devices are under attack, their abnormal telemetry data values can reflect the anomaly state of those devices. Those telemetry data-based IoT NIDS methods detect intrusion events from a different view, focusing on the attack impact, from the traditional network traffic-based NIDS, which focuses on analyzing attack behavior. The telemetry data-based NIDS is more suitable for IoT devices without built-in security mechanisms. Considering the smart home IoT scenario, which has a smaller scope and a limited number of IoT devices compared to other IoT scenarios, both NIDS views can work independently. This motivated us to propose a novel ML-based NIDS to combine the network traffic-based and telemetry data-based NIDS together. In this paper, we propose a Transformer-based IoT NIDS method to learn the behaviors and effects of attacks from different types of data that are generated in the heterogeneous IoT environment. The proposed method utilizes a self-attention mechanism to learn contextual embeddings for input network features. Based on the contextual embeddings, our method can solve the feature set challenge, including both continuous and categorical features. Our method is the first to utilize both network traffic data and IoT sensors’ telemetry data at the same time for intrusion detection. Experiments reveal the effectiveness of our method on a realistic network traffic intrusion detection dataset named ToN_IoT, with an accuracy of 97.95% for binary classification and 95.78% for multiple classifications on pure network data. With the extra IoT information, the performance of our method has been improved to 98.39% and 97.06%, respectively. A comparative study with existing works shows that our method can achieve state-of-the-art performance on the ToN_IoT dataset.

Adversarial attacks, widely recognized as significant threats to machine learning (ML) models in computer vision and natural language processing, can have more severe consequences when targeting ML-based Network Intrusion Detection Systems (NIDS). These attacks, characterized by data manipulation, necessitate a focused investigation grounded in the unique attributes of the data and practical constraints inherent to the target scenario, as opposed to indiscriminately applying methodologies borrowed from other domains. Since network traffic is complex unstructured data, ML models are commonly used in existing studies to explore how perturbations can defeat ML-based IDS. However, two challenges persist in the realm of traffic-space adversarial attack generation. First, raw traffic data cannot be directly input into ML models. Second, determining the appropriate perturbation scale and direction is challenging, particularly in the case of multi-class NIDS. In this work, we propose a projection-based adversarial attack generation framework, ProGen, to address these two challenges. ProGen is inspired by two observed characteristics of the NIDS scenario: flexible representation and clear objective. ProGen uses a basic feature sequence (BFS) space to represent network traffic in a way that aligns with realistic requirements. To achieve a clear objective, ProGen utilizes a traffic space generative adversarial network (GAN) to approximate distribution mapping between malicious traffic and benign traffic. To better apply the generative model for adversarial attacks, we further design constraints to preserve the functions of the adversarial traffic. We’ve successfully demonstrated the effectiveness of ProGen on six common ML models using the CSE-CIC-IDS2018, CIC-IDS-2017, and UNSW-NB15 datasets; however, we’re yet to validate these findings in real network environments. We visualize the generated distributions of the BFS elements to illustrate the projecting effect under the designed realistic constraints. The results of attack effectiveness tests show that attacks generated from ProGen can significantly reduce the detection performance across different ML models.

Utilizing machine learning (ML)-based approaches for network intrusion detection systems (NIDSs) raises valid concerns due to the inherent susceptibility of current ML models to various threats. Of particular concern are two significant threats associated with ML: adversarial attacks and distribution shifts. Although there has been a growing emphasis on researching the robustness of ML, current studies primarily concentrate on addressing specific challenges individually. These studies tend to target a particular aspect of robustness and propose innovative techniques to enhance that specific aspect. However, as a capability to respond to unexpected situations, the robustness of ML should be comprehensively built and maintained in every stage. In this paper, we aim to link the varying efforts throughout the whole ML workflow to guide the design of ML-based NIDSs with systematic robustness. Toward this goal, we conduct a methodical evaluation of the progress made thus far in enhancing the robustness of the targeted NIDS application task. Specifically, we delve into the robustness aspects of ML-based NIDSs against adversarial attacks and distribution shift scenarios. For each perspective, we organize the literature in robustness-related challenges and technical solutions based on the ML workflow. For instance, we introduce some advanced potential solutions that can improve robustness, such as data augmentation, contrastive learning, and robustness certification. According to our survey, we identify and discuss the ML robustness research gaps and future direction in the field of NIDS. Finally, we highlight that building and patching robustness throughout the life cycle of an ML-based NIDS is critical.

Developing early and transferable Network Intrusion Detection Systems (NIDSs) is essential for robust network security. Early detection prevents further damage, while transferable NIDSs enables reuse across diverse networks. For Machine Learning (ML) and Deep Learning (DL)-based NIDSs, transferability significantly reduces data collection and annotation costs for timely attack mitigation. Current DL-based early intrusion detection studies often focus on identifying attacks from the first few packets, neglecting the crucial aspect of adjustable early detection. Additionally, most DL-based NIDS methods overlook transferability during both design and evaluation phases. To address these limitations, we propose K-GetNID, a knowledge-guided graph learning-based NIDS that excels in both early and transferability. We introduce a Heterogeneous Temporal Graph (HTGraph) to represent the dynamic feature series of network flows, providing enough information for early detection. Additionally, we construct this HTGraph format based on prior knowledge about feature types and correlations to assist the neural network in learning general and transferable knowledge for intrusion detection. We develop a corresponding Heterogeneous Temporal Graph Neural Network (HTGNN) model to learn from the HTGraph format. Furthermore, an Adjustable Early Detection Decoder is designed to enhance the generalization of the proposed model to the input distribution shifts caused by early detection. Experiments on CIC-IDS-2017 and UNSW-NB15 datasets show that K-GetNID matches the performance of deep learning methods, excelling in adjustable early intrusion detection and transferability.

Packet classification is a key function for software-defined networking switches. For example, OpenFlow Switch examines up to 15 required fields, against thousands of rules. With the proliferation of new header fields in a ruleset, it becomes a great challenge to design a high performance packet classification solution. In this paper, we present a scalable many-field packet classification algorithm and its prototype implementation on a graphics processing unit (GPU). By exploiting the sparsity of ruleset, our algorithm uses a few effective bits to divide a large ruleset into multiple subsets with low rule replication for economic memory usage at the offline stage. These effective bits are chosen based on our selection metrics: wildcard ratio, independence index, and diversity index. Using these effective bits, our algorithm can quickly filter out the majority rules which do not need to be full-matched to improve the online system performance. Moreover, the choice of effective bits is adjustable to meet the implementation requirements of varying environments for good performance scalability. Our prototype on a single NVIDIA K20C GPU achieves more than 160 MPPS throughput for 100K 15-field synthetic ruleset.