Intrusion Detection Systems (IDSs) automatically analyze event logs and network traffic in order to detect malicious activity and policy violations. Because IDSs have a large number of false positives and false negatives and the technical nature of their alerts requires a lot of manual analysis, the researchers proposed approaches that automate the analysis of alerts to detect large-scale attacks and predict the attacker’s next steps. Unfortunately, many such approaches use unique datasets and success metrics, making comparison difficult. This survey provides an overview of the state of the art in detecting and projecting cyberattack scenarios, with a focus on evaluation and the corresponding metrics. Representative papers are collected while using Google Scholar and Scopus searches. Mutually comparable success metrics are calculated and several comparison tables are provided. Our results show that commonly used metrics are saturated on popular datasets and cannot assess the practical usability of the approaches. In addition, approaches with knowledge bases require constant maintenance, while data mining and ML approaches depend on the quality of available datasets, which, at the time of writing, are not representative enough to provide general knowledge regarding attack scenarios, so more emphasis needs to be placed on researching the behavior of attackers.

Detecting anomalies in large networks is a major challenge. Nowadays, many studies rely on machine learning techniques to solve this problem. However, much of this research depends on synthetic or limited datasets and tends to use specialized machine learning methods to achieve good detection results. This study focuses on analyzing firewall logs from a large industrial control network and presents a novel method for generating anomalies that simulate real attacker actions within the network without the need for a dedicated testbed or installed security controls. To demonstrate that the proposed method is feasible and that the constructed logs behave as one would expect real-world logs to behave, different supervised and unsupervised learning models were compared using different feature subsets, feature construction methods, scaling methods, and aggregation levels. The experimental results show that unsupervised learning methods have difficulty in detecting the injected anomalies, suggesting that they can be seamlessly integrated into existing firewall logs. Conversely, the use of supervised learning methods showed significantly better performance compared to unsupervised approaches and a better suitability for use in real systems.

Detecting anomalies in large networks is often a difficult task. Nowadays, many works employ machine learning techniques to address this problem, but much of this work often relies on synthetic or small datasets and uses only some specific machine learning techniques. In this research, we focus on analyzing firewall logs obtained from an industrial control network used in a production environment combined with generated anomalies representing real attacker steps in the network. We compared the results of unsupervised learning based on different models, subsets of attributes, feature construction methods, scaling methods, and aggregation levels, while the results of supervised learning were compared by using different classifiers at different aggregation levels. Based on the results of our experiments, we showed that the unsupervised learning method had a difficult task to detect our injected anomalies, which shows us that they are well integrated with the existing firewall logs. On the other hand, the injected anomalies allowed us to use supervised learning methods, and the results showed that using these methods gave much better results.

Information technology system (ITS), informally, is a set of workstations, servers, laptops, installed software, databases, LANs, firewalls, etc. Nowadays, every company has an ITS, but rarely is information about it available outside the company that owns it. However, there are many situations where the availability of such data would be beneficial. For example, cyber ranges emulate IT systems and need their description, and various algorithms in cybersecurity, in particular attack tree generation, need to be validated on models of IT systems. In this paper, we describe a system we call the Generator, that as inputs takes requirements such as the number of employees and the vertical to which the company belongs, and outputs a model of an ITS that satisfies the given requirements. A very important property, that we have put special emphasis on, is that the generated ITS models a large amount of details, and ideally resembles a real system. To the best of our knowledge, we are the first to have attempted to build something like this. We made a proof-of-concept implementation of the Generator, validated it by generating an ITS model for a simplified fictional financial institution, and analyzed its performance with respect to the problem size. The conducted experiments show that our approach is feasible. In the future, we intend to extend this prototype to allow probabilistic generation of IT systems when only a subset of parameters is explicitly defined, and further validate our approach with the help of domain experts.

Recent decades saw the development of a plethora of approaches that aim to use artificial intelligence to detect anomalies and potential signs of compromise in a computer network. These approaches have commonly been trained and evaluated using only a small number of datasets, which were often criticised in literature. Developing new datasets for this purpose tends to be very resource consuming, as they usually rely on testbeds and network emulation. While this level of details is important for anomaly detection over network traffic, which inspects details of network packets, it is superfluous in cases when such algorithms work with logs of security controls, such as in SIEM systems and approaches for alert correlation. Moreover, evaluation over a testbed generated dataset may not be relevant for the target IT system. In this paper, we propose a light-weight method to enrich existing security control logs with carefully crafted synthetic records that would be produced in case of cyber attacks. This method does not require running a dedicated testbed or comparable specialized equipment. We prepare a set of attack records with emphasis on network scans, and perform experiments with real-world firewall logs and several common anomaly detection algorithms to demonstrate that the injected records are appropriately integrated into the original logs. In the end, we propose future experiments to properly validate the quality of the datasets produced using the proposed method.