Using unsupervised machine learning models for anomaly detection is a common thing nowadays. While much research focuses on improving and testing these models, there is a lack of those that focus on threshold selection, which is an important step in implementing a good anomaly detection system. In this paper, we examine several supervised and unsupervised threshold selection methods found in the network anomaly detection literature. A total of five supervised and twenty unsupervised methods were found, all of which are described, categorized, and implemented in this paper. The unsupervised methods were further categorized by the input data they expect, the type of output data they produce, and whether or not they are parametric, and divided into six groups according to the idea behind these methods: Statistics-based, Distribution-based, Clustering-based, Density-based, Graphical-based methods and Other. In order to test all the methods found, two different testing scenarios are created. The first one focuses on using data with anomalies and the second one focuses only on the normal data. Based on these two scenarios, tests were performed on real firewall log data that contained three types of injected anomalies. The results are presented in the form of boxplots of the Matthews correlation coefficient for nine datasets. To draw a conclusion, both the method groups and the individual methods were compared in terms of evaluation metrics and execution times, and in comparison to the methods already implemented in the PyThresh toolkit.

Detecting anomalies in large networks is a major challenge. Nowadays, many studies rely on machine learning techniques to solve this problem. However, much of this research depends on synthetic or limited datasets and tends to use specialized machine learning methods to achieve good detection results. This study focuses on analyzing firewall logs from a large industrial control network and presents a novel method for generating anomalies that simulate real attacker actions within the network without the need for a dedicated testbed or installed security controls. To demonstrate that the proposed method is feasible and that the constructed logs behave as one would expect real-world logs to behave, different supervised and unsupervised learning models were compared using different feature subsets, feature construction methods, scaling methods, and aggregation levels. The experimental results show that unsupervised learning methods have difficulty in detecting the injected anomalies, suggesting that they can be seamlessly integrated into existing firewall logs. Conversely, the use of supervised learning methods showed significantly better performance compared to unsupervised approaches and a better suitability for use in real systems.

Detecting anomalies in large networks is often a difficult task. Nowadays, many works employ machine learning techniques to address this problem, but much of this work often relies on synthetic or small datasets and uses only some specific machine learning techniques. In this research, we focus on analyzing firewall logs obtained from an industrial control network used in a production environment combined with generated anomalies representing real attacker steps in the network. We compared the results of unsupervised learning based on different models, subsets of attributes, feature construction methods, scaling methods, and aggregation levels, while the results of supervised learning were compared by using different classifiers at different aggregation levels. Based on the results of our experiments, we showed that the unsupervised learning method had a difficult task to detect our injected anomalies, which shows us that they are well integrated with the existing firewall logs. On the other hand, the injected anomalies allowed us to use supervised learning methods, and the results showed that using these methods gave much better results.

In this paper, we present research on the analysis of the design space for cybersecurity visualizations in VizSec. At the beginning of this research, we analyzed 17 survey papers in the field of cybersecurity visualization. Based on the analysis of the focus areas in each of these survey papers, we identified five key components of visualization design, i.e. Input Data, Security Tasks, Visual Encoding, Interactivity, and Evaluation. To show how research papers align with these components, we analyzed 60 papers published at the IEEE Symposium on Visualization for Cyber Security (VizSec) between 2016 and 2021 in the context of the five identified components. As a result, each research paper was classified into several categories derived from the selected components of the visualization design. Our contributions are: (i) an analysis of the focus areas in survey papers on cybersecurity visualization and (ii) the classification of 60 research papers in the context of the selected components of the visualization design. Finally, we highlighted the main findings of the analysis and drew conclusions.

Recent decades saw the development of a plethora of approaches that aim to use artificial intelligence to detect anomalies and potential signs of compromise in a computer network. These approaches have commonly been trained and evaluated using only a small number of datasets, which were often criticised in literature. Developing new datasets for this purpose tends to be very resource consuming, as they usually rely on testbeds and network emulation. While this level of details is important for anomaly detection over network traffic, which inspects details of network packets, it is superfluous in cases when such algorithms work with logs of security controls, such as in SIEM systems and approaches for alert correlation. Moreover, evaluation over a testbed generated dataset may not be relevant for the target IT system. In this paper, we propose a light-weight method to enrich existing security control logs with carefully crafted synthetic records that would be produced in case of cyber attacks. This method does not require running a dedicated testbed or comparable specialized equipment. We prepare a set of attack records with emphasis on network scans, and perform experiments with real-world firewall logs and several common anomaly detection algorithms to demonstrate that the injected records are appropriately integrated into the original logs. In the end, we propose future experiments to properly validate the quality of the datasets produced using the proposed method.