Selecting the best configuration of hyperparameter values for a Machine Learning model yields directly in the performance of the model on the dataset. It is a laborious task that usually requires deep knowledge of the hyperparameter optimizations methods and the Machine Learning algorithms. Although there exist several automatic optimization techniques, these usually take significant resources, increasing the dynamic complexity in order to obtain a great accuracy. Since one of the most critical aspects in this computational consume is the available dataset, among others, in this paper we perform a study of the effect of using different partitions of a dataset in the hyperparameter optimization phase over the efficiency of a Machine Learning algorithm. Nonparametric inference has been used to measure the rate of different behaviors of the accuracy, time, and spatial complexity that are obtained among the partitions and the whole dataset. Also, a level of gain is assigned to each partition allowing us to study patterns and allocate whose samples are more profitable. Since Cybersecurity is a discipline in which the efficiency of Artificial Intelligence techniques is a key aspect in order to extract actionable knowledge, the statistical analyses have been carried out over five Cybersecurity datasets.
2 September 2022 in Expert Systems with Applications
The detection, treatment and prediction of cybersecurity events is a priority research area for the security response centers. In this scenario, the automation of several processes is essential in order to manage and solve, in an efficient way, the possible damaged caused by the threats and attacks to the institutions, citizens and companies. In this work, automatic multi‐class categorization models are created by using machine learning techniques in order to assign the severity for different types of cybersecurity events. Once the machine learning algorithms are applied, we propose a mathematical formula to decide, automatically and based on a scoring system, which model is the best for each type of event. Also, we present the scoring system applied over a real cybersecurity events data store, then the features are extracted from the usual registers that are collected in a center of response to cyber‐events. The results show that the ensemble methods are the most suitable for obtaining more than 99% of accuracy.