With the increasing growth of cyber-attack incidences, it is important to develop innovative and effective techniques to assess and defend networked systems against cyber attacks. One of the well-known techniques for this is performing penetration testing which is carried by a group of security professionals (i.e, red team). Penetration testing is also known to be effective to find existing and new vulnerabilities, however, the quality of security assessment can be depending on the quality of the red team members and their time and devotion to the penetration testing. In this paper, we propose a novel automation framework for cyber-attacks generation named ‘HARMer’ to address the challenges with respect to manual attack execution by the red team. Our novel proposed framework, design, and implementation is based on a scalable graphical security model called Hierarchical Attack Representation Model (HARM). (1) We propose the requirements and the key phases for the automation framework. (2) We propose security metrics-based attack planning strategies along with their algorithms. (3) We conduct experiments in a real enterprise network and Amazon Web Services. The results show how the different phases of the framework interact to model the attackers’ operations. This framework will allow security administrators to automatically assess the impact of various threats and attacks in an automated manner.

Cyberspace is full of uncertainty in terms of advanced and sophisticated cyber threats that are equipped with novel approaches to learn the system and propagate themselves, such as AI-powered threats. To debilitate these types of threats, a modern and intelligent Cyber Situation Awareness (SA) system needs to be developed that has the ability of monitoring and capturing various types of threats, analyzing, and devising a plan to avoid further attacks. This article provides a comprehensive study on the current state-of-the-art in the cyber SA to discuss the following aspects of SA: key design principles, framework, classifications, data collection, analysis of the techniques, and evaluation methods. Last, we highlight misconceptions, insights, and limitations of this study and suggest some future work directions to address the limitations.