The National Cyber Range (NCR) is an innovative Department of Defense (DoD) resource originally established by the Defense Advanced Research Projects Agency (DARPA) and now under the purview of the Test Resource Management Center (TRMC). It provides a unique environment for cyber security testing throughout the program development life cycle using unique methods to assess resiliency to advanced cyberspace security threats. This paper describes what a cyber security range is, how it might be employed, and the advantages a program manager (PM) can gain in applying the results of range events. Creating realism in a test environment isolated from the operational environment is a special challenge in cyberspace. Representing the scale and diversity of the complex DoD communications networks at a fidelity detailed enough to realistically portray current and anticipated attack strategies (e.g., Malware, distributed denial of service attacks, cross-site scripting) is complex. The NCR addresses this challenge by representing an Internet-like environment by employing a multitude of virtual machines and physical hardware augmented with traffic emulation, port/protocol/service vulnerability scanning, and data capture tools. Coupled with a structured test methodology, the PM can efficiently and effectively engage with the Range to gain cyberspace resiliency insights. The NCR capability, when applied, allows the DoD to incorporate cyber security early to avoid high cost integration at the end of the development life cycle. This paper provides an overview of the resources of the NCR which may be especially helpful for DoD PMs to find the best approach for testing the cyberspace resiliency of their systems under development.

There is an increasing demand for processing large volumes of unstructured data for a wide variety of applications. However, protection measures for these big data sets are still in their infancy, which could lead to significant security and privacy issues. Attribute-based access control (ABAC) provides a dynamic and flexible solution that is effective for mediating access. We analyzed and implemented a prototype application of ABAC to large dataset processing in Amazon Web Services, using open-source versions of Apache Hadoop, Ranger, and Atlas. The Hadoop ecosystem is one of the most popular frameworks for large dataset processing and storage and is adopted by major cloud service providers. We conducted a rigorous analysis of cybersecurity in implementing ABAC policies in Hadoop, including developing a synthetic dataset of information at multiple sensitivity levels that realistically represents healthcare and connected social media data. We then developed Apache Spark programs that extract, connect, and transform data in a manner representative of a realistic use case. Our result is a framework for securing big data. Applying this framework ensures that serious cybersecurity concerns are addressed. We provide details of our analysis and experimentation code in a GitHub repository for further research by the community.

There is a lack of scientific testing of commercially available malware detectors, especially those that boast accurate classification of never-before-seen (i.e., zero-day) files using machine learning (ML). Consequently, efficacy of malware detectors is opaque, inhibiting end users from making informed decisions and researchers from targeting gaps in current detectors. In this article, we present a scientific evaluation of four prominent commercial malware detection tools to assist an organization with two primary questions: To what extent do ML-based tools accurately classify previously and never-before-seen files? Is purchasing a network-level malware detector worth the cost? To investigate, we tested each tool against 3,536 total files (2,554 or 72% malicious and 982 or 28% benign) of a variety of file types, including hundreds of malicious zero-days, polyglots, and APT-style files, delivered on multiple protocols. We present statistical results on detection time and accuracy, consider complementary analysis (using multiple tools together), and provide two novel applications of the recent cost–benefit evaluation procedure of Iannacone and Bridges. Although the ML-based tools are more effective at detecting zero-day files and executables, the signature-based tool might still be an overall better option. Both network-based tools provide substantial (simulated) savings when paired with either host tool, yet both show poor detection rates on protocols other than HTTP or SMTP. Our results show that all four tools have near-perfect precision but alarmingly low recall, especially on file types other than executables and office files: Thirty-seven percent of malware, including all polyglot files, were undetected. Priorities for researchers and takeaways for end users are given. Code for future use of the cost model is provided.

Data intensive computing research and technology developments offer the potential of providing significant improvements in several security log management challenges. Approaches to address the complexity, timeliness, expense, diversity, and noise issues have been identified. These improvements are motivated by the increasingly important role of analytics. Machine learning and expert systems that incorporate attack patterns are providing greater detection insights. Finding actionable indicators requires the analysis to combine security event log data with other network data such and access control lists, making the big-data problem even bigger. Automation of threat intelligence is recognized as not complete with limited adoption of standards. With limited progress in anomaly signature detection, movement towards using expert systems has been identified as the path forward. Techniques focus on matching behaviors of attackers to patterns of abnormal activity in the network. The need to stream, parse, and analyze large volumes of small, semi-structured data files can be feasibly addressed through a variety of techniques identified by researchers. This report highlights research in key areas, including protection of the data, performance of the systems and network bandwidth utilization.

This paper provides an overview of the security service controls that are applied in a big data processing (BDP) system to defend against cyber security attacks. We validate this approach by modeling attacks and effectiveness of security service controls in a sequence of states and transitions. This Finite State Machine (FSM) approach uses the probable effectiveness of security service controls, as defined in the National Institute of Standards and Technology (NIST) Risk Management Framework (RMF). The attacks used in the model are defined in the ATT&CK™ framework. Five different BDP security architecture configurations are considered, spanning from a low-cost default BDP configuration to a more expensive, industry supported layered security architecture. The analysis demonstrates the importance of a multi-layer approach to implementing security in BDP systems. With increasing interest in using BDP systems to analyze sensitive data sets, it is important to understand and justify BDP security architecture configurations with their significant costs. The output of the model demonstrates that over the run time, larger investment in security service controls results in significantly more uptime. There is a significant increase in uptime with a linear increase in security service control investment. We believe that these results support our recommended BDP security architecture. That is, a layered architecture with security service controls integrated into the user interface, boundary, central management of security policies, and applications that incorporate privacy preserving programs. These results enable making BDP systems operational for sensitive data accessed in a multi-tenant environment.

In this paper, we present a scientific evaluation of four prominent malware detection tools to assist an organization with two primary questions: To what extent do ML-based tools accurately classify previously- and never-before-seen files? Is it worth purchasing a network-level malware detector? To identify weaknesses, we tested each tool against 3,536 total files (2,554 or 72\% malicious, 982 or 28\% benign) of a variety of file types, including hundreds of malicious zero-days, polyglots, and APT-style files, delivered on multiple protocols. We present statistical results on detection time and accuracy, consider complementary analysis (using multiple tools together), and provide two novel applications of the recent cost-benefit evaluation procedure of Iannacone \& Bridges. While the ML-based tools are more effective at detecting zero-day files and executables, the signature-based tool may still be an overall better option. Both network-based tools provide substantial (simulated) savings when paired with either host tool, yet both show poor detection rates on protocols other than HTTP or SMTP. Our results show that all four tools have near-perfect precision but alarmingly low recall, especially on file types other than executables and office files -- 37% of malware tested, including all polyglot files, were undetected. Priorities for researchers and takeaways for end users are given.