What makes an industrial control system security testbed credible and acceptable? Towards a design consideration framework

The convergence of Industrial Control System (ICS) with Information Technologies (IT) coupled with the resulting and widely publicized cyber security incidents have made ICS security and resilience issues of critical concern to operators and governments. The inability to apply traditional IT security practice to ICSs further complicates the challenges of effectively securing critical industrial systems. To investigate these challenges without impacting upon live system operations, testbeds are being widely used as viable options to explore, develop and assess security risks and controls. However, how an ICS testbed is designed, and its attributes, can directly impact not only on its viability but also its credibility and acceptance for use as a whole. Through a systematic review and analysis of ICS security testbed design factors, a novel outline conceptual mapping of design factors for building credibility and acceptance is proposed. These design considerations include: design objectives, implementation approach, architectural component coverage, core operational characteristics, and evaluation approach

Design Considerations for Building Credible Security Testbeds : A Systematic Study of Industrial Control System Use Cases

This paper presents a mapping framework for design factors and implementation process for building credible Industrial Control Systems (ICS) security testbeds. The resilience of ICSs has become a critical concern to operators and governments following widely publicised cyber security events. The inability to apply conventional Information Technology security practice to ICSs further compounds challenges in adequately securing critical systems. To overcome these challenges, and do so without impacting live environments, testbeds for the exploration, development and evaluation of security controls are widely used. However, how a testbed is designed and its attributes, can directly impact not only its viability but also its credibility as a whole. Through a combined systematic and thematic analysis and mapping of ICS security testbed design attributes, this paper suggests that the expertise of human experimenters, design objectives, the implementation approach, architectural coverage, core characteristics, and evaluation methods; are considerations that can help establish or enhance confidence, trustworthiness and acceptance; thus, credibility of ICS security testbeds

Modeling and simulation for cyber-physical system security research, development and applications.

This paper describes a new hybrid modeling and simulation architecture developed at Sandia for understanding and developing protections against and mitigations for cyber threats upon control systems. It first outlines the challenges to PCS security that can be addressed using these technologies. The paper then describes Virtual Control System Environments (VCSE) that use this approach and briefly discusses security research that Sandia has performed using VCSE. It closes with recommendations to the control systems security community for applying this valuable technology

A testbed to simulate cyber attacks on nuclear power plants

Nuclear power plants are critical infrastructures that must be safe and secure from undesirable intrusions: these intrusions are both physical and cyber. The increasing usage of digital control and computer systems, for supervisory control and data acquisition in the control rooms of new generation nuclear reactors, has introduced several cyber security issues that must be addressed. One of the most significant problems is that this new technology has increased the vulnerability of the nuclear power plant to cyber security threats. Furthermore, this exposed vulnerability is one of the main reasons that the transition to digital control rooms connected to enterprise network (or the internet) has been slow and hesitant. In order to address these issues and ensure that a digital control system is safe and secure from undesirable intrusions, the system must go through extensive tests and validation. These tests will verify that systems are safe and properly functioning. The vulnerabilities of a nuclear power plant can be determined through conducting cyber security exercises, cyber security attacks scenarios, and simulated attacks. All these events can be performed using the control room in the nuclear power plant, but it is a complicated and hampered process because of the complex hardware and software interactions that must be considered. Control rooms are also not ideal places to test various cyber attacks and scenarios because any mishap can lead to detrimental impacts on the nearby surroundings. This research attempts to present our approach to build a comparative testbed that captures the relevant complexity of a nuclear power plant. A testbed is developed and designed to assess the vulnerabilities that are introduced by using public networks for communications. The testbed is also used to simulate different cyber attack scenarios and it will serve to present detection mechanisms that are based on the understanding of the controlled physical system

Design Considerations for Building Credible Security Testbeds: Perspectives from Industrial Control System Use Cases

This paper presents a mapping framework for design factors and an implementation process for building credible Industrial Control Systems (ICS) security testbeds. The security and resilience of ICSs has become a critical concern to operators and governments following widely publicised cyber security events. The inability to apply conventional Information Technology security practice to ICSs further compounds challenges in adequately securing critical systems. To overcome these challenges, and do so without impacting live environments, testbeds are widely used for the exploration, development, and evaluation of security controls. However, how a testbed is designed and its attributes, can directly impact not only its viability but also its credibility. Combining systematic and thematic analysis, and the mapping of identified ICS security testbed design attributes, we propose a novel relationship map of credibility-supporting design factors (and their associated attributes) and a process implementation flow structure for ICS security testbeds. The framework and implementation process highlight the significance of demonstrating some design factors such as user/experimenter expertise, clearly defined testbed design objectives, simulation implementation approach, covered architectural components, core structural and functional characteristics covered, and evaluations to enhance confidence, trustworthiness and acceptance of ICS security testbeds as credible. These can streamline testbed requirement definition, improve design consistency and quality while reducing implementation costs

Cyber security of the smart grid: Attack exposure analysis, detection algorithms, and testbed evaluation

While smart grid technologies are deployed to help achieve improved grid resiliency and efficiency, they also present an increased dependency on cyber resources which may be vulnerable to attack. This dissertation introduces three components that provide new methods to enhancing the cyber security of the smart grid. First, a quantitative exposure analysis model is presented to assess risks inherited from the communication and computation of critical information. An attack exposure metric is then presented to provide a quantitative means to analyze the model. The metric\u27s utility is then demonstrated by analyzing smart grid environments to contrast the effectiveness of various protection mechanisms and to evaluate the impact of new cyber vulnerabilities. Second, a model-based intrusion detection system is introduced to identify attacks against electric grid substations. The system expands previous research to incorporate temporal and spatial analysis of substation control events in order to differentiate attacks from normal communications. This method also incorporates a hierarchical detection approach to improve correlation of physical system events and identify sophisticated coordinated attacks. Finally, the PowerCyber testbed is introduced as an accurate cyber-physical envi- ronment to help facilitate future smart grid cyber security research needs. The testbed implements a layered approach of control, communication, and power system layers while incorporating both industry standard components along with simulation and emulation techniques. The testbed\u27s efficacy is then evaluated by performing various cyber attacks and exploring their impact on physical grid simulations

A Survey on Industrial Control System Testbeds and Datasets for Security Research

The increasing digitization and interconnection of legacy Industrial Control Systems (ICSs) open new vulnerability surfaces, exposing such systems to malicious attackers. Furthermore, since ICSs are often employed in critical infrastructures (e.g., nuclear plants) and manufacturing companies (e.g., chemical industries), attacks can lead to devastating physical damages. In dealing with this security requirement, the research community focuses on developing new security mechanisms such as Intrusion Detection Systems (IDSs), facilitated by leveraging modern machine learning techniques. However, these algorithms require a testing platform and a considerable amount of data to be trained and tested accurately. To satisfy this prerequisite, Academia, Industry, and Government are increasingly proposing testbed (i.e., scaled-down versions of ICSs or simulations) to test the performances of the IDSs. Furthermore, to enable researchers to cross-validate security systems (e.g., security-by-design concepts or anomaly detectors), several datasets have been collected from testbeds and shared with the community. In this paper, we provide a deep and comprehensive overview of ICSs, presenting the architecture design, the employed devices, and the security protocols implemented. We then collect, compare, and describe testbeds and datasets in the literature, highlighting key challenges and design guidelines to keep in mind in the design phases. Furthermore, we enrich our work by reporting the best performing IDS algorithms tested on every dataset to create a baseline in state of the art for this field. Finally, driven by knowledge accumulated during this survey's development, we report advice and good practices on the development, the choice, and the utilization of testbeds, datasets, and IDSs

Event and Intrusion Detection Systems for Cyber-Physical Power Systems

High speed data from Wide Area Measurement Systems (WAMS) with Phasor Measurement Units (PMU) enables real and non-real time monitoring and control of power systems. The information and communication infrastructure used in WAMS efficiently transports information but introduces cyber security vulnerabilities. Adversaries may exploit such vulnerabilities to create cyber-attacks against the electric power grid. Control centers need to be updated to be resilient not only to well-known power system contingencies but also to cyber-attacks. Therefore, a combined event and intrusion detection systems (EIDS) is required that can provide precise classification for optimal response. This dissertation describes a WAMS cyber-physical power system test bed that was developed to generate datasets and perform cyber-physical power system research related to cyber-physical system vulnerabilities, cyber-attack impact studies, and machine learning algorithms for EIDS. The test bed integrates WAMS components with a Real Time Digital Simulator (RTDS) with hardware in the loop (HIL) and includes various sized power systems with a wide variety of implemented power system and cyber-attack scenarios. This work developed a novel data processing and compression method to address the WAMS big data problem. The State Tracking and Extraction Method (STEM) tracks system states from measurements and creates a compressed sequence of states for each observed scenario. Experiments showed STEM reduces data size significantly without losing key event information in the dataset that is useful to train EIDS and classify events. Two EIDS are proposed and evaluated in this dissertation. Non-Nested Generalized Exemplars (NNGE) is a rule based classifier that creates rules in the form of hyperrectangles to classify events. NNGE uses rule generalization to create a model that has high accuracy and fast classification time. Hoeffding adaptive trees (HAT) is a decision tree classifier and uses incremental learning which is suitable for data stream mining. HAT creates decision trees on the fly from limited number of instances, uses low memory, has fast evaluation time, and adapts to concept changes. The experiments showed NNGE and HAT with STEM make effective EIDS that have high classification accuracy, low false positives, low memory usage, and fast classification times

Event and Intrusion Detection Systems for Cyber-Physical Power Systems

High speed data from Wide Area Measurement Systems (WAMS) with Phasor Measurement Units (PMU) enables real and non-real time monitoring and control of power systems. The information and communication infrastructure used in WAMS efficiently transports information but introduces cyber security vulnerabilities. Adversaries may exploit such vulnerabilities to create cyber-attacks against the electric power grid. Control centers need to be updated to be resilient not only to well-known power system contingencies but also to cyber-attacks. Therefore, a combined event and intrusion detection systems (EIDS) is required that can provide precise classification for optimal response. This dissertation describes a WAMS cyber-physical power system test bed that was developed to generate datasets and perform cyber-physical power system research related to cyber-physical system vulnerabilities, cyber-attack impact studies, and machine learning algorithms for EIDS. The test bed integrates WAMS components with a Real Time Digital Simulator (RTDS) with hardware in the loop (HIL) and includes various sized power systems with a wide variety of implemented power system and cyber-attack scenarios. This work developed a novel data processing and compression method to address the WAMS big data problem. The State Tracking and Extraction Method (STEM) tracks system states from measurements and creates a compressed sequence of states for each observed scenario. Experiments showed STEM reduces data size significantly without losing key event information in the dataset that is useful to train EIDS and classify events. Two EIDS are proposed and evaluated in this dissertation. Non-Nested Generalized Exemplars (NNGE) is a rule based classifier that creates rules in the form of hyperrectangles to classify events. NNGE uses rule generalization to create a model that has high accuracy and fast classification time. Hoeffding adaptive trees (HAT) is a decision tree classifier and uses incremental learning which is suitable for data stream mining. HAT creates decision trees on the fly from limited number of instances, uses low memory, has fast evaluation time, and adapts to concept changes. The experiments showed NNGE and HAT with STEM make effective EIDS that have high classification accuracy, low false positives, low memory usage, and fast classification times

A holistic approach for measuring the survivability of SCADA systems

Supervisory Control and Data Acquisition (SCADA) systems are responsible for controlling and monitoring Industrial Control Systems (ICS) and Critical Infrastructure Systems (CIS) among others. Such systems are responsible to provide services our society relies on such as gas, electricity, and water distribution. They process our waste; manage our railways and our traffic. Nevertheless to say, they are vital for our society and any disruptions on such systems may produce from financial disasters to ultimately loss of lives. SCADA systems have evolved over the years, from standalone, proprietary solutions and closed networks into large-scale, highly distributed software systems operating over open networks such as the internet. In addition, the hardware and software utilised by SCADA systems is now, in most cases, based on COTS (Commercial Off-The-Shelf) solutions. As they evolved they became vulnerable to malicious attacks. Over the last few years there is a push from the computer security industry on adapting their security tools and techniques to address the security issues of SCADA systems. Such move is welcome however is not sufficient, otherwise successful malicious attacks on computer systems would be non-existent. We strongly believe that rather than trying to stop and detect every attack on SCADA systems it is imperative to focus on providing critical services in the presence of malicious attacks. Such motivation is similar with the concepts of survivability, a discipline integrates areas of computer science such as performance, security, fault-tolerance and reliability. In this thesis we present a new concept of survivability; Holistic survivability is an analysis framework suitable for a new era of data-driven networked systems. It extends the current view of survivability by incorporating service interdependencies as a key property and aspects of machine learning. The framework uses the formalism of probabilistic graphical models to quantify survivability and introduces new metrics and heuristics to learn and identify essential services automatically. Current definitions of survivability are often limited since they either apply performance as measurement metric or use security metrics without any survivability context. Holistic survivability addresses such issues by providing a flexible framework where performance and security metrics can be tailored to the context of survivability. In other words, by applying performance and security our work aims to support key survivability properties such as recognition and resistance. The models and metrics here introduced are applied to SCADA systems as such systems insecurity is one of the motivations of this work. We believe that the proposed work goes beyond the current status of survivability models. Holistic survivability is flexible enough to support the addition of other metrics and can be easily used with different models. Because it is based on a well-known formalism its definition and implementation are easy to grasp and to apply. Perhaps more importantly, this proposed work is aimed to a new era where data is being produced and consumed on a large-scale. Holistic survivability aims to be the catalyst to new models based on data that will provide better and more accurate insights on the survivability of systems