A Survey on Industrial Control System Testbeds and Datasets for Security Research

The increasing digitization and interconnection of legacy Industrial Control Systems (ICSs) open new vulnerability surfaces, exposing such systems to malicious attackers. Furthermore, since ICSs are often employed in critical infrastructures (e.g., nuclear plants) and manufacturing companies (e.g., chemical industries), attacks can lead to devastating physical damages. In dealing with this security requirement, the research community focuses on developing new security mechanisms such as Intrusion Detection Systems (IDSs), facilitated by leveraging modern machine learning techniques. However, these algorithms require a testing platform and a considerable amount of data to be trained and tested accurately. To satisfy this prerequisite, Academia, Industry, and Government are increasingly proposing testbed (i.e., scaled-down versions of ICSs or simulations) to test the performances of the IDSs. Furthermore, to enable researchers to cross-validate security systems (e.g., security-by-design concepts or anomaly detectors), several datasets have been collected from testbeds and shared with the community. In this paper, we provide a deep and comprehensive overview of ICSs, presenting the architecture design, the employed devices, and the security protocols implemented. We then collect, compare, and describe testbeds and datasets in the literature, highlighting key challenges and design guidelines to keep in mind in the design phases. Furthermore, we enrich our work by reporting the best performing IDS algorithms tested on every dataset to create a baseline in state of the art for this field. Finally, driven by knowledge accumulated during this survey's development, we report advice and good practices on the development, the choice, and the utilization of testbeds, datasets, and IDSs

A Framework to Support ICS Cyber Incident Response and Recovery

During the past decade there has been a steady increase in cyber attacks targeting Critical National Infrastructure. In order to better protect against an ever-expanding threat landscape, governments, standards bodies, and a plethora of industry experts have produced relevant guidance for operators in response to incidents. However, in a context where safety, reliability, and availability are key, combined with the industrial nature of operational systems, advice on the right practice remains a challenge. This is further compounded by the volume of available guidance, raising questions on where operators should start, which guidance set should be followed, and how confidence in the adopted approach can be established. In this paper, an analysis of existing guidance with a focus on cyber incident response and recovery is provided. From this, a work in progress framework is posited, to better support operators in the development of response and recovery operations

An open virtual testbed for industrial control system security research

ICS security has been a topic of scrutiny and research for several years, and many security issues are well known. However, research efforts are impeded by a lack of an open virtual industrial control system testbed for security research. This thesis describes a virtual testbed framework using Python to create discrete testbed components (including virtual devices and process simulators). This testbed is designed such that the testbeds are interoperable with real ICS devices and that the virtual testbeds can provide comparable ICS network behavior to a laboratory testbed. Two testbeds based on laboratory testbeds have been developed and have been shown to be interoperable with real industrial control systemequipment and vulnerable to attacks in the samemanner as a real system. Additionally, these testbeds have been quantitatively shown to produce traffic close to laboratory systems (within 90% similarity on most metrics)

A comprehensive dataset from a smart grid testbed for machine learning based CPS security research

Data-sets play a crucial role in advancing the research. However, getting access to real-world data becomes difficult when it comes to critical infrastructures and more so if that data is being acquired for security research. In this work, a comprehensive dataset from a real-world smart electric grid testbed is collected and shared with the research community. A few of the unique features of the dataset and testbed are highlighted