Formal Analysis and Redesign of a Neural Network-Based Aircraft Taxiing System with VerifAI

We demonstrate a unified approach to rigorous design of safety-critical autonomous systems using the VerifAI toolkit for formal analysis of AI-based systems. VerifAI provides an integrated toolchain for tasks spanning the design process, including modeling, falsification, debugging, and ML component retraining. We evaluate all of these applications in an industrial case study on an experimental autonomous aircraft taxiing system developed by Boeing, which uses a neural network to track the centerline of a runway. We define runway scenarios using the Scenic probabilistic programming language, and use them to drive tests in the X-Plane flight simulator. We first perform falsification, automatically finding environment conditions causing the system to violate its specification by deviating significantly from the centerline (or even leaving the runway entirely). Next, we use counterexample analysis to identify distinct failure cases, and confirm their root causes with specialized testing. Finally, we use the results of falsification and debugging to retrain the network, eliminating several failure cases and improving the overall performance of the closed-loop system.Comment: Full version of a CAV 2020 pape

A review of cyber-ranges and test-beds:current and future trends

Cyber situational awareness has been proven to be of value in forming a comprehensive understanding of threats and vulnerabilities within organisations, as the degree of exposure is governed by the prevailing levels of cyber-hygiene and established processes. A more accurate assessment of the security provision informs on the most vulnerable environments that necessitate more diligent management. The rapid proliferation in the automation of cyber-attacks is reducing the gap between information and operational technologies and the need to review the current levels of robustness against new sophisticated cyber-attacks, trends, technologies and mitigation countermeasures has become pressing. A deeper characterisation is also the basis with which to predict future vulnerabilities in turn guiding the most appropriate deployment technologies. Thus, refreshing established practices and the scope of the training to support the decision making of users and operators. The foundation of the training provision is the use of Cyber-Ranges (CRs) and Test-Beds (TBs), platforms/tools that help inculcate a deeper understanding of the evolution of an attack and the methodology to deploy the most impactful countermeasures to arrest breaches. In this paper, an evaluation of documented CR and TB platforms is evaluated. CRs and TBs are segmented by type, technology, threat scenarios, applications and the scope of attainable training. To enrich the analysis of documented CR and TB research and cap the study, a taxonomy is developed to provide a broader comprehension of the future of CRs and TBs. The taxonomy elaborates on the CRs/TBs dimensions, as well as, highlighting a diminishing differentiation between application areas

Development of a Cyber Range with description language for network topology definition

Cyber Ranges are an essential tool for cybersecurity trainings and experiments because they enable to setup virtual, isolated and reproducible environments that can be safely used to execute different types of tests and scenarios. The preparation of scenarios is the most time-consuming phase, which includes the configuration of the virtual machines and the definition of the network topology, so it is important for a Cyber Range to include tools that simplify this operation. This work focuses on how to implement and setup a Cyber Range that includes the necessary features and tools to simplify the setup phase, in particular for large topologies. The literature review provides an analysis of the selected open-source and research solutions currently available for Cyber Ranges and their configuration for use in different scenarios. This work presents the development of a Cyber Range based on the open-source framework OpenStack and the entire design process of a new Description Language, starting from the analysis of the requirements for the defined use-cases, defining and designing the required features, the implementation of all the required components, and the testing of the correctness and effectiveness of the whole system. A comparison of the implemented solution against the selected solutions in the literature study is provided, summarising the unique features offered by this approach. The validation of the Description Language implementation with the defined use cases demonstrated that it can reduce the complexity and length of the required template, which can help to make the setup of scenarios faster

Cyber Security of Critical Infrastructures

Critical infrastructures are vital assets for public safety, economic welfare, and the national security of countries. The vulnerabilities of critical infrastructures have increased with the widespread use of information technologies. As Critical National Infrastructures are becoming more vulnerable to cyber-attacks, their protection becomes a significant issue for organizations as well as nations. The risks to continued operations, from failing to upgrade aging infrastructure or not meeting mandated regulatory regimes, are considered highly significant, given the demonstrable impact of such circumstances. Due to the rapid increase of sophisticated cyber threats targeting critical infrastructures with significant destructive effects, the cybersecurity of critical infrastructures has become an agenda item for academics, practitioners, and policy makers. A holistic view which covers technical, policy, human, and behavioural aspects is essential to handle cyber security of critical infrastructures effectively. Moreover, the ability to attribute crimes to criminals is a vital element of avoiding impunity in cyberspace. In this book, both research and practical aspects of cyber security considerations in critical infrastructures are presented. Aligned with the interdisciplinary nature of cyber security, authors from academia, government, and industry have contributed 13 chapters. The issues that are discussed and analysed include cybersecurity training, maturity assessment frameworks, malware analysis techniques, ransomware attacks, security solutions for industrial control systems, and privacy preservation methods

Development of Economic Water Usage Sensor and Cyber-Physical Systems Co-Simulation Platform for Home Energy Saving

In this thesis, two Cyber-Physical Systems (CPS) approaches were considered to reduce residential building energy consumption. First, a flow sensor was developed for residential gas and electric storage water heaters. The sensor utilizes unique temperature changes of tank inlet and outlet pipes upon water draw to provide occupant hot water usage. Post processing of measured pipe temperature data was able to detect water draw events. Conservation of energy was applied to heater pipes to determine relative internal water flow rate based on transient temperature measurements. Correlations between calculated flow and actual flow were significant at a 95% confidence level. Using this methodology, a CPS water heater controller can activate existing residential storage water heaters according to occupant hot water demand. The second CPS approach integrated an open-source building simulation tool, EnergyPlus, into a CPS simulation platform developed by the National Institute of Standards and Technology (NIST). The NIST platform utilizes the High Level Architecture (HLA) co-simulation protocol for logical timing control and data communication. By modifying existing EnergyPlus co-simulation capabilities, NIST’s open-source platform was able to execute an uninterrupted simulation between a residential house in EnergyPlus and an externally connected thermostat controller. The developed EnergyPlus wrapper for HLA co-simulation can allow active replacement of traditional real-time data collection for building CPS development. As such, occupant sensors and simple home CPS product can allow greater residential participation in energy saving practices, saving up to 33% on home energy consumption nationally

A Nature-Inspired Approach for Scenario-Based Validation of Autonomous Systems

Scenario-based approaches are cost and time effective solutions to autonomous cyber-physical system testing to identify bugs before costly methods such as physical testing in a controlled or uncontrolled environment. Every bug in an autonomous cyber-physical system is a potential safety risk. This paper presents a scenario-based method for finding bugs and estimating boundaries of the bug profile. The method utilizes a nature-inspired approach adapting low discrepancy sampling with local search. Extensive simulations demonstrate the performance of the approach with various adaptations

Buildings-to-Grid Integration Framework

This paper puts forth a mathematical framework for Buildings-to-Grid (BtG) integration in smart cities. The framework explicitly couples power grid and building's control actions and operational decisions, and can be utilized by buildings and power grids operators to simultaneously optimize their performance. Simplified dynamics of building clusters and building-integrated power networks with algebraic equations are presented---both operating at different time-scales. A model predictive control (MPC)-based algorithm that formulates the BtG integration and accounts for the time-scale discrepancy is developed. The formulation captures dynamic and algebraic power flow constraints of power networks and is shown to be numerically advantageous. The paper analytically establishes that the BtG integration yields a reduced total system cost in comparison with decoupled designs where grid and building operators determine their controls separately. The developed framework is tested on standard power networks that include thousands of buildings modeled using industrial data. Case studies demonstrate building energy savings and significant frequency regulation, while these findings carry over in network simulations with nonlinear power flows and mismatch in building model parameters. Finally, simulations indicate that the performance does not significantly worsen when there is uncertainty in the forecasted weather and base load conditions.Comment: In Press, IEEE Transactions on Smart Gri