Uncertainty-wise Test Case Generation and Minimization for Cyber-Physical Systems

Cyber-Physical Systems (CPSs) typically operate in highly indeterminateenvironmental conditions, which require the development of testing methods that must explicitly consider uncertainty in test design, test generation, and test optimization. Towards this direction, we propose a set of uncertainty-wise test case generation and test case minimizationstrategies that rely on test ready models explicitly specifying subjective uncertainty. We propose two test case generation strategies and four test case minimizationstrategies based on the Uncertainty Theory and multi-objectivesearch. These strategies include a novel methodology for designing and introducing indeterminacy sources in the environment during test execution and a novel set of uncertainty-wise test verdicts. We performed an extensive empirical study to select the bestalgorithm out of eight commonly used multi-objective search algorithms, for each of the four minimizationstrategies, with five use cases of two industrial CPS case studies. The minimizedset of test cases obtained with the best algorithm for each minimizationstrategy were executedon the two real CPSs. The results showed that our best test strategy managed to observe 51% more uncertainties due to unknown indeterminate behaviorsof the physical environmentsof the CPSs as compared to the other test strategies. Also, the same test strategy managed to observe 118% more unknown uncertainties as compared to the unique number of known uncertainties.submittedVersio

Distributed reactive power feedback control for voltage regulation and loss minimization

We consider the problem of exploiting the microgenerators dispersed in the power distribution network in order to provide distributed reactive power compensation for power losses minimization and voltage regulation. In the proposed strategy, microgenerators are smart agents that can measure their phasorial voltage, share these data with the other agents on a cyber layer, and adjust the amount of reactive power injected into the grid, according to a feedback control law that descends from duality-based methods applied to the optimal reactive power flow problem. Convergence to the configuration of minimum losses and feasible voltages is proved analytically for both a synchronous and an asynchronous version of the algorithm, where agents update their state independently one from the other. Simulations are provided in order to illustrate the performance and the robustness of the algorithm, and the innovative feedback nature of such strategy is discussed

Applying and Extending the Delta Debugging Algorithm for Elevator Dispatching Algorithms (Experience Paper)

Elevator systems are one kind of Cyber-Physical Systems (CPSs), and as such, test cases are usually complex and long in time. This is mainly because realistic test scenarios are employed (e.g., for testing elevator dispatching algorithms, typically a full day of passengers traveling through a system of elevators is used). However, in such a context, when needing to reproduce a failure, it is of high benefit to provide the minimal test input to the software developers. This way, analyzing and trying to localize the root-cause of the failure is easier and more agile. Delta debugging has been found to be an efficient technique to reduce failure-inducing test inputs. In this paper, we enhance this technique by first monitoring the environment at which the CPS operates as well as its physical states. With the monitored information, we search for stable states of the CPS during the execution of the simulation. In a second step, we use such identified stable states to help the delta debugging algorithm isolate the failure-inducing test inputs more efficiently. We report our experience of applying our approach into an industrial elevator dispatching algorithm. An empirical evaluation carried out with real operational data from a real installation of elevators suggests that the proposed environment-wise delta debugging algorithm is between 1.3 to 1.8 times faster than the traditional delta debugging, while producing a larger reduction in the failure-inducing test inputs. The results provided by the different implemented delta debugging algorithm versions are qualitatively assessed with domain experts. This assessment provides new insights and lessons learned, such as, potential applications of the delta debugging algorithm beyond debugging

Artificial Intelligence for Resilience in Smart Grid Operations

Today, the electric power grid is transforming into a highly interconnected network of advanced technologies, equipment, and controls to enable a smarter grid. The growing complexity of smart grid requires resilient operation and control. Power system resilience is defined as the ability to harden the system against and quickly recover from high-impact, low-frequency events. The introduction of two-way flows of information and electricity in the smart grid raises concerns of cyber-physical attacks. Proliferated penetration of renewable energy sources such as solar photovoltaic (PV) and wind power introduce challenges due to the high variability and uncertainty in generation. Unintentional disruptions and power system component outages have become a threat to real-time power system operations. Recent extreme weather events and natural disasters such as hurricanes, storms, and wildfires demonstrate the importance of resilience in the power system. It is essential to find solutions to overcome these challenges in maintaining resilience in smart grid. In this dissertation, artificial intelligence (AI) based approaches have been developed to enhance resilience in smart grid. Methods for optimal automatic generation control (AGC) have been developed for multi-area multi-machine power systems. Reliable AI models have been developed for predicting solar irradiance, PV power generation, and power system frequencies. The proposed short-horizon AI prediction models ranging from few seconds to a minute plus, outperform the state-of-art persistence models. The AI prediction models have been applied to provide situational intelligence for power system operations. An enhanced tie-line bias control in a multi-area power system for variable and uncertain environments has been developed with predicted PV power and bus frequencies. A distributed and parallel security-constrained optimal power flow (SCOPF) algorithm has been developed to overcome the challenges in solving SCOPF problem for large power networks. The methods have been developed and tested on an experimental laboratory platform consisting of real-time digital simulators, hardware/software phasor measurement units, and a real-time weather station

A Tractable Fault Detection and Isolation Approach for Nonlinear Systems with Probabilistic Performance

This article presents a novel perspective along with a scalable methodology to design a fault detection and isolation (FDI) filter for high dimensional nonlinear systems. Previous approaches on FDI problems are either confined to linear systems or they are only applicable to low dimensional dynamics with specific structures. In contrast, shifting attention from the system dynamics to the disturbance inputs, we propose a relaxed design perspective to train a linear residual generator given some statistical information about the disturbance patterns. That is, we propose an optimization-based approach to robustify the filter with respect to finitely many signatures of the nonlinearity. We then invoke recent results in randomized optimization to provide theoretical guarantees for the performance of the proposed filer. Finally, motivated by a cyber-physical attack emanating from the vulnerabilities introduced by the interaction between IT infrastructure and power system, we deploy the developed theoretical results to detect such an intrusion before the functionality of the power system is disrupted

Electric Vehicle - Smart Grid Integration: Load Modeling, Scheduling, and Cyber Security

The modern world has witnessed the surge of electric vehicles (EVs) driven by government policy worldwide to reduce transportation’s dependence on fossil fuels. According to (Slowik, 2019), the global EV market has grown sharply with the annual light-duty EV sales surpassing 2 million in 2018, which is about a 70% increase from 2017. The increase in EV population implies the rise in energy demand, and that introduces new challenges to the electricity sector. EV charging load demand in high penetration scenarios, which is foreseen, may lead to stability and quality issues in power grids. Generation capacity and the electricity infrastructure upgrade may be required to address those issues; however, it increases generation costs significantly. The most common EV chargers installed today deliver around 7 kW of power, which is over four times that of an average household power consumption in the US. EV charging load often shows two peaks in a day, one in the morning when people plug in the EV at the workplace and the other in the evening when people get home from work. Without proper energy management for EV charging, the vast power demand due to a large number of plugged-in EVs can stress the electric grid, degrade the electric power quality, and impact the wholesale electricity market. Although an EV battery may store energy up to 80 kWh, which requires more than 10 hours to charge at 7kW from empty, we found that most EVs need only 12 kWh per charge or 1.7 hours at 7 kW to meet daily commute requirement while they stay in the parking garage for a more extended period. This implies that EVs can have considerable time-flexibility for charging, and it is not necessary to start chargingright after plugging in, which is likely to result in the charging power add-up. A proper EV charging schedule can well allocate the charging load to prevent power peaks. Therefore, EV charging scheduling can play a significant role in mitigating the adverse effects of vast EV charging demand without upgrading the power grid capacity.To optimize the EV charging schedule while satisfies EVs’ charging demand, each EV’s stay duration and energy need are essential parameters for the optimization. Those parameters are based on predictions to minimize human intervention. Nonetheless, the uncertainty of EV user behavior poses a challenge to the prediction accuracy. Therefore, this dissertation demonstrates an ensemble machine learning-based method to model and predict the EV loads accurately, thereby improving the performance of EV charging scheduling.On the other hand, this smart EV-grid integration, which requires massive communication, including collecting, transmitting, and distributing real-time data within the network, makes it more susceptible to cyber-physical threats. Potential breaches could not only affect grid operation but also reduce consumers’ willingness to adopting EVs over conventional fuel-powered vehicles. This dissertation also presents the vulnerability analysis and risk assessment for a smart EV charging system to develop the countermeasures to secure the network. Also, while it is inevitable that the security has flaws, this dissertation provides a novel anomaly detection approach based on the invariant correlations of different measurements within the EV charging network

Simulation-based testing of highly configurable cyber-physical systems: automation, optimization and debugging

Sistema Ziber-Fisikoek sistema ziber digitalak sistema fisikoekin uztartzen dituzte. Sistema hauen aldakortasuna handitzen ari da erabiltzaileen hainbat behar betetzeko. Ondorioz, sistema ziber-fisikoa aldakorrak edota produktu lerroak ari dira garatzen eta sistema hauek milaka edo milioika konfiguraziotan konfiguratu daitezke. Sistema ziber-fisiko aldakorren test eta balidazioa prozesua garestia da, batez ere probatu beharreko konfigurazio kopuruaren ondorioz. Konfigurazio kopuru altuak sistemaren prototipo bat erabiltzea ezinezkoa egiten du. Horregatik, sistema ziber-fisiko aldagarriak simulazio modeloak erabilita probatzen dira. Hala ere, simulazio bidez sistema ziber-fisikoak probatzea erronka izaten jarraitzen du. Hasteko, simulazio denbora altua izaten da normalki, software-az aparte, sistema fisikoa simulatu behar delako. Sistema fisiko hau normalean modelo matematiko konplexuen bitartez modelatzen da, konputazionalki garestia delarik. Jarraitzeko, sistema ziber-fisikoek ingeniaritzaren domeinu ezberdinak dituzte tartean, adibidez mekanika edo elektronika. Domeinu bakoitzak bere simulazio erremienta erabiltzen du, eta erremienta guzti hauek interkonektatzeko ko-simulazioa erabiltzen da. Nahiz eta ko-simulazioa abantaila bat izan ematen duen flexibilitateagatik, simulagailu ezberdinen erabilerak simulazio denbora handiagotzen du. Azkenik, sistema ziber-fisikoak simulaziopean probatzean, probak maila ezberdinetan egin behar dira (adb., Model, Software eta Hardware-in-the-Loop mailak), eta honek, proba-kasuak exekutatzeko denbora handitzen du. Tesi honen helburua sistema ziber-fisiko aldakorren test jardunbideak hobetzea da, horretarako automatizazio, optimizazio eta arazketa metodoak proposatzen ditu. Automatizazioari dagokionez, lehenengo, erremienta-bidezko metodologia bat proposatzen da. Metodologia hau test sistema instantziak automatikoki sortzeko gai da, test sistema hauek sistema ziber-fisiko aldagarrien konfigurazioak automatikoki probatzeko gai dira (adb., test orakuluen bitartez). Bigarren, test frogak automatikoki sortzeko planteamendu bat proposatzen da helburu anitzeko bilaketa algoritmoak erabilita. Optimizazioari dagokionez, test frogen aukeraketarako planteamendu bat eta test frogen priorizaziorako beste planteamendu bat proposatzen dira, biak bilaketa alix goritmoak erabiliz, sistema ziber-fisiko aldakorrak test maila ezberdinetan probatzeko helburuarekin. Arazketari dagokionez, “espektroan oinarritutako falten lokalizazioa” izeneko teknika bat produktu lerroen testuingurura adaptatu da, eta faltak isolatzeko metodo bat proposatzen da. Honek, falta ezberdinak lokalizatzea errezten du ez bakarrik sistema ziber-fisiko aldakorretan, baizik eta edozein produktu lerrotan non “feature model” delako modeloak erabiltzen diren aldakortasuna kudeatzeko.Los sistemas cyber-físicos (CPSs) integran tecnologías digitales con procesos físicos. La variabilidad de estos sistemas está creciendo para responder a la demanda de diferentes clientes. Como consecuencia de ello, los CPSs están volviéndose configurables e incluso líneas de producto, lo que significa que pueden ser configurados en miles y millones de configuraciones. El testeo de sistemas cyber-físicos configurables es un proceso costoso, en general debido a la cantidad de configuraciones que han de ser testeadas. El número de configuraciones a testear hace imposible el uso de un prototipo del sistema. Por ello, los sistemas CPSs configurables están siendo testeadas utilizando modelos de simulación. Sin embargo, el testeo de sistemas cyber-físicos bajo simulación sigue siendo un reto. Primero, el tiempo de simulación es normalmente largo, ya que, además del software, la capa física del CPS ha de ser testeada. Esta capa física es típicamente modelada con modelos matemáticos complejos, lo cual es computacionalmente caro. Segundo, los sistemas cyber-físicos implican el uso de diferentes dominios de la ingeniería, como por ejemplo la mecánica o la electrónica. Por ello, para interconectar diferentes herramientas de modelado y simulación hace falta el uso de la co-simulación. A pesar de que la co-simulación es una ventaja en términos de flexibilidad para los ingenieros, el uso de diferentes simuladores hace que el tiempo de simulación sea más largo. Por último, al testear sistemas cyberfísicos haciendo uso de simulación, existen diferentes niveles (p.ej., Model, Software y Hardware-in-the-Loop), lo cual incrementa el tiempo para ejecutar casos de test. Esta tesis tiene como objetivo avanzar en la práctica actual del testeo de sistemas cyber-físicos configurables, proponiendo métodos para la automatización, optimización y depuración. En cuanto a la automatización, primero, se propone una metodología soportada por una herramienta para generar automáticamente instancias de sistemas de test que permiten testear automáticamente configuraciones del sistema CPS configurable (p.ej., haciendo uso de oráculos de test). Segundo, se propone un enfoque para generación de casos de test basado en algoritmos de búsqueda multiobjetivo, los cuales generan un conjunto de casos de test. En cuanto a la optimización, se propone un enfoque para selección y otro para priorización de casos de test, ambos basados en algoritmos de búsqueda, de cara a testear eficientemente sistemas cyberfísicos configurables en diferentes niveles de test. En cuanto a la depuración, se adapta una técnica llamada “Localización de Fallos Basada en Espectro” al contexto de líneas de productos y proponemos un método de aislamiento de fallos. Esto permite localizar bugs no solo en sistemas cyber-físicos configurables sino también en cualquier línea de producto donde se utilicen modelos de características para gestionar la variabilidad.Cyber-Physical Systems (CPSs) integrate digital cyber technologies with physical processes. The variability of these systems is increasing in order to give solution to the different customers demands. As a result, CPSs are becoming configurable or even product lines, which means that they can be set into thousands or millions of configurations. Testing configurable CPSs is a time consuming process, mainly due to the large amount of configurations that need to be tested. The large amount of configurations that need to be tested makes it infeasible to use a prototype of the system. As a result, configurable CPSs are being tested using simulation. However, testing CPSs under simulation is still challenging. First, the simulation time is usually long, since apart of the software, the physical layer needs to be simulated. This physical layer is typically modeled with complex mathematical models, which is computationally very costly. Second, CPSs involve different domains, such as, mechanical and electrical. Engineers of different domains typically employ different tools for modeling their subsystems. As a result, co-simulation is being employed to interconnect different modeling and simulation tools. Despite co-simulation being an advantage in terms of engineers flexibility, the use of different simulation tools makes the simulation time longer. Lastly, when testing CPSs employing simulation, different test levels exist (i.e., Model, Software and Hardware-in-the-Loop), what increases the time for executing test cases. This thesis aims at advancing the current practice on testing configurable CPSs by proposing methods for automation, optimization and debugging. Regarding automation, first, we propose a tool supported methodology to automatically generate test system instances that permit automatically testing configurations of the configurable CPS (e.g., by employing test oracles). Second, we propose a test case generation approach based on multi-objective search algorithms that generate cost-effective test suites. As for optimization, we propose a test case selection and a test case prioritization approach, both of them based on search algorithms, to cost-effectively test configurable CPSs at different test levels. Regarding debugging, we adapt a technique named Spectrum-Based Fault Localization to the product line engineering context and propose a fault isolation method. This permits localizing bugs not only in configurable CPSs but also in any product line where feature models are employed to model variability

Uncertainty-Aware Prediction Validator in Deep Learning Models for Cyber-Physical System Data

The use of Deep learning in Cyber-Physical Systems (CPSs) is gaining popularity due to its ability to bring intelligence to CPS behaviors. However, both CPSs and deep learning have inherent uncertainty. Such uncertainty, if not handled adequately, can lead to unsafe CPS behavior. The first step toward addressing such uncertainty in deep learning is to quantify uncertainty. Hence, we propose a novel method called NIRVANA (uNcertaInty pRediction ValidAtor iN Ai) for prediction validation based on uncertainty metrics. To this end, we first employ prediction-time Dropout-based Neural Networks to quantify uncertainty in deep learning models applied to CPS data. Second, such quantified uncertainty is taken as the input to predict wrong labels using a support vector machine, with the aim of building a highly discriminating prediction validator model with uncertainty values. In addition, we investigated the relationship between uncertainty quantification and prediction performance and conducted experiments to obtain optimal dropout ratios. We conducted all the experiments with four real-world CPS datasets. Results show that uncertainty quantification is negatively correlated to prediction performance of a deep learning model of CPS data. Also, our dropout ratio adjustment approach is effective in reducing uncertainty of correct predictions while increasing uncertainty of wrong predictions.publishedVersio