MTTF Estimation Using Importance Sampling on Markov Models

Very complex systems occur nowadays quite frequently in many technological areas and they are often required to comply with high dependability standards. To study their availability and reliability characteristics, Markovian models are commonly used. Due to the size and complexity of the systems, and due to the rarity of system failures, both analytical solutions and "crude" simulation can be inefficient or even non-relevant. A number of variance reduction Monte Carlo techniques have been proposed to overcome this difficulty; importance sampling methods are among the most efficient. The objective of this paper is to survey existing importance sampling schemes, to propose some improvements and to discuss on their different properties

Techniques for the Fast Simulation of Models of Highly dependable Systems

With the ever-increasing complexity and requirements of highly dependable systems, their evaluation during design and operation is becoming more crucial. Realistic models of such systems are often not amenable to analysis using conventional analytic or numerical methods. Therefore, analysts and designers turn to simulation to evaluate these models. However, accurate estimation of dependability measures of these models requires that the simulation frequently observes system failures, which are rare events in highly dependable systems. This renders ordinary Simulation impractical for evaluating such systems. To overcome this problem, simulation techniques based on importance sampling have been developed, and are very effective in certain settings. When importance sampling works well, simulation run lengths can be reduced by several orders of magnitude when estimating transient as well as steady-state dependability measures. This paper reviews some of the importance-sampling techniques that have been developed in recent years to estimate dependability measures efficiently in Markov and nonMarkov models of highly dependable system

Estimating Discrete Markov Models From Various Incomplete Data Schemes

The parameters of a discrete stationary Markov model are transition probabilities between states. Traditionally, data consist in sequences of observed states for a given number of individuals over the whole observation period. In such a case, the estimation of transition probabilities is straightforwardly made by counting one-step moves from a given state to another. In many real-life problems, however, the inference is much more difficult as state sequences are not fully observed, namely the state of each individual is known only for some given values of the time variable. A review of the problem is given, focusing on Monte Carlo Markov Chain (MCMC) algorithms to perform Bayesian inference and evaluate posterior distributions of the transition probabilities in this missing-data framework. Leaning on the dependence between the rows of the transition matrix, an adaptive MCMC mechanism accelerating the classical Metropolis-Hastings algorithm is then proposed and empirically studied.Comment: 26 pages - preprint accepted in 20th February 2012 for publication in Computational Statistics and Data Analysis (please cite the journal's paper

Aircraft state estimation using cameras and passive radar

Multiple target tracking (MTT) is a fundamental task in many application domains. It is a difficult problem to solve in general, so applications make use of domain specific and problem-specific knowledge to approach the problem by solving subtasks separately. This work puts forward a MTT framework (MTTF) which is based on the Bayesian recursive estimator (BRE). The MTTF extends a particle filter (PF) to handle the multiple targets and adds a probabilistic graphical model (PGM) data association stage to compute the mapping from detections to trackers. The MTTF was applied to the problem of passively monitoring airspace. Two applications were built: a passive radar MTT module and a comprehensive visual object tracking (VOT) system. Both applications require a solution to the MTT problem, for which the MTTF was utilized. The VOT system performed well on real data recorded at the University of Cape Town (UCT) as part of this investigation. The system was able to detect and track aircraft flying within the region of interest (ROI). The VOT system consisted of a single camera, an image processing module, the MTTF module and an evaluation module. The world coordinate frame target localization was within ±3.2 km and these results are presented on Google Earth. The image plane target localization has an average reprojection error of ±17.3 pixels. The VOT system achieved an average area under the curve value of 0.77 for all receiver operating characteristic curves. These performance figures are typical over the ±1 hr of video recordings taken from the UCT site. The passive radar application was tested on simulated data. The MTTF module was designed to connect to an existing passive radar system developed by Peralex Electronics Pty Ltd. The MTTF module estimated the number of targets in the scene and localized them within a 2D local world Cartesian coordinate system. The investigations encompass numerous areas of research as well as practical aspects of software engineering and systems design

Investigating Performance and Reliability of Process Bus Networks for Digital Protective Relaying

To reduce the cost of complex and long copper wiring, as well as to achieve flexibility in signal communications, IEC 61850 part 9-2 proposes a process bus communication network between process level switchyard equipments, and bay level protection and control (P&C) Intelligent Electronic Devices (IEDs). After successful implementation of Ethernet networks for IEC 61850 standard part 8-1 (station bus) at several substations worldwide, major manufacturers are currently working on the development of interoperable products for the IEC 61850-9-2 based process bus. The major technical challenges for applying Ethernet networks at process level include: 1) the performance of time critical messages for protection applications; 2) impacts of process bus Ethernet networks on the reliability of substation protection systems. This work starts with the performance analysis in terms of time critical Sampled Value (SV) messages loss and/or delay over the IEC 61850-9-2 process bus networks of a typical substation. Unlike GOOSE, the SV message is not repeated several times, and therefore, there is no assurance that each SV message will be received from the process bus network at protection IEDs. Therefore, the detailed modeling of IEC 61850 based substation protection devices, communication protocols, and packet format is carried out using an industry-trusted simulation tool OPNET, to study and quantify number of SV loss and delay over the process bus. The impact of SV loss/delay on digital substation protection systems is evident, and recognized by several manufacturers. Therefore, a sample value estimation algorithm is developed in order to enhance the performance of digital substation protection functions by estimating the lost and delayed sampled values. The error of estimation is evaluated in detail considering several scenarios of power system relaying. The work is further carried out to investigate the possible impact of SV loss/delay on protection functions, and test the proposed SV estimation algorithm using the hardware setup. Therefore, a state-of-the-art process bus laboratory with the protection IEDs and merging unit playback simulator using industrial computers on the QNX hard-real-time platform, is developed for a typical IEC 61850-9-2 based process bus network. Moreover, the proposed SV estimation algorithm is implemented as a part of bus differential and transmission line distance protection IEDs, and it is tested using the developed experimental setup for various SV loss/delay scenarios and power system fault conditions. In addition to the performance analysis, this work also focuses on the reliability aspects of protection systems with process bus communication network. To study the impact of process bus communication on reliability indices of a substation protection function, the detailed reliability modeling and analysis is carried out for a typical substation layout. First of all, reliability analysis is done using Reliability Block Diagrams (RBD) considering various practical process bus architectures, as well as, time synchronization techniques. After obtaining important failure rates from the RBD, an extended Markov model is proposed to analyze the reliability indices of protection systems, such as, protection unavailability, abnormal unavailability, and loss of security. It is shown with the proposed Markov model that the implementation of sampled value estimation improves the reliability indices of a protection system

Importance Sampling Simulations of Markovian Reliability Systems using Cross Entropy

This paper reports simulation experiments, applying the cross entropy method suchas the importance sampling algorithm for efficient estimation of rare event probabilities in Markovian reliability systems. The method is compared to various failurebiasing schemes that have been proved to give estimators with bounded relativeerrors. The results from the experiments indicate a considerable improvement ofthe performance of the importance sampling estimators, where performance is mea-sured by the relative error of the estimate, by the relative error of the estimator,and by the gain of the importance sampling simulation to the normal simulation

On Age-of-Information Aware Resource Allocation for Industrial Control-Communication-Codesign

Unter dem Überbegriff Industrie 4.0 wird in der industriellen Fertigung die zunehmende Digitalisierung und Vernetzung von industriellen Maschinen und Prozessen zusammengefasst. Die drahtlose, hoch-zuverlässige, niedrig-latente Kommunikation (engl. ultra-reliable low-latency communication, URLLC) – als Bestandteil von 5G gewährleistet höchste Dienstgüten, die mit industriellen drahtgebundenen Technologien vergleichbar sind und wird deshalb als Wegbereiter von Industrie 4.0 gesehen. Entgegen diesem Trend haben eine Reihe von Arbeiten im Forschungsbereich der vernetzten Regelungssysteme (engl. networked control systems, NCS) gezeigt, dass die hohen Dienstgüten von URLLC nicht notwendigerweise erforderlich sind, um eine hohe Regelgüte zu erzielen. Das Co-Design von Kommunikation und Regelung ermöglicht eine gemeinsame Optimierung von Regelgüte und Netzwerkparametern durch die Aufweichung der Grenze zwischen Netzwerk- und Applikationsschicht. Durch diese Verschränkung wird jedoch eine fundamentale (gemeinsame) Neuentwicklung von Regelungssystemen und Kommunikationsnetzen nötig, was ein Hindernis für die Verbreitung dieses Ansatzes darstellt. Stattdessen bedient sich diese Dissertation einem Co-Design-Ansatz, der beide Domänen weiterhin eindeutig voneinander abgrenzt, aber das Informationsalter (engl. age of information, AoI) als bedeutenden Schnittstellenparameter ausnutzt. Diese Dissertation trägt dazu bei, die Echtzeitanwendungszuverlässigkeit als Folge der Überschreitung eines vorgegebenen Informationsalterschwellenwerts zu quantifizieren und fokussiert sich dabei auf den Paketverlust als Ursache. Anhand der Beispielanwendung eines fahrerlosen Transportsystems wird gezeigt, dass die zeitlich negative Korrelation von Paketfehlern, die in heutigen Systemen keine Rolle spielt, für Echtzeitanwendungen äußerst vorteilhaft ist. Mit der Annahme von schnellem Schwund als dominanter Fehlerursache auf der Luftschnittstelle werden durch zeitdiskrete Markovmodelle, die für die zwei Netzwerkarchitekturen Single-Hop und Dual-Hop präsentiert werden, Kommunikationsfehlerfolgen auf einen Applikationsfehler abgebildet. Diese Modellierung ermöglicht die analytische Ableitung von anwendungsbezogenen Zuverlässigkeitsmetriken wie die durschnittliche Dauer bis zu einem Fehler (engl. mean time to failure). Für Single-Hop-Netze wird das neuartige Ressourcenallokationsschema State-Aware Resource Allocation (SARA) entwickelt, das auf dem Informationsalter beruht und die Anwendungszuverlässigkeit im Vergleich zu statischer Multi-Konnektivität um Größenordnungen erhöht, während der Ressourcenverbrauch im Bereich von konventioneller Einzelkonnektivität bleibt. Diese Zuverlässigkeit kann auch innerhalb eines Systems von Regelanwendungen, in welchem mehrere Agenten um eine begrenzte Anzahl Ressourcen konkurrieren, statistisch garantiert werden, wenn die Anzahl der verfügbaren Ressourcen pro Agent um ca. 10 % erhöht werden. Für das Dual-Hop Szenario wird darüberhinaus ein Optimierungsverfahren vorgestellt, das eine benutzerdefinierte Kostenfunktion minimiert, die niedrige Anwendungszuverlässigkeit, hohes Informationsalter und hohen durchschnittlichen Ressourcenverbrauch bestraft und so das benutzerdefinierte optimale SARA-Schema ableitet. Diese Optimierung kann offline durchgeführt und als Look-Up-Table in der unteren Medienzugriffsschicht zukünftiger industrieller Drahtlosnetze implementiert werden.:1. Introduction 1 1.1. The Need for an Industrial Solution . . . . . . . . . . . . . . . . . . . 3 1.2. Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 2. Related Work 7 2.1. Communications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 2.2. Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 2.3. Codesign . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 2.3.1. The Need for Abstraction – Age of Information . . . . . . . . 11 2.4. Dependability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 2.5. Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 3. Deriving Proper Communications Requirements 17 3.1. Fundamentals of Control Theory . . . . . . . . . . . . . . . . . . . . 18 3.1.1. Sampling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 3.1.2. Performance Requirements . . . . . . . . . . . . . . . . . . . 21 3.1.3. Packet Losses and Delay . . . . . . . . . . . . . . . . . . . . . 22 3.2. Joint Design of Control Loop with Packet Losses . . . . . . . . . . . . 23 3.2.1. Method 1: Reduced Sampling . . . . . . . . . . . . . . . . . . 23 3.2.2. Method 2: Markov Jump Linear System . . . . . . . . . . . . . 25 3.2.3. Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 3.3. Focus Application: The AGV Use Case . . . . . . . . . . . . . . . . . . 31 3.3.1. Control Loop Model . . . . . . . . . . . . . . . . . . . . . . . 31 3.3.2. Control Performance Requirements . . . . . . . . . . . . . . . 33 3.3.3. Joint Modeling: Applying Reduced Sampling . . . . . . . . . . 34 3.3.4. Joint Modeling: Applying MJLS . . . . . . . . . . . . . . . . . 34 3.4. Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 4. Modeling Control-Communication Failures 43 4.1. Communication Assumptions . . . . . . . . . . . . . . . . . . . . . . 43 4.1.1. Small-Scale Fading as a Cause of Failure . . . . . . . . . . . . 44 4.1.2. Connectivity Models . . . . . . . . . . . . . . . . . . . . . . . 46 4.2. Failure Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 4.2.1. Single-hop network . . . . . . . . . . . . . . . . . . . . . . . . 49 4.2.2. Dual-hop network . . . . . . . . . . . . . . . . . . . . . . . . 51 4.3. Performance Metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 4.3.1. Mean Time to Failure . . . . . . . . . . . . . . . . . . . . . . . 54 4.3.2. Packet Loss Ratio . . . . . . . . . . . . . . . . . . . . . . . . . 55 4.3.3. Average Number of Assigned Channels . . . . . . . . . . . . . 57 4.3.4. Age of Information . . . . . . . . . . . . . . . . . . . . . . . . 57 4.4. Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 5. Single Hop – Single Agent 61 5.1. State-Aware Resource Allocation . . . . . . . . . . . . . . . . . . . . 61 5.2. Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 5.3. Erroneous Acknowledgments . . . . . . . . . . . . . . . . . . . . . . 67 5.4. Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70 6. Single Hop – Multiple Agents 71 6.1. Failure Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72 6.1.1. Admission Control . . . . . . . . . . . . . . . . . . . . . . . . 72 6.1.2. Transition Probabilities . . . . . . . . . . . . . . . . . . . . . . 73 6.1.3. Computational Complexity . . . . . . . . . . . . . . . . . . . 74 6.1.4. Performance Metrics . . . . . . . . . . . . . . . . . . . . . . . 75 6.2. Illustration Scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . 76 6.3. Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 6.3.1. Verification through System-Level Simulation . . . . . . . . . 78 6.3.2. Applicability on the System Level . . . . . . . . . . . . . . . . 79 6.3.3. Comparison of Admission Control Schemes . . . . . . . . . . 80 6.3.4. Impact of the Packet Loss Tolerance . . . . . . . . . . . . . . . 82 6.3.5. Impact of the Number of Agents . . . . . . . . . . . . . . . . . 84 6.3.6. Age of Information . . . . . . . . . . . . . . . . . . . . . . . . 84 6.3.7. Channel Saturation Ratio . . . . . . . . . . . . . . . . . . . . 86 6.3.8. Enforcing Full Channel Saturation . . . . . . . . . . . . . . . 86 6.4. Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88 7. Dual Hop – Single Agent 91 7.1. State-Aware Resource Allocation . . . . . . . . . . . . . . . . . . . . 91 7.2. Optimization Targets . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 7.3. Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 7.3.1. Extensive Simulation . . . . . . . . . . . . . . . . . . . . . . . 96 7.3.2. Non-Integer-Constrained Optimization . . . . . . . . . . . . . 98 7.4. Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 8. Conclusions and Outlook 105 8.1. Key Results and Conclusions . . . . . . . . . . . . . . . . . . . . . . . 105 8.2. Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107 A. DC Motor Model 111 Bibliography 113 Publications of the Author 127 List of Figures 129 List of Tables 131 List of Operators and Constants 133 List of Symbols 135 List of Acronyms 137 Curriculum Vitae 139In industrial manufacturing, Industry 4.0 refers to the ongoing convergence of the real and virtual worlds, enabled through intelligently interconnecting industrial machines and processes through information and communications technology. Ultrareliable low-latency communication (URLLC) is widely regarded as the enabling technology for Industry 4.0 due to its ability to fulfill highest quality-of-service (QoS) comparable to those of industrial wireline connections. In contrast to this trend, a range of works in the research domain of networked control systems have shown that URLLC’s supreme QoS is not necessarily required to achieve high quality-ofcontrol; the co-design of control and communication enables to jointly optimize and balance both quality-of-control parameters and network parameters through blurring the boundary between application and network layer. However, through the tight interlacing, this approach requires a fundamental (joint) redesign of both control systems and communication networks and may therefore not lead to short-term widespread adoption. Therefore, this thesis instead embraces a novel co-design approach which keeps both domains distinct but leverages the combination of control and communications by yet exploiting the age of information (AoI) as a valuable interface metric. This thesis contributes to quantifying application dependability as a consequence of exceeding a given peak AoI with the particular focus on packet losses. The beneficial influence of negative temporal packet loss correlation on control performance is demonstrated by means of the automated guided vehicle use case. Assuming small-scale fading as the dominant cause of communication failure, a series of communication failures are mapped to an application failure through discrete-time Markov models for single-hop (e.g, only uplink or downlink) and dual-hop (e.g., subsequent uplink and downlink) architectures. This enables the derivation of application-related dependability metrics such as the mean time to failure in closed form. For single-hop networks, an AoI-aware resource allocation strategy termed state-aware resource allocation (SARA) is proposed that increases the application reliability by orders of magnitude compared to static multi-connectivity while keeping the resource consumption in the range of best-effort single-connectivity. This dependability can also be statistically guaranteed on a system level – where multiple agents compete for a limited number of resources – if the provided amount of resources per agent is increased by approximately 10 %. For the dual-hop scenario, an AoI-aware resource allocation optimization is developed that minimizes a user-defined penalty function that punishes low application reliability, high AoI, and high average resource consumption. This optimization may be carried out offline and each resulting optimal SARA scheme may be implemented as a look-up table in the lower medium access control layer of future wireless industrial networks.:1. Introduction 1 1.1. The Need for an Industrial Solution . . . . . . . . . . . . . . . . . . . 3 1.2. Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 2. Related Work 7 2.1. Communications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 2.2. Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 2.3. Codesign . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 2.3.1. The Need for Abstraction – Age of Information . . . . . . . . 11 2.4. Dependability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 2.5. Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 3. Deriving Proper Communications Requirements 17 3.1. Fundamentals of Control Theory . . . . . . . . . . . . . . . . . . . . 18 3.1.1. Sampling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 3.1.2. Performance Requirements . . . . . . . . . . . . . . . . . . . 21 3.1.3. Packet Losses and Delay . . . . . . . . . . . . . . . . . . . . . 22 3.2. Joint Design of Control Loop with Packet Losses . . . . . . . . . . . . 23 3.2.1. Method 1: Reduced Sampling . . . . . . . . . . . . . . . . . . 23 3.2.2. Method 2: Markov Jump Linear System . . . . . . . . . . . . . 25 3.2.3. Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 3.3. Focus Application: The AGV Use Case . . . . . . . . . . . . . . . . . . 31 3.3.1. Control Loop Model . . . . . . . . . . . . . . . . . . . . . . . 31 3.3.2. Control Performance Requirements . . . . . . . . . . . . . . . 33 3.3.3. Joint Modeling: Applying Reduced Sampling . . . . . . . . . . 34 3.3.4. Joint Modeling: Applying MJLS . . . . . . . . . . . . . . . . . 34 3.4. Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 4. Modeling Control-Communication Failures 43 4.1. Communication Assumptions . . . . . . . . . . . . . . . . . . . . . . 43 4.1.1. Small-Scale Fading as a Cause of Failure . . . . . . . . . . . . 44 4.1.2. Connectivity Models . . . . . . . . . . . . . . . . . . . . . . . 46 4.2. Failure Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 4.2.1. Single-hop network . . . . . . . . . . . . . . . . . . . . . . . . 49 4.2.2. Dual-hop network . . . . . . . . . . . . . . . . . . . . . . . . 51 4.3. Performance Metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 4.3.1. Mean Time to Failure . . . . . . . . . . . . . . . . . . . . . . . 54 4.3.2. Packet Loss Ratio . . . . . . . . . . . . . . . . . . . . . . . . . 55 4.3.3. Average Number of Assigned Channels . . . . . . . . . . . . . 57 4.3.4. Age of Information . . . . . . . . . . . . . . . . . . . . . . . . 57 4.4. Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 5. Single Hop – Single Agent 61 5.1. State-Aware Resource Allocation . . . . . . . . . . . . . . . . . . . . 61 5.2. Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 5.3. Erroneous Acknowledgments . . . . . . . . . . . . . . . . . . . . . . 67 5.4. Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70 6. Single Hop – Multiple Agents 71 6.1. Failure Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72 6.1.1. Admission Control . . . . . . . . . . . . . . . . . . . . . . . . 72 6.1.2. Transition Probabilities . . . . . . . . . . . . . . . . . . . . . . 73 6.1.3. Computational Complexity . . . . . . . . . . . . . . . . . . . 74 6.1.4. Performance Metrics . . . . . . . . . . . . . . . . . . . . . . . 75 6.2. Illustration Scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . 76 6.3. Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 6.3.1. Verification through System-Level Simulation . . . . . . . . . 78 6.3.2. Applicability on the System Level . . . . . . . . . . . . . . . . 79 6.3.3. Comparison of Admission Control Schemes . . . . . . . . . . 80 6.3.4. Impact of the Packet Loss Tolerance . . . . . . . . . . . . . . . 82 6.3.5. Impact of the Number of Agents . . . . . . . . . . . . . . . . . 84 6.3.6. Age of Information . . . . . . . . . . . . . . . . . . . . . . . . 84 6.3.7. Channel Saturation Ratio . . . . . . . . . . . . . . . . . . . . 86 6.3.8. Enforcing Full Channel Saturation . . . . . . . . . . . . . . . 86 6.4. Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88 7. Dual Hop – Single Agent 91 7.1. State-Aware Resource Allocation . . . . . . . . . . . . . . . . . . . . 91 7.2. Optimization Targets . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 7.3. Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 7.3.1. Extensive Simulation . . . . . . . . . . . . . . . . . . . . . . . 96 7.3.2. Non-Integer-Constrained Optimization . . . . . . . . . . . . . 98 7.4. Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 8. Conclusions and Outlook 105 8.1. Key Results and Conclusions . . . . . . . . . . . . . . . . . . . . . . . 105 8.2. Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107 A. DC Motor Model 111 Bibliography 113 Publications of the Author 127 List of Figures 129 List of Tables 131 List of Operators and Constants 133 List of Symbols 135 List of Acronyms 137 Curriculum Vitae 13

Online disturbance prediction for enhanced availability in smart grids

A gradual move in the electric power industry towards Smart Grids brings new challenges to the system's efficiency and dependability. With a growing complexity and massive introduction of renewable generation, particularly at the distribution level, the number of faults and, consequently, disturbances (errors and failures) is expected to increase significantly. This threatens to compromise grid's availability as traditional, reactive management approaches may soon become insufficient. On the other hand, with grids' digitalization, real-time status data are becoming available. These data may be used to develop advanced management and control methods for a sustainable, more efficient and more dependable grid. A proactive management approach, based on the use of real-time data for predicting near-future disturbances and acting in their anticipation, has already been identified by the Smart Grid community as one of the main pillars of dependability of the future grid. The work presented in this dissertation focuses on predicting disturbances in Active Distributions Networks (ADNs) that are a part of the Smart Grid that evolves the most. These are distribution networks with high share of (renewable) distributed generation and with systems in place for real-time monitoring and control. Our main goal is to develop a methodology for proactive network management, in a sense of proactive mitigation of disturbances, and to design and implement a method for their prediction. We focus on predicting voltage sags as they are identified as one of the most frequent and severe disturbances in distribution networks. We address Smart Grid dependability in a holistic manner by considering its cyber and physical aspects. As a result, we identify Smart Grid dependability properties and develop a taxonomy of faults that contribute to better understanding of the overall dependability of the future grid. As the process of grid's digitization is still ongoing there is a general problem of a lack of data on the grid's status and especially disturbance-related data. These data are necessary to design an accurate disturbance predictor. To overcome this obstacle we introduce a concept of fault injection to simulation of power systems. We develop a framework to simulate a behavior of distribution networks in the presence of faults, and fluctuating generation and load that, alone or combined, may cause disturbances. With the framework we generate a large set of data that we use to develop and evaluate a voltage-sag disturbance predictor. To quantify how prediction and proactive mitigation of disturbances enhance availability we create an availability model of a proactive management. The model is generic and may be applied to evaluate the effect of proactive management on availability in other types of systems, and adapted for quantifying other types of properties as well. Also, we design a metric and a method for optimizing failure prediction to maximize availability with proactive approach. In our conclusion, the level of availability improvement with proactive approach is comparable to the one when using high-reliability and costly components. Following the results of the case study conducted for a 14-bus ADN, grid's availability may be improved by up to an order of magnitude if disturbances are managed proactively instead of reactively. The main results and contributions may be summarized as follows: (i) Taxonomy of faults in Smart Grid has been developed; (ii) Methodology and methods for proactive management of disturbances have been proposed; (iii) Model to quantify availability with proactive management has been developed; (iv) Simulation and fault-injection framework has been designed and implemented to generate disturbance-related data; (v) In the scope of a case study, a voltage-sag predictor, based on machine- learning classification algorithms, has been designed and the effect of proactive disturbance management on downtime and availability has been quantified

Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs